UWWTD SIIF national dissemination system
Specific contract No. 07.0307/2013/SFRA/669101/ENV.C.2
implementing Framework Service Contract No. ENV.D.2/FRA/2012/0013
Functional specifications
of
UWWTD SIIF national dissemination system -
For visualisation and
exchange of information with the EEA
Draft version 2.0
February - April 2014
The draft document was prepared by
International Office for Water (OIEau).
Date: 28 February 2014
Authors: Benoît Fribourg-Blanc, Laurence Guédet (both OIEau)
Contract Manager: Edith Hödl-Kreuzbauer (UBA)
RESTRICTION: The Functional specifications document has the following limited distribution:
External: European Commission, European Environment Agency, ETC-ICM, Pilot Member State (MS)
Internal: Contract Manager and Authors
The contents of this document are subject to copyright and all rights are reserved as defined in the framework contract under Article I.9 – use of the results. No part of this document may be reproduced, stored in a retrieval system or transmitted, in any form or by any means electronic, mechanical, photocopying, recording or otherwise, without the prior written consent of the copyright owner (Commission).
This document has been produced by OIEau.
Table of Contents
Table of Contents 3
1 Background and objectives 1
2 What do we expect from the national nodes of the UWWTD SIIF? 1
3 The functionalities of the UWWTD SIIF national node 3
3.1 The national node structure 3
3.2 The data collecting process 4
3.2.1 Upload of the data 4
3.2.2 Q & A controls 5
3.2.3 Mapping system and data model conversion 5
3.3 The reporting data process 6
3.3.1 A common parser 7
3.3.2 Process of data reporting using WFS 7
3.4 Data visualisation 7
3.4.1 Repository dataset management 8
3.4.2 Complementary dataset management 8
3.4.3 Access to national UWWTD data 9
Dynamic cartography service 9
A dynamic map (or graph) rendering system 1
Dataset download 1
WFS 2
3.5 Miscellaneous - Information web pages 2
3.6 Miscellaneous – Websites directory 2
4 The UWWTD SIIF national node organisation 3
4.1 The Web portal 3
4.2 The Web services module 3
/ Service Contract for the support to the Implementation of Directive 91/271/EECon Urban Waste Water Treatment /
Functional specifications of
UWWTD SIIF national dissemination system
1 Background and objectives
The overall objective is to assist voluntary MSs to implement the national node of the UWWTD SIIF. The objective is to establish a fully functional system for 3 pilot MSs.
Based on the specific countries’ needs, OIEau will elaborate a generic system reaching all expectations from the 3 EU-Member State (MS), for data collection, data visualization and communication at national level, and data reporting. The contractor - i.e. OIEau - will define the system using the widest perimeter for the data to be reported, to allow for an information system compatible with reporting in all EU28 MS.
The system to be designed should contain all information needed to clarify the main provisions of the Directive reporting as well as to demonstrate its implementation on the ground.
2 What do we expect from the national nodes of the UWWTD SIIF?
The basic postulate is that the national node of the UWWTD SIIF won’t be the production tool of the UWWTD data, but it will be one of the dissemination tools towards the EU platform and the European and national web users, amongst the water community.
The datasets held on the platform will be considered as a whole that is to says that the web portal won’t allow the update of the dataset items (i.e. data) but it will rather allow administrators and contributors to manage datasets (entirely) and keep tracks of their revisions (if required) except of reported dataset validated by the EU.
In this pilot exercise EU pilot MS don't have to fulfil the new SIIF UWWTD data model directly. Their official reporting database under article 15.4 and the new article 17 excel file (if they use it) will be translated to create the new dataset via specific mapping tools embedded in the platform developed by the contractor.
The main goals of the UWWTD SIIF national node to be expected are the following:
- Reporting data using the UWWTD SIIF data model with INSPIRE compliance through Web Feature Service towards the EEA platform, from 2013 data set.
2. Providing data visualisation for the national UWWWTD data, both for
§ The UWWTD reported datasets
- The available UWWTD national datasets
Depending on national users’ needs the visualisations and the access to the datasets will be done through:
- dynamic geographical maps,
- data uploads,
- WFS,
- tables
- and graphics
The web portals, to be designed in the 3 pilot MS countries, will be broadly inspired by the Greek and the French national web portals introducing their national UWWTD data (see figures below).
Figure 1: Example of the Greek national website“Sewage Treatment Plants - Database Monitoring Operation”
http://ypeka.plexscape.com/Services/Pages/Browse.aspx / Figure 2: Example of the French national website
“Information on municipal sanitation portal”
http://assainissement.developpement-durable.gouv.fr/
3 The functionalities of the UWWTD SIIF national node
3.1 The national node structure
The UWWTD SIIF national node will be composed of:
- a web portal, based on a CMS (Content Management System), it will allow the national administrators to operate and to manage it without any programming skills. The update and management of datasets, available on the web portal, will be operated through web mastering forms.
- a web services module - Geoserver or Mapserver - to operate the data reporting with the WFS protocol.
The web portal and the web service module are independent in the sense that the web service module could run without the web portal, but only if the dataset is being made available in the new model structure in a database, which is compatible with Web Feature Service (WFS) protocol. Indeed one of the functionality of the web portal is to produce, by the data mapping and Q&A control, the data in the new model structure ready to be reported through the web services module.
In the case of only the web services module is used, a parser could be set up at a central point (such as the EU platform) in order to control the structure and the content of the reported data prior to the reporting process.
The functionalities offered by the UWWTD SIIF national node, will be operated by the web portal or by the web service module. They are:
The functionalities / Operated on the web portal / Operated using the Web service moduleThe data collecting process :
· Upload of the data / X
· Q & A controls / X
· Mapping system and data model conversion / X
The reporting data process:
· Checking reported data structure and content (external parser) / X
· Process of data reporting through WFS / X
Data visualisation part:
· Repository dataset management / X
· Complementary dataset management / X
· Access to national UWWTD data
o Dynamic cartography service / X
o A dynamic map (or graph) rendering system / X
o Dataset download / X
o WFS/WMS (for other system to access datasets) / X
Miscellaneous
o Information pages / X
o Website directory / X
3.2 The data collecting process
The data collecting process is composed of those sub-processes, which will be operated on the web portal:
- the upload of the data to be reported to EU or displayed on the national web portal from a CSV file and the storage of this dataset
- A set of Q&A controls for the dataset
- A mapping system in order to convert 2013 UWWTD model based data into the new UWWTD SIIF model with the INSPIRE compliance.
Figure 3: Mapping between 2013 report tables and the new SIIF model
3.2.1 Upload of the data
In this part the national web portal administrator will be able to upload the UWWTD dataset to be reported or to be displayed from a CSV file, or both.
The datasets will have the 2013 reported data model. (
Each dataset will be described with the following Meta data
- Title of the dataset
- Date of production
- Name and organisation or the operator of this dataset
- The name of the producer of the dataset and the contact
- The type of dataset: “dataset intended for reporting” or “regular dataset”.
- Status on the web portal : “undergoing treatment and invisible to the public” or “ready and visible to all”
- INSPIRE metadata
- All the relevant statutory information (to be developed)
- All the relevant technical information (to be developed)
Availability of the functionality
This functionality is only available for the national web portal administrators and contributors. It won’t be available to anonymous users.
Prerequisites for the implementation this functionality
This functionality will be available on the web portal.
The availability of the data set for the 2013 UWWTD reporting.
3.2.2 Q & A controls
Once the dataset has been uploaded on the website, it will go through a set of control routines, in order to check its format and its compliance.
The Q&A control routines, which will be implemented on the web portal, are based on the ones already existing on the Reportnet platform.
The process of Q&A control will run till the end and will document on the screen and through a report to be printed, the errors which were found. The dataset will have to be corrected outside the web portal and its file will have to be uploaded again.
Once the dataset has gone through the entire process without any error it will be ready for data mapping, prior to the reporting process, and for data visualisation on the web portal.
Availability of the functionality
This functionality is only available for the national web portal administrators.
Prerequisites for the implementation this functionality
This functionality will be available on the web portal.
3.2.3 Mapping system and data model conversion
Once the dataset has been uploaded and it passed all the Q&A controls, the mapping will transform the data set in compliance with the INSPIRE model.
Only after the completion of this process the dataset could be made available for data reporting and displaying.
Availability of the functionality
This functionality is only available for the national web portal administrators.
Prerequisites for the implementation this functionality
This functionality will be available on the web portal.
3.3 The reporting data process
The data reporting process is operated by the web services module:
- The actual data transmission through WFS will be carried out on the side of the web portal. The web portal will do the mapping in between previous data schema and the new one, and once the data are structured correctly, the transfer will be done through web services.
The web services module can operate on a stand alone mode, without the web portal. In that case the data should be made available in a database with in the Inspire structure model. - In order to check the structure of the dataset as well as its content, in case the dataset won’t have been produced by the web portal, it is suggested to set up a parser at the European level in order to operate these controls.
Figure 4: The IT system(s) to be designed - Reporting data
3.3.1 A common parser
The parser functionalities will be to run Q&A controls on the to-be-reported dataset structure and content. This parser will operate through web services.
The parser will be set up on the European platform in order to make it available for all MS.
3.3.2 Process of data reporting using WFS
This functionality will be handled by the web services module packaged into simple executable files independent from the web portal.
The web portal will allow the administrator to manage and update the datasets to be reported and to make them available through the WFS. The trigger of the WFS in order to execute the actual reporting will be done through the web portal or manually.
On the web portal side, the web administrator will be able to update the datasets intended for reporting however once the reporting process is closed (by EU acceptance) the update of these datasets won’t be allowed and the dataset will be “frozen”.
Availability of the functionality
This functionality is only available for the national web portal administrators.
Prerequisites for the implementation this functionality
In case the web portal is not used by the MS, the data should be made available in a database with the Inspire model. The database will have to be compatible with WFS. Most of the databases are such as Oracle, Postgres, …
3.4 Data visualisation
The data visualisation part will provide the following features:
- The management of complementary data such as national repositories or additional national data sets (depending on national needs)
- The visualisation of data:
- a dynamic cartography service in order for the web user to zoom in and out in order to peruse the information from a map.
- a dynamic map (or graph) rendering system in order for the web user to generate on line from the available data, maps (or graph) as pictures in order to integrate them within reports, websites or any other documents.
- Download of any data sets available on the portal
- Make available the datasets through WFS
Figure 4: The IT system(s) to be designed - Data visualisation process
3.4.1 Repository dataset management
This functionality will allow the administrator to manage - create, update, - all the data repositories, as datasets, necessary for the display of the information, such as GIS repositories,..
Availability of the functionality
This functionality is only available for the national web portal administrators.