Revising the WISE SoETransitional, Coastal, and Marine waters
data collection model
- streamliningSoE data models and reporting process -
Prepared by / compiled by:
Irene del Barrio (EEA)
Marek Staroň (EEA)
Jørgen Nørrevang Jensen(ICES)
Hans Mose Jensen (ICES)
Monika Peterlin (IWRS)
Anita Künitzer (UFZ)
EEA project manager: Irene del Barrio, Marek Staroň
1
Contents
1Purpose of WISE- State of the Environment Transitional, Coastal, and Marine water data flows revision
2Why are the data needed? - Products derived from the WISE SoE TCM
3Revision background
4Revision of WISE SoE TCM (WISE-TCM and WISE-2)
4.1Revision of WISE-TCM and WISE-2
4.2Reporting monitoring sites under WISE-5
4.3Changes in code lists
4.3.1Common code list of determinands
4.4Streamlining of the data processing and quality control
4.5Country comments and questions on Data-flow revision
5Annexes
ANNEX I. Fields of WFD/WISE monitoring stations data-model
ANNEX II. New tables of the revised WISE SoE TCM data-model
1
1Purpose of WISE-State of the Environment Transitional, Coastal, and Marine water data flows revision
As announced in 2014. EEA has undertaken a review of the WISE-State of the Environment Transitional, Coastal, and Marine data flows (WISE SoE TCM data flows), also known as reporting obligations WISE-TCM and WISE-2. This review is being conductedto simplify data collection specifications in order to increase the consistency of reported data and,to reduce the reporting burden of Member Countries, and to ensure consistency with the WISE-SoE Freshwater data flows. The review has encompassed the underlying data model, and reporting requirements.
The WISE SoE TCM data flows are managed through the Eionet NRC coastal, marine, maritime network. However, the data flows overlap with similar dataflows established under Helcom and OSPAR. To avoid double reporting we have adopted a practice where data reported under the Regional Sea Convention mechanisms are incorporated into the WISE-SoE data flow. This is possible due to the open access data policy of these conventions. The revised datamodel has considered the data models usedfor the Helcom and OSPAR flows so that this practice can continue, and the changes will not affect the existing reporting obligations under the Regional Sea Conventions.
Since the agreement to incorporate data reported to Regional Sea Conventions with EEA is only set up for HELCOM & OSPAR and since there is no open access data policy for the other two RSCs - Med Sea & BSC yet, the latter two data flows from Regional Sea conventions are not taken into consideration in the revised SoE TCM data collection model.
At present, the WISE SoE TCM data flow has two components, the WISE-TCM (nutrients, chlorophyll-a, and hazardous substances) and WISE-2 (biological quality elements). Both these flows also include the position of monitoring stations. The two main changes of the new conceptual approach is to split the reporting ofobservations from the reporting ofmonitoring stations, and to combine the biological quality elements with WISE-TCM. These changes will result in a new obligation to report monitoring sites as part of the WISE-5 data flow and a revised WISE-TCM data flow that also includes biological quality elements. Furthermore, several unused tables and fields have been removed and terminology has been streamlined(Table 1).
Table 1: Data flows on State of Environment (SoE) transitional, coastal and marine data
WISE-TCM / WISE-2 / Revised WISE-TCM / WISE-5Until 2014 / Stations, nutrients, chlorophyll-a, and hazardous substances / biological quality elements / N/A / N/A
From 2016 / N/A / N/A / Nutrients, chlorophyll-a, and hazardous substances, biological quality elements / Stations
Note: N/A: not applicable
The new approach enables alignment of the spatial reference data model both with WFD reporting, with Helcomand/or OSPAR reporting, and with INSPIRE requirements.A station dictionary will be established and the observations ofnutrients, chlorophyll –a, hazardous substances and biological quality elements will be related to their geographical location via the station dictionary.
The change to report monitoring sites under WISE-5 and the revised WISE-TCM should become operational with the 2016 data call.
2Why are the data needed? -Products derived from the WISE SoE TCM
The WISE SoE TCM is the information basis to produce 3 of the EEA’s marine indicators, 2 of them being part of the EEA Core Set of Indicators (CSI):
- Nutrients in transitional, coastal and marine waters (CSI 021/MAR 005): part of the CSI, and is updated every 2 years,last published in March 2015:
- Chlorophyll in transitional, coastal and marine waters (CSI 023/MAR 006): part of the CSI, and is updated every 2 years, last published in March 2015:
- Hazardous substances in marine organisms (MAR 001): part of the marine set of indicators, and is also updated every 2 years. Last published in March 2015:
- A new indicator: status of marine invertebrate fauna in transitional and coastal waters is currently under development.
EEA also published “Waterbase - Transitional, coastal and marine waters”, which is a publicly available database including data on stations and measurements in transitional, coastal and marine waters, as well as Direct discharges and Riverine inputs:
3Revision background
EEA has been revising the freshwater WISE-SoE dataflows. As the WISE SoE TCM flows are linked this has created a need to streamline.The aim is tosimplifiethe WISE SoEdata collection specifications,to enable better harmonisation and interlinking of WISE SoEdata, and to progressive align with INSPIRE requirements.As part of the freshwater revision it was found advantageous to decouple reporting of information regarding monitoring stations from the reporting of measurements, and this has been taken forward in the revised TCM datamodel.
Furthermore, it will support the automation of the data processing phase. To this effect, quality control procedures will be automated to the extent possible. These procedures will be supported by the Reportnet QC functionalities, and other procedures and routines. This process will also increase consistency and complementarity with HELCOM and OSPAR data-flows.
4Revision of WISE SoETCM (WISE-TCM and WISE-2)
The existing data models for the WISE SoE TCM data flows WISE-TCM and WISE-2 have been revised. This revision focused on identifying:
•Key parameters (parameters used for the indicators or for quality control)
•Redundant parameters
•Poorly filled tables and fields.
•Options for merging tables (e.g. nutrients and hazardous substances in water)
•Options for deleting tables (e.g. PressuresTCM)
•Option of merging the data-flows (WISE-TCM and WISE-2) into one
To the extent possible, the terminology used in table titles and fields were streamlined with the terminology used for WFD (e.g. monitoring sites instead of monitoring stations) or the INSPIRE spatial data themes.
The existing WISE-TCM and WISE-2 data flows aredefined in the data dictionary describing the two flows.
Following a review of the existing WISE-TCM and WISE-2 data flows, it was found that a much simpler data model could be established. This could be achieved by separatingthe more permanent information related to monitoring sites from the changing information related to environmental observations (Figure 1). This data model has several benefits. It enables the establishment of a dictionary/vocabulary of monitoring sites, reducing the reporting burden once the site dictionary/vocabulary has been established. Reporting of monitoring sites can take place through the existing WISE-5 data flow.
Figure 1.Revised data-flow model for TCM and Biology in TC.
Note: In grey, the monitoring sites (WISE-5). In white, annually updated observations (Revised WISE-TCM).
Furthermore, the tabular data reported as part of WISE-TCM and in WISE-2 have been merged into the same data-model, so in the future we will only make reference to (a revised) WISE-TCM that also includes biological elements as part of the WISE-SoE TCM data flow. Several tables and fields have been removed and where needed, terminology has been made consistent.
Some countries use the option of reporting to the WISE-SoE TCM via reporting to Helcom (COMBINE dataflow) and OSPAR (CEMP dataflow). In this case, there will be no change in the dataflow. Instead the data reported to the Conventions will be transformed by the ICES data center into a structure that matches the EEA data model. The ICES DOME database will not be influenced by these changes, nor the existing ICES reporting formats.
It should be noted that parallel to the revision of the WISE data model, HELCOM is revising its data needs in relation to the HOLAS II assessment. It is very likely that HELCOM will request more information on issues in relation to the QA procedures used in the collection of data on hazardous substances in seawater, sediment and biota. These changes would relate to issues like e.g. sampling methods, analytical methods etc. To accommodate such potential changes reporting of HELCOM data to Eionet, there would be a need to add several fields to the suggested new data structure. Reporting to these fields would only be relevant for Helcom countries among Eionet in order to streamline reporting and ensure that Eionet data can also be used for Regional Sea assessments.
4.1Revision of WISE-TCM and WISE-2
As a consequence of the revision a number of changes have been made to the data model with the aim of simplifying the data flow (Figure 1):
- Measurements in water (nutrients and hazardous substances) have been combined into one table (DisaggregatedData_Water)
- Merging of TCM (water quality) and Biology data: both have been included in the same data-model, so all data will be submitted in the same file. This means there will be 4 tables related to water quality (DisaggregatedData_Water, DisaggregatedData_Sediment, DisaggregatedData_Biota and BiologyEQRData). The data must be submitted against the monitoring stations reported under the new model.
- The classification system for TC waters, in the Biology data-flow, is now considered a metadata table (BiologyEQRClassifProcedure), and will only need to be reported in the next data-call (2016), and then updated when changed.
- Discharges tables: there will be 2 tables related to discharges of nutrient and hazardous substances loads to the sea (Riverine inputs and direct discharges). The Riverine inputs will be linked to the flux stations reported as monitoring stations (Riverine inputs to be marked as MonitoringPurpose value for the monitoring site). The Direct discharges will be linked to water bodies.
- The Pressures table, the table related to Biology results at the water body level (“WaterbodyBiologyTC_Agg”), and fields that were not used or redundant have been removed.
The following changes refer exclusively to TCM elements:
- Salinity (which was an attribute in tables NutrientsSeawater_Disaggand HazSubstSeawater_Disagg) and OrganicCarbon (which was an attribute in table HazSubstSediment_Disagg), are no longer included as attributes in tables DisaggregatedData_Water and DisaggregatedData_Sediment, but are expected to be reported as Determinands (both included in the revised Determinandscodelist).
- Measurements in sediments are expected in Dry weight, while measurements in Biota are expected in Wet weight. ThereforeDryWetRatio is no longer required.
- For some of the elements exclusively in the TCM, codelists have been developed: Particle size in analysed fraction, Sampler and Species.
- Basis, for sediment data, has been removed, since only 0.9 % of the data were marked as W (wet weight basis) or L (Fat weight basis).
The new tables and elements of the data-model are compiled in the ANNEX II and the new version of the Data Dictionary is available at:
Again it should be noted that for countries opting to report via the regional Sea Conventions,Helcom and OSPAR, these changes will have no effect. The data reported to the Conventions will be transformed by the ICES data center into a structure that matches the EEA data model. The ICES DOME database will not be influenced by these changes.
4.2Reporting monitoring sites under WISE-5
The existing WISE SoE TCM data flow operates with 3 different types of stations: StationsTCM (for water quality measurements), flux stations (for riverine discharges estimations) and BiologyStations (for WFD biology measurements in transitional and coastal waters). In the revised model, monitoring sites can be reported under the WISE Spatial data flow (WISE-5). The revised model for WISE-5can be found in ANNEX I. and further information on the schema and GIS guidance can be found in the WFD reporting resources web-page[1]. The new WISE-5 model will be used for the WISE SoE TCM 2016 data-call.
For data reported directly to the EEA, it will be a requirement to report monitoring sites to WISE-5 before observations can be reported. Hence the annual reporting of observations can only take place when they have been linked to monitoring stations. For those using the option to report via regional sea conventions, there will be no changes. Instead the data reported to the Conventions will be transformed by the ICES data center into a structure that matches the EEA data model. The ICES DOME database will not be influenced by these changes, nor the existing ICES reporting formats.
4.3Changes in code lists
The revision of the data-model has included the necessary changes in the Data Dictionary and the QA/QC rules to be applied. Within these changes, there are relevant ones related to the code list of determinands, as well as the spatial reference data (mainly how to report Identifiers).
4.3.1Common code list of determinands
With regard to the determinands, the common code list developed for the freshwaters will be used. This was an outcome of the content related review[2]. The WISE SoE code list has been harmonised with the WFD code list.
The common code list includes nutrients, hazardous substances, and supportive determinands. Whenever available, CAS (Chemical Abstract Service) registry numbers are used as identifiers. If a determinant does not have a CAS number (e.g. total nitrogen) then an EEA-generated code is provided.
A separate code list is kept for biological determinands. However, currently, it only includes the biological quality elements (BQEs) that are reported using ecological quality ratios (EQR).
Other code lists were also reviewed, e.g. the Units of Measure code list which now follows the Unified Code for Units of Measure (UCUM) in order to facilitate the data exchange and avoid ambiguous representations of the units, while keeping human readability. UCUM is also used as the recommended standard representation for units of measure for Observations and Measurements (O&M) in OGC standards and INSPIRE specifications, which should facilitate its adoption also at national level.
4.4Streamlining of the data processing and quality control
An additional goal to the review process is to improve the data processing at the EEA and reduce the time and effort required to publish updated European data sets and indicators.
This goal requires the automation of various steps of the data processing phase, which is only feasible if the data being delivered conforms to the data collection model.
The quality control procedures are thus undergoing a technical review. Extended QC procedures will be implemented at Reportnet level, so that Data Providers can receive better feedback upon delivery of an envelope.
The QC procedures at Reportnet level will screen fordifferenttypes of situations:
BLOCKER
A critical error. The envelope can not be released. Normally, a blocker is an error in the format of the file, or in the structure or content of the data. Such a critical error makes it impossible for the delivery to be harvested and integrated into the European database. The envelope can only be released if every incorrect file is removed and replaced by corrected files.
ERROR
A non-critical error. The envelope can be released, but part of its content may be excluded from the European database (or be marked as having low reliability). Data Reporters are strongly advised to correct the non-critical errors. If the automated QC returned errors, a clarification or a resubmission may be requested by the Data Client, when the data is processed and the final feedback is added to the envelope. The delivery scoring evaluation may be reduced if errors are present.
WARNING
An issue that may be an error. Data Reporters are advised to check the correctness of the records or values that raised the warning. The envelope can be released. If the automated QC returned warnings, a clarification may be requested by the Data Client, when the data is processed and the final feedback is added to the envelope.
INFO
Other issues related to the quality of the data. The envelope can be released. A clarification may be requested by the Data Client, when the data is processed and the final feedback is added to the envelope.
OK
The automatic QC did not detect quality issues. The envelope can be released.
4.5Country comments and questions on Data-flow revision
Countries are requested to review the proposed changes to the WISE SoE TCM data collection model and to comment on them.
Questionsto this document and the new data model should be directed , where they will be registered and replied to. Please mark such emails as ‘Question’.
Comments to the proposed changes should as well be sent to . Please mark such emails as ‘Comment’. They will not be replied to but analysed as basis for further updating of the data model.
5Annexes
ANNEX I. Fields ofWFD/WISE monitoring stationsdata-model
Can be found here:
ANNEX II. New tables of the revised WISE SoE TCM data-model
Can be found here:
1
[1]
[2] See document “Content related SoE review- Maintenance and content development of data flows
(SoE and WFD) and its annexes”, presented in and finalised after the EIONET Freshwater Workshop 2015: