Data Management PlanAugust 2006
Coastal and Marine Resource Atlas
Data Management Plan
August 2006
Contents
1Introduction
1.1Intended Audience…………………………………………………………..
2Organisational Roles
3Process of Creating the Digital Atlas
3.1Identifying relevant datasets and data owners/publishers………………
3.2Standard file format and co-ordinate system for the project…………….
3.3Requesting relevant data and metadata…………………………………..
3.3.1Metadata
3.4Receiving data in a variety of file formats…………………………………
3.5Data processing and correction……………………………………………
3.6Quality assurance……………………………………………………………
3.7Producing an ArcGIS desktop project for initial visualisation…………...
3.8Loading datasets onto Internet mapping demonstration…….…………..
3.9Data owner consultation exercise…………………………………………
3.10Corrections based on consultations……………………………………...10
3.11Passing datasets and display information to MAGICTeam…………....
3.12Pre-release MAGIC consultation..………………………………………..
3.13Final data owner consultation and correcitons………………………….10
3.14Launch………………………………………………………………………
4Dissemination
4.1Defra MAGIC website……………………………………………………...
5Data Management
5.1On-going role of the MAGIC Development Team………………………
5.1.1Update frequency
5.1.2Receiving updates
5.1.3New Datasets
5.1.4Quality Assurance
Appendices
Appendix 1Datasets, data providers and contact details.
Appendix 2 Datasets grouped and ordered by Data Provider
Appendix 3Datasets that required processing.
Appendix 4Metadata forms
Appendix 5Data Supply Procedures
List of Figures
Figure 1 Flow’ diagram illustrating the steps involved in the process of constructing digital atlas…………………………………………………………….5
Figure 2 Example metadata form………………………………………………….7
Figure 3 Flow diagram’ illustrating the steps involved in the data
review process……………………………………………………………………...13
1Introduction
The Coastal and Marine Resource Atlas provides a powerful tool for the spatial understanding of ecologically sensitive sites for national contingency planning. The project is a component of the Defra MAGIC (Multi-Agency Geographic Information for the Countryside) website and will be managed by the MAGIC Development Team. It is important to recognise that the value of the system will diminish over time if information is not updated when and where required. The purpose of this Data Management Plan (DMP) is to document the data related processes that have been undertaken to compile the atlas products and specify responsibilities and processes for maintaining the data.
1.1Intended Audience
The project is aimed at specialist information professionals working in the area of coastal and marine environmental management in the private, public and voluntary sectors. Its public availability through the MAGIC website will mean that other users such as politicians, pressure groups, educators, students and the general public will be able to access the information. The site will be aimed primarily at professionals, but every effort will be made to make it useful to the wider public.
Information collected as part of this project will be of benefit to marine and coastal planners and operational staff conducting the following functions:
- Audit and planning – collating relevant datasets and allowing effective plans to be drawn up in a consistent manner over large areas.
- Operational response at time of oil spill - meeting the urgent needs of operational staff immediately on notification of a spill incident at any time of day or night.
- Operational use for clean-up operations – meeting the less time-critical needs of operational staff in managing clean up operations effectively.
- Assessment of damage and issues of responsibility – assessing the impact of a spill and producing detailed management reports.
- Sharing coastal and marine resource data more widely – effective sharing of these data to all organisations involved in coastal management.
2Organisational Roles
Data Collectors:Organisations that capture new data (for example through field survey) or process existing data to generate effectively new data. Although Data Collectors are often third party contractors, they tend to be working on behalf of other organisations that, in most cases, should be considered the true Data Collector.
Data Managers:Organisations that manage data and maintain it over time.
Data Owners:Organisations that take responsibility for owning a specific dataset – generally they are the Data Collector and Data Manager for that specific dataset. It is possible that some datasets are jointly owned by multiple organisations often where a dataset is made up of data from multiple Data Collectors.
Data Collators:Organisations that pull multiple datasets together from third party Data Collectors generally using minimal processing resources – they are often Data Publishers and Data Gatekeepers but they do not necessarily have to have these roles.
Data Publishers:Organisations that are either Data Collectors and/or Data Collators that then publish third party datasets on their behalf.
Data Gatekeepers:Organisations that act as third party Data Managers and Data Publishers on behalf of, generally, multiple other organisations. As such they have a specific role in multi-organisation data management and publishing.
Data users:Organisations that make use of the data.
These roles are obviously not mutually exclusive. The MAGIC Development Team act, for example, as Data Collators and Data Gatekeepers. Many larger organisations will act as Data Collectors, Data Managers, Data Collators, Data Publishers and Data Users.
3Process of Creating the Digital Atlas
Figure 1. ‘Flow’ diagram illustrating the steps involved in the process of constructing digital atlas.
3.1Identifying relevant datasets and data owners/publishers
Oakwood identified data providers during Phase I of the project. For a full list of datasets and data owners/publishers refer to Appendix 1.
3.2Standard file format and co-ordinate system for the project
The standard file format for this project is the ESRI shape file, used by the ESRI desktop GIS suite of software. It was agreed that data would be saved and displayed in GB National Grid (OSGB36 datum) so that it is compatible with terrestrial datasets held on the MAGIC website. This system is not ideal for displaying offshore data because it will often be outside the area for which the OSGB36 datum is designed. This leads to spatial distortions in the way the data is displayed. It was therefore agreed that offshore data would also be held by MAGIC in the ED50 datum which is appropriate for use in offshore data mapping and is employed by the offshore oil and gas industry.
3.3Requesting relevant data and metadata
At the beginning of Phase II, Oakwood formally requested data from the data owners/publishers identified during Phase I consultations. Each organisation was asked to complete and submit a metadata form with each dataset provided.
3.3.1Metadata
A simple definition of metadata is “data about data”, or more usefully, structured information about a resource. Metadata often includes information like dataset name, dataset originator, frequency of update, last update, and supplier contact. It may also have a brief description of the data. This information makes it easier to discover data that is useful to your needs. Additional information is often included that helps the user to understand how information should be used and displayed, what its limitations and accuracy are, and who has access rights to view a dataset. It will also explain codes and terminology included in the data that may not be understood by users other than the data producer.
The use of metadata is good practice when producing or storing any dataset, but for a project that involves the collection, management, archiving and dissemination of a large volume and variety of data to others it is essential. In line with best practice, each data provider was asked to submit a completed metadata form for each dataset. This metadata form was based on a standard form used by the Defra MAGIC GIS website.
Figure 2.
Example metadata form.
table 3aName / Name of the dataset
Theme / Defining characteristic
Labelling convention / Description of the geographic references used
Definition / The way in which geographic referencing object instances are defined
Domain of use / Geographic area within which the object type occurs
Owner / Name of organisation able to create and maintain
Version number / Serial number incremented by one each time a new version is created*
Version date / Date of creation of this version of the object
Parent / Name of a geographic referencing object type of which this is a sub-division
Child / Name of a geographic referencing object type which is a sub-division of this one
table 3b
Responsible authority / Name of authority responsible for capture, maintenance and supply of data set to MAGIC
Last update date / Date that the data set was last updated
Frequency of supply / How often the data set is to be supplied in an updated format to MAGIC
Version number / Serial number incremented by one each time a new version is provided for the MAGIC application*
Source Description / Source of the data representing the boundary
Scale / Basic scale of data capture
Data capture process / Process used to capture the data
Quality / Details of quality assurance regime
Symbology / Symbology used to display data set
Scale threshold minimum / Minimum scale at which the data should be displayed in MAGIC
Scale threshold maximum / Maximum scale at which the data should be displayed in MAGIC
Positional accuracy / Positional accuracy of digitised feature
Precision / Precision of co-ordinates
Measurement / Type of measurement for features
Unit of measurement / Unit of measurement used for features
Dimension / Nature of the measurement provided for each feature (area or length)
Table 4a
Element Name / Element Identifier / Element Definition
Unique reference code / Ref code / Unique reference code supplied by the Responsible Authority for each feature
Feature name / Name / Name or description of the feature
Measurement / Measure / Area or linear measurement of the feature
Designation date / Date / Date the site was designated
Hotlink / Hotlink / Hotlink to information about the attribute
Table 4b
Unique reference code / Feature name
3.4Receiving data in a variety of file formats
Mapping information was collated where available as digital files that are compatible for import into GIS. It was confirmed during the Phase I consultations that most information would be GIS compatible.
Where possible, data owners were asked to supply data in the project standard ESRI GIS shapefile format. However, data was supplied to Oakwood in a variety of other file formats. One of these was MapInfo GIS, which, like ESRI GIS, is widely used by industry and government departments. Spreadsheets and databases were also widely used to store point data and these are also GIS compatible provided information is georeferenced using a suitably accurate longitude and latitude coordinate or northing and easting, and provided the datum to which the co-ordinates referred was known.
Some information was only available in hard copy or paper maps and required digitising by Oakwood.
3.5Data processing and correction
One of the key tasks carried out during data compilation was to convert data, received in a variety of forms, into a common format. MapInfo GIS files can be easily imported into ESRI ArcGIS (see Appendix 5) as can spreadsheets and database files provided the information is geographically referenced. Some time and effort was often required to change the layout of spreadsheets and databases to be ‘cleaner’, ready for import into GIS. It was agreed that where data was only available in hard copy or paper maps it would be digitised by Oakwood using in-house digitising tablets or using ‘on-screen’ digitising techniques. One dataset produced through digitising was standing approval areas for the use of dispersants.
The most common data processing issue was the use by data owners of multiple coordinate systems – many datasets were delivered in latitude and longitude coordinates using different datums. These were generally converted to GB National Grid (including marine datasets) for consistency. It may not be practical to insist on all Data Publishers supporting GB National Grid. Given the ease with which ArcGIS deals with multiple coordinate systems, it may be better to accept that some conversion overhead may be required in a number of cases, particularly for marine-based datasets.
Appendix 3 identifies those datasets where some degree of processing was required and the approach taken.
3.6Quality assurance
When Oakwood received datasets from data owner/publishers, they were visualised using MapInfo GIS. Often it was possible to open files directly into the GIS viewing package following small modifications to the file format or a projection conversion. It was not possible to properly quality assure data supplied with coordinates in a text, spreadsheet or database file, so these required processing so that the data could be viewed in a mapped GIS format. The data was checked for gaps in geographical coverage, spatial resolution and accuracy, and projection problems. The attribute database for each GIS layer was also checked to ensure that it contained useful and relevant information about the features mapped. Where information about the same subject or theme came from different data providers and for different areas, the datasets were also checked for consistency in terms of data collection methods, category definitions and spatial resolution. The datasets were then passed on to GeoWise for further investigation and formatting.
3.7Producing an ArcGIS desktop project for initial visualisation
All data supplied was assessed and a table produced of potential work required on a dataset-by-dataset basis. The datasets were incorporated into an ArcGIS project where they were re-checked visually for spatial discrepancies and attribute data was again checked for completeness. Within this ArcGIS project, further data processing was performed where necessary and decisions were made about how to display the different information themes such as the use of appropriate colours and symbols.
3.8Loading datasets onto Internet mapping demonstration
A demonstration application was set up to deliver the datasets online using ESRI ArcIMS software. Datasets were loaded into this demo and grouped into appropriate themes.Oakwood undertook a detailed evaluation of the datasets using this online demonstrator to identify any outstanding problems and, in particularly, to assess whether the data was presented appropriately.
3.9Data owner consultation exercise
Geowise distributed the URL of the Internet mapping demonstration website to data publishers and project partners and requested feedback by the 25th February 2005. Recipients were asked to consider the following issues, bearing in mind that the aim was to present the data in a way that would allow a non-specialist to interpret the data without ambiguity:
- Is the data correct?
- Is the data presented appropriately and compatible with other maps e.g. correct name and key, colour/shading, symbology, displayed at an appropriate scale?
- Are there any gaps or inconsistencies? If so, are there any other related datasets to fill such gaps?
- Any other comments?
3.10Corrections based on consultation
Feedback from this consultation was actioned by modification of the ArcIMS project and, where necessary, the underlying shape files.
3.11Passing datasets and display information to MAGIC Team
Once finalised, datasets were issued to DEFRA. In reality, and due to time constraints, this process was undertaken in a series of tranches.
3.12Pre-release MAGIC consultation
A demonstration site was set up by the DEFRA GI Team and a URL circulated again to project stakeholders allowing them to review the final project within the MAGIC application infrastructure.
3.13Final data owner consultation and corrections
Where possible, any feedback from the consultation was incorporated into the application.
3.14Launch
The Coastal and Marine Resource Atlaswas formally launched at the Coastal Futures conference, London in January 2006.
4Dissemination
4.1Defra MAGIC website
MAGIC is the first web-based interactive map to bring together information on key environmental schemes and designations in one place. MAGIC is a partnership project involving seven government organisations that have responsibilities for rural policy-making and management, and although it has been designed to meet the needs of the partner organisations, the facility is available to anyone over the Internet.
MAGIC makes use of standard GIS tools to allow people to view and query the available data. Users do not require specialist software and can access maps using a standard web browser. The information available in MAGIC has expanded considerably since its launch in July 2002. Originally the interactive map was designed to show only datasets for England as this was the area common to all the partners. Since the launch, feedback has demonstrated that information for Scotland, Wales and marine areas is also required, so MAGIC is widening its geographic extent with the purpose of meeting the needs of MAGIC partners with GB-wide interests (MAGIC website,
The MAGIC website is an ideal host for the coastal and marine resource atlas. It is already used by environmental professionals, has an expert data management team in place, and conforms to best practice in storing and disseminating geographical information. The inclusion of this project will also help MAGIC to fulfil the requirement for English, Scottish and Welsh marine data.
5Data Management
Upon completion of the digital atlas, Oakwood and GeoWise passed responsibility for data management to Defra’s Geographical Information (GI) Team which supports the existing MAGIC project. The MAGIC team will undertake the on-going management of the application and all its component data sets together with associated metadata information through a file-based central data store. The partnership with the MAGIC Development Team demonstrates an innovative and welcome approach to project delivery and ensures that: (1) new systems are not procured for new applications thereby reducing duplication and significantly cutting costs; and (2) the project is sustainable and data will be maintained.
The process by which Oakwood and GeoWise Ltd compiled the digital atlas is described and illustrated in Figure 1 and Section 3. It is useful to appreciate this process, not least because many of the same tasks will need to be preformed in order to effectively maintain and update the application. The process of updating the atlas is illustrated in Figure 3.