Office of Enforcement and Compliance Assurance

Office of Enforcement and Compliance Assurance

Integrated Compliance Information System

ICIS Batch

Permitted Feature –Technical Specification

Version 1.7

Revised Final

November 3, 2017

- 1

Office of Enforcement and Compliance Assurance

Document Change History

Version Number / Date / Description
0.1 / August 10, 2009 / Initial Draft Release.
1.0 / September 4, 2009 / •Incorporated comments provided by EPA.
•Updated all process flow tables to include details of the specific functions needed to perform the transaction.
•Corrected typo in the Change Permitted Feature Processing flow table, Step 3A.
•Removed business rule #1, which stated the valid transaction types. This rule was not needed because all five transaction types are valid for Permitted Features.
•Removed business rule #21. A module-generic form of this rule was added to the Contacts and Addresses attachment.
1.1 / September 21, 2009 / •Updated the Change and Replaces flows and supporting tables to set Reach ID and Water Body Name to blank if Latitude and Longitude are also blank, per CR 447.
•Added text to Section 2.6 listing the fields that should not automatically be blanked out on a Replace transaction because they cannot be submitted through batch, per CR 448.
1.2 / October 22, 2009 / •Updated flow chart supporting tables to show the correct permissions needed to complete each type of transaction, per CR 477.
1.3 / July 13, 2010 / •Removed Business Rule 7 (PF200) due to CR505 which prevented a Mass Delete from removing a Permitted Feature if any of its Limit Setshave Enforcement Action Limits
•Updated Mass Delete flow, supporting table, and examples to reflect change due to CR505
•Added BR 20, 21. These are web rules that have to be enforced via Batch
•Added Notes to BR3 and BR4 to not check for Replace transactions where the Permitted Feature exists
•Updated XML Schema Section for consistency across technical specifications
•Changed Row ID to ID in Business Rule Table for consistency across Technical Specifications
1.4 / February 22, 2011 / •Updated text for describing Processing Order in Section 2: Validation and Processing
1.5 / September 9, 2013 / •Updated for CR OMCR 283: Update ICIS to support NeT MSGP
1.6 / February 18, 2017 / •EP-1127 – Add CGP EDT-only data elements
•Updated Section 2.1.2 to add new multi-value items
•Added BR 22, 23 for new CGP data elements and mapped to processing flows
•Added data element mapping for new CGP data elements
•Updated Table A-1: Acronym List
1.7 / November 3,2017 / •Epic EP-1562 – GMG Offshore Oil and Gas Permit Application data elements
•Added data element mapping for new GMG data elements
•Updated Table A-1: Acronym List
Note: The code changes for GMG Data were released to production in ICIS 7.7.11, but, per EPA’s request, the updated ICIS XML Schema was not deployed to production.

Table of Contents

1. Introduction

1.1 Purpose

1.2 Assumptions and Constraints

1.3 Document Overview

2. Validation and Processing

2.1 General Batch Processing Rules

2.1.1 Asterisks

2.1.2 Multi-Value Items

2.2 Mass Delete (X) Permitted Feature Processing

2.2.1 Mass Delete Permitted Feature Processing Flow

2.3 Delete (D) Permitted Feature Processing

2.3.1 Delete Permitted Feature Processing Flow

2.4 New (N) Permitted Feature Processing

2.4.1 New Permitted Feature Processing Flow

2.5 Change (C) Permitted Feature Processing

2.5.1 Change Permitted Feature Processing Flow

2.6 Replace (R) Permitted Feature Processing

2.6.1 Replace Permitted Feature Processing Flow

2.7 Business Rules

3. Data Element Mapping

4. XML Schema

Appendix A: Acronyms

List of Tables

Table 21: Adding to a Multi-Value Item List

Table 22: Changing to a Multi-Value Item List

Table 23: Deleting from a Multi-Value Item List

Table 24: Mass Delete Permitted Feature Processing

Table 25: Mass Delete Permitted Feature Example 1

Table 26: Mass Delete Permitted Feature Example 2

Table 27: Delete Permitted Feature Processing

Table 28: Delete Permitted Feature Example 1

Table 29: Delete Permitted Feature Example 2

Table 210: New Permitted Feature Processing

Table 211: New Permitted Feature Example 1

Table 212: New Permitted Feature Example 2

Table 213: Change Permitted Feature Processing

Table 214: Change Permitted Feature Example 1

Table 215: Change Permitted Feature Example 2

Table 216: Change Permitted Feature Example 3

Table 217: Replace Permitted Feature Processing

Table 218: Replace Permitted Feature Example 1

Table 219: Replace Permitted Feature Example 2

Table 220: Replace Permitted Feature Example 3

Table 221: Replace Permitted Feature Example 4

Table 222: Batch Permitted Features Business Rules

Table 31: Batch Permitted Feature Data Element Mapping

Table A1: Acronym List

List of Figures

Figure 21: Mass Delete Permitted Feature Processing Flow

Figure 22: Delete Permitted Feature Processing Flow

Figure 23: New Permitted Feature Processing Flow

Figure 24: Change Permitted Feature Processing Flow

Figure 25: Replace Permitted Feature Processing Flow

ICIS Batch – Permitted Feature Technical Specification, Version 1.7

Office of Enforcement and Compliance Assurance

1. Introduction

Many states already have their own tracking systems for National Pollutant Discharge Elimination System (NPDES) data. When those states are added to the Integrated Compliance Information System (ICIS), having them enter these data in ICIS through the Web interface would require them to perform duplicative data entry. Instead, users from these states may enter data in ICIS through batch submissions by which they extract data from their state system and submit it to ICIS in Extensible Markup Language (XML) files. The type of data a state submits through batch depends on the type of data tracked in their state system.

Current users of ICIS include Headquarters, Regions, direct-user states and hybrid states using Batch Discharge Monitoring Report (DMR) functionality. The remaining batch modules are being added in groups, and the Permitted Feature module is part of Full Batch Phase 1. The focus of this technical specification is the submission of Permitted Feature data to ICIS through batch XML transactions.

The general process for states to submit batch data was defined for batch DMR processing and remains unchanged. States do not submit batch data directly to ICIS. Instead, batch files are submitted to the Environmental Protection Agency’s (EPA’s) Central Data Exchange (CDX) which then passes the files to ICIS;states do not submit batch data directly to ICIS. To submit data to CDX, the state must have a CDX User ID and password. This ID and password are specific to CDX and are completely unrelated to ICIS. An ICIS User ID must also be provided in the ID tag in the header of each XML file so that when ICIS receives the batch file(s), it can determine if the transactions in the file can be performed by the user submitting the batch. After receiving a batch from the state, CDX performs several important functions. They perform a virus scan to ensure that the state files are free of viruses and then assign a unique Transaction ID to the batch. This Transaction ID maps directly to the Batch ID that ICIS uses internally to manage processing. ICIS uses this Transaction ID to communicate information about the batch to CDX and the state. CDX then archives the batch and validates the XML files based on the rules in the XML schema. If problems are detected, CDX notifies the state so that the problems can be corrected. Upon completion of these tasks, CDX sends the error-free batches to ICIS.

For purposes of this document:

•“CDX User ID” refers to the ID the user must have to log in to CDX.

•“ICIS User ID” refers to the state user’s ID in the ICIS system.

•“Transaction ID” refers to the identifier CDX provides for each batch.

•“Batch ID” in all communications with users (e.g., audit reports, batch processing confirmation report) refers to the identifier CDX provides for each batch (i.e., the Transaction ID).

•“Batch ID” in the ICIS Batch Operational Database refers to the batch identifier assigned by ICIS to make processing more efficient.

A batch may contain many XML files, and within each XML file there canbeup to one Payload for each Submission Type (e.g., Permitted Feature). Each Payload may contain one or many XML transactions, each of which contains the Permitted Feature data and a specific transaction type that identifies how ICIS should process the data. For Permitted Features there are five valid Permitted Feature XML transaction types: Mass Delete, Delete, New, Change, and Replace. The details of these Permitted Feature XML transaction types are described in Section 2: Validation and Processing. After receiving a batch from CDX, ICIS parses it and saves each Permitted Feature XML transaction as a Payload so that the individual Permitted Feature XML transactions can be ordered and processed. After processing is complete for all files in a batch, ICIS sends a response notification to CDX, which then notifies the state when processing is complete.

ICIS contains Permitted Feature (e.g., pipe) data defined within a Permit. Batch Permitted Feature functionality allows users to update the Permitted Features for a permit. A Permitted Feature is uniquely identified in ICIS by the combination of NPDES ID and Permitted Feature ID.

1.1 Purpose

The purpose of this document is to provide a comprehensive overview of the submission of Permitted Feature data through batch XML transactions using text descriptions, tables, and figures.

A major component of this Permitted Feature Technical Specification, Section 2: Validation and Processing, details the five Permitted Feature XML transaction types: Mass Delete, Delete, New, Change, and Replace. Provided with these transactions are the business rules that govern batch Permitted Feature transactions, as well as the accompanying error/warning messages, serving to notify users of the data in error and provide them with the information necessary to correct the problems.

1.2 Assumptions and Constraints

The following assumptions and constraints apply to the ICIS Batch Permitted Feature Technical Specification:

•ICIS will process batches within a state in the order they are received from CDX. CDX will not apply a timestamp to each batch that is submitted by the state. In addition, CDX cannot guarantee that batches will be sent to ICIS in the same order that CDX received them from the state. As a result, ICIS cannot guarantee that batches will be processed in the order in which the state submitted them.

•Users will submit batch files to CDX in the correct chronological order. A procedure will be put in place by each state to ensure that a state sends only one batch at a time, and does not send a new batch until they have received confirmation that the previous batch has been processed.

•CDX will perform a schema validation on every batch. ICIS will not perform another schema validation. If schema errors exist that are not caught by the CDX validation, unexpected results will occur.

•The business rules for Permitted Features entered via batch should be the same as Permitted Features entered via the web application. Any differences will be noted in the “Where Enforced (Web)” and “Where Enforced (Batch)” Columns of the Business Rules table in Section 2: Validation and Processing.

•Design decisions will be made to minimize software changes that will be needed to incorporate the batch entry of other data families. EPA will be consulted, as appropriate, as these decisions are made.

•ICIS will not save a transaction if there are any errors within the transaction, though transactions will be saved if only warnings exist. The rules for accepting and rejecting transactions are described in detail in Section 2: Validation and Processing.

1.3 Document Overview

The following sections comprise the remainder of this technical specification:

•Section 2: Validation and Processing – This section describes the processing of Mass Delete, Delete, New, Change, and Replace Permitted Feature transactions, and the business rules that apply to each transaction type.

•Section 3: Data Element Mapping – This section provides a mapping between the Permit Compliance System (PCS) Acronym, XML Tag Name, ICIS Screen Name, and ICIS Database Name for Permitted Feature data elements.

•Section 4: XML Schema – This section provides a list of the XML schemas related to Permitted Features.

•Appendix A: Acronyms – This section provides a list of all acronyms used in the document.

•Attachment A:Contacts and Addresses – This attachment provides the Business Rules and Data Element Mapping for the Contacts & Addresses data.

2. Validation and Processing

After receiving a batch from CDX, ICIS parses the batch into individual Permitted Feature XML transactions, saves each as a Payload, and groups them by type (Replace, New, Change, Delete, and Mass Delete). ICIS must process these groups in the proper order to achieve the desired results. The ICIS Batch Design Document Appendix D: ICIS Batch Submission Types and Processing Order details the processing order for all ICIS Full Batch Submissions. This section describes the detailed processing of the five Permitted Feature transaction types.

The following sub-sections describe general processing rules related to asterisks and multi-value items and the detailed processing of Permitted Feature XML transactions.

2.1 General Batch Processing Rules

It is important to understand the following general batch processing rules for ICIS:

2.1.1 Asterisks

Users must have the ability to blank out data fields through batch transactions. This is accomplished through the use of asterisks (*). In a Change transaction, an asterisk in a tag is used to indicate that a field should be blanked out. The asterisk is not stored in ICIS, but instead is used to signal the system that the field should be blanked out. Asterisks can also be submitted in New and Replace transactions. When that happens, they are interpreted by the system as blanks. It is not necessary for the user to submit asterisks in New and Replace transactions because leaving the tag out of the XML transaction achieves the same result (i.e., the field is left blank), but ICIS has been designed to process asterisks in those cases.

2.1.2 Multi-Value Items

Data fields or sections for which multiple values can be entered are referred to in this technical specification as multi-value items. Whenever tags for one of these multi-value items are included in a payload, ICIS will replace all existing values for that item with the values submitted in the tag(s), regardless of the transaction type submitted. There are two categories of multi-value items:

Individual Data Tag – The simplest kind of multi-value item consists of one data tag that can be repeated multiple times within a transaction. Below is an example of a Permitted Feature Individual Data Tagmulti-value item (PermittedFeatureCharacteristics) repeated multiple times:

Multiple Data Tags – Some multi-value items contain multiple data tags which can be repeated, as a group, multiple times within a transaction. Below is an example of a Permitted Feature Multiple Data Tag multi-value item (Contact – a subsection of SiteOwnerContact) repeated multiple times:

<MiddleName>William</MiddleName>

<IndividualTitleText>Chief Executive Officer</IndividualTitleText>

</Contact>

<LastName>Smith</LastName>

<IndividualTitleText>Chief Financial Officer</IndividualTitleText>

</Contact>

</SiteOwnerContact>

If data already exist for one of these multi-value items for the Permitted Feature in ICIS and the user wishes to add new values while keeping the existing values, they would include all of the values that they wish to have for the field (i.e., all existing values, plus the new values) in their XML submission. The table below provides an example.

Table 21: Adding to a Multi-Value Item List

PermittedFeatureCharacteristics in ICIS Database (DB) / PermittedFeatureCharacteristics in XML Submission / Result in ICIS DB After Processing
SW / SW / SW
GRW / GRW / GRW
DEW / DEW

If data already exist for one of these multi-value items for the Permitted Feature in ICIS and the user wishes to change one of the values, they would include all of the values that they wish to have for the field (i.e., existing values they wish to keep, plus new values) in their XML submission. The table below provides an example.

Table 22: Changing to a Multi-Value Item List

PermittedFeatureCharacteristics in ICIS DB / PermittedFeatureCharacteristics in XML Submission / Result in ICIS DB After Processing
SW / SW / SW
GRW
DEW / DEW

If data already exist for one of these multi-value items for the Permitted Feature in ICIS and the user wishes to remove one of the values, they would include all of the values that they wish to have for the field (i.e., existing values they wish to keep) in their XML submission. The table below provides an example.

Table 23: Deleting from a Multi-Value Item List

PermittedFeatureCharacteristics in ICIS DB / PermittedFeatureCharacteristics in XML Submission / Result in ICIS DB After Processing
SW / SW / SW
GRW
DEW

Users must also have the ability to blank out all values for these multi-value items. This is accomplished by submitting one row of the multi-value item with an asterisk as the value. The rules for doing this are described below:

•Individual Data Tag: If an asterisk is submitted in an individual data tag, ICIS will blank out all existing values for the corresponding field.

•Multiple Data Tags:

If asterisks are submitted in all required tags and the optional tags are not included, ICIS will blank out all existing values for the corresponding multi-value item.