SGIP TCC End-to-End Test Template

Energy Services Provider InterfaceTesting

Version: Draft v0.1

Release Date: 2011/08/11

Page 1 of 19

SGIP TCC End-to-End Test Template

Document History

The following individuals and their companies are members of the OpenADE task force and have contributed and/or provided support to the work of the End-to-End Test Working Group End-to-End Test Case template document:

Document History

Revision History

Revision Number / Revision Date / Revision
By / Comments
.1 / 20111018 / Marty Burns, Phil Beecher / Initial seed draft

Contents

1Introduction

1.1Objective

1.2Scope

1.3Abbreviations and Definitions

1.4References

2Document Overview

3Test Use Case

3.1Summary

3.2Business Rules

3.3Assumptions

4Developing Test Requirements

4.1Test Architecture

4.1.1Test Component View

4.1.2Test Information View

4.1.3Test Security View

4.2Interoperability Functional Statements (IFS) Proforma

4.3Test Groups

4.4Test Purpose

4.5Additional Requirements

4.6Testing Context and Methods

4.6.1Testing Context and Methods

4.6.2Test Steps

4.6.3Test Sequence Diagram

5Testing Process

5.1Overview

5.2Policies and Principles

5.3Test Assessment

5.3.1Application Submission

5.3.2Test Environment Submission

5.4Test Preparation

5.4.1Test Plan

5.4.2Test Design

5.4.3Test Configuration

5.5Testing

5.6Test Results

6Feedback

6.1ITCA

6.2User Group

1Introduction

This document provides a test plan for the NAESB Energy Services Provider Interface standard. Roles and responsibilities are included in this document.

1.1Objective

The purpose of this document is to provide a set of steps required to support how energy vendors and providerscan test Smart Grid End-to-End business functions in order to achieve interoperability. The focus of this framework is testing the integration among systems and/or applications to enable Smart Grid business processes at the inter-systems level within the NIST Conceptual Model domains and sub-domains. The test requirements will be driven by business processes and the technical requirements will be driven by SGIP Smart Grid architectural standards. The end-state of this effort will contribute to the development of testing efforts required to support interoperable Smart Grid solutions as business functions, standards and technologies evolve.

The TCC End-to-End Testing Work Group has created a set of categories below for comprehensive coverage of End-to-End Testing. See figure 1.1

To help establish End-to-End interoperability, a current set of test use cases have been placed within the identified categories. Below is a current list of potential test use-cases that have a different focus, but provide a good context for establishing End-to-End Interoperability. This list below is not a complete set and will change based on additional use-cases.

  • Installation / Configuration
  • End Device installation (Meter, HAN, etc.)
  • End Device configuration and registration
  • Control / Command
  • Connect and disconnect (smart meter)
  • Demand Response Direct Load Control
  • Events
  • Notification, text messages
  • Pricing signals
  • Events and alarms (i.e Outage Events)
  • Measurement / Monitoring
  • Batch meter reading (one way)
  • On-demand meter reading (two way)
  • Energy Usage

1.2Scope

The scope of this test case document is on <…>.

  • In scope:
  • Out of scope:

1.3Abbreviations and Definitions

This section provides a list of acronyms and abbreviations required to properly interpret the document.

Acronym / Name
AMI / Advanced Metering Infrastructure
CIM / IEC TC57 Common Information Model
ESI / Energy Service Interface
HAN / Home Area Network
PCT / Programmable Communicating Thermostat
WS-I / Web Service Interoperability Organization
IUT / Implementation Under Test
QI / Qualified Implementation
MoC / Means of Communication
SUT / System Under Test
ESPI / Energy Services Provider Interface
NAESB / North American Energy Standards Board
Name / Definition
Actor / A generic name for devices, systems, or programs that make decisions and exchange information necessary for performing applications: smart meters, solar generators, and control systems represent examples of devices and systems.
AMI / Advanced Metering Infrastructure
Applications / Tasks performed by one or more actors within a domain.
CIM / IEC TC57 Common Information Model
Common Web Portal / Web interface for Regional Transmission Operator, customers, retail electric providers and transmission distribution service provider to function as a clearing house for energy information. Commonly used in deregulated markets.
Energy Service Interface / Provides security and, often, coordination functions that enable secure interactions between relevant Home Area Network Devices and the Utility. Permits applications such as remote load control, monitoring and control of distributed generation, in-home display of customer usage, reading of non-energy meters, and integration with building management systems. Also provides auditing/logging functions that record transactions to and from Home Area Networking Devices.
Home Area Network / A network of energy management devices, digital consumer electronics, signal-controlled or enabled appliances, and applications within a home environment that is on the home side of the electric meter.
Programmable Communicating Thermostat / A device within the premise that has communication capabilities and controls heating, ventilation and cooling systems.
End-to-End Abstract Test Case / A collection of abstract test cases designed to prove the ability of two (or more) systems and standards to interoperate
System Under Test / A collection of either one or more QA and an AUT
WS-I / Web Service Interoperability Organization

1.4References

References are listed below:

  • OpenSG - SGConformity
  • ETSI EG 202 237 Generic approach to interoperability testing

2Document Overview

This document provides a set of guidelinesfor developing thesteps associated with End-to-End interoperability testing. The intent of this template is to apply a framework to a set of common Smart Grid use cases and technologies that support the Smart Grid business functions from End-to-End. This document provides a set of testing guidelines to help ensure that the implemented Smart Grid systems of systems deliver the intended business functions and benefits.

Each main category to support End-to-End testing contains a set of sub steps and provides an explanation for how it can be used. Below are the four main categories used in this document.

Test Use Case:This section provides a Smart Grid common set of business requirements and processes for customers to adopt and for vendors to help implement

Test Requirements: This section is primarily aimed at developing the abstract test suites for testing devices and systems that span acrossmultiple business services of the NIST Smart Grid Conceptual Model.

Testing Process: This section provides the principles, arrangement, planning, and execution of End-to-End testing

Feed Back: This section provides a common process and format to communicate back to the SDO’s ITCA and user groups the observed events for End-to-End testing.

3Test Use Case

A test use case is summarized in this section.

3.1Summary

< This section provides a narrative that provides the end-to-end business process without specifying a specific set of standards and technologies. The starting point in the development of a test case narrative should be a set ratified requirements and specifications that support Smart Grid business functions usually in a form of a use case that has been identified and accepted by the industry.>

3.2Business Rules

< This section provides.>

3.3Assumptions

< This section provides.>

4Developing Test Requirements

This section provides the steps involved for developing End-to-End abstract test cases to support this particular test use case and help ensure End-to-End Interoperability.

4.1Test Architecture

<This section specifies three main views to understand therelevant functions, information and considerations for the End-to-End solution that will be tested.

The following bullets summarize the testing architecture to be assembled for verifying conformance to ESPI.

  • break the standard down into its basic services, data types, and Use Case scenarios (comes right out of the standard).
  • For each service, there is a set of test outcomes from success to various failures. I intend to enumerate potential failures and suggest a way to inject them.
  • Similar there are data structures that may be sound or not sound.
  • Then the Use Cases provide a defined sequence of using the services exchanging the data structures.
  • We would script the Use Cases using Selenium and the interface concept I designed with John Teeter. We would design a test coverage set of exercising the Use Cases and permutations of the success and fault injections.

4.1.1Test Component View

This section provides an abstract architecture view of logical business components, relevant interfaces, standards and supporting technologies that support the end-to-end processes documented in the test use case.

The following table shows the services that are used to implement ESPI:

Actor / Description / Logical / Physical
Data Custodian / Ability to get service status / ReadServiceStatus / ServiceStatus
Data Custodian / Initiate signed request_token request per RFC 5849 / CreateRequestToken / request_token
Data Custodian / Initiate signed authorize request per RFC 5849 / Authorize / authorize
Data Custodian / Initiate signed access_token request per RFC 5849 / CreateAccessToken / access_token
Data Custodian / Update existing authorization / NotifyUpdateAuthorization_ / Authorization
Data Custodian / Revoke existing authorization (Retail Customer) / NotifyUpdateAuthorization_ / Authorization
Data Custodian / Terminate existing authorization via service / UpdateAuthorization / Authorization
Data Custodian / Request subscription to authorized resource / CreateSubscription / Subscription (from config)
Data Custodian / Request authorized data resource(s) / RequestData / (request_token scope)
Data Custodian / Receive requested and subscribed EUI / ReadData / (dataCustodianDefaultBatchResource)
Data Custodian / Request and receive authorized EUI / ReadData_ / (request_token scope)
Third Party / Initiate callback specified in request_token per RFC 5849 / ProvideAuthorization / (request_token callback)
Third Party / Revoke existing authorization (Data Custodian) / NotifyUpdateAuthorization_ / Authorization
Third Party / Send requested and subscribed EUI / UpdateData_ / (thirdPartyDefaultBatchResource)
Third Party / Notify requested and subscribed EUI is available / NotifyData_ / (thirdPartyDefaultNotifyResource)

4.1.2Test Information View

< This section provides an information view of key information objects and interfaces that support the end-to-end processes documented in the test use case. >

ESPI Model Element / Type
(OAuth) access_token
(OAuth 2.0) expires_in
UsagePoint entry.id / URN
UsagePoint entry.title / String
UsagePoint.status / UInt8
ServiceCategory.kind / ServiceKind
ServiceKind 0
ServiceKind 1
ServiceKind 2
MeterReading entry.id / URN
MeterReadingentry.title / String
ReadingType entry.id / URN
ReadingTypeentry.title / String
ReadingType.defaultQuality / QualityOfReading
ReadingType.flowDirection / FlowDirectionType
ReadingType.intervalLength / UInt32
ReadingType.kind / KindType
ReadingType.powerOfTenMultiplier / PowerOfTenMultiplierType
ReadingType.uom / UomType
ReadingType.accumulationBehaviour / AccumulationBehaviourType
ReadingType.dataQualifier / DataQualifierType
ReadingType.tou / TOUType
ReadingType.currency / CurrencyCode
ReadingType.commodity / CommodityType
ReadingType.consumptionTier / ConsumptionTierType
ReadingType.phase / PhaseCode
IntervalBlock entry.id / URN
IntervalBlockentry.title / String
IntervalBlock.interval / DateTimeInterval
Reading.cost / UInt48
Reading.timePeriod / DateTimeInterval
Reading.value / UInt48
ReadingQuality.quality / QualityOfReading
DateTimeInterval.start / TimeType
DateTimeInterval.duration / UInt32
4.1.2.1.1Master Data Considerations

< This section provides a list of relevant technical master data objects that arekey linkages to support the end-to-end processes documented in the test use case. >

4.1.3Test Security View

< This section provides a security view of logical business components, relevant interfaces, standards and supporting technologies that support the end-to-end processes documented in the test use case. >

4.1.3.1Security Considerations

< Need some help……

4.2Interoperability Functional Statements (IFS) Proforma

This section provides a description of which features / functions of the SUT that are implemented. The starting point in the development of an IFS should be the Protocol Implementation Conformance Statement(PICS), which should identify the options and conditions which apply to the protocol to be tested. The Following are some characteristics of an IFS:

  • Format is typically a questionnaire
  • Each function / feature of the specification(s) should be clearly described with reference to the specific clause(s) in the specification(s)
  • All features / functions should be clearly described as mandatory or optional, with dependencies described

Determine mandatory and optional requirements needed for End-to-End Interoperability testing

We need template?????

4.2.1Conformity

Conformant implementations must support the following test groups.

  • Protocol
  • Fundamental
  • Batch and/or Online
  • UsagePoint and one or more information classes and associated classes
  • All attributes of implemented classes

4.2.2Conformity statements from the NAESB Standard

  • Note that not all attributes are required to be sent, but are required to be accepted

Conformant Data Custodian implementations include the following:

  • Subscriptions (Optional)
  • Accept POST to Subscription, Batch resource
  • Allow subscriptions to authorized resources
  • Delivery (Optional)
  • Accept GET to Batch resource, specific to each Authorized Third Party.
  • Support POST to Authorized Third Party Notification resource
  • Support POST to Authorized Third Party Batch resource
  • Support GET of resources directly

Conformant Third Party implementations include the following:

  • Subscriptions (Optional)
  • Submit POST requests to Subscription, Batch resource
  • Sign requests with access tokens
  • Delivery (Optional)
  • Submit GET request to Batch resource
  • Accept POST to Authorized Third Party Notification resource
  • Accept POST to Authorized Third Party Batch resource
  • GET resources directly

All conformant implementations include the following:

  • Security
  • Server certificates and mutually authenticated HTTPS
  • Send and/or accept requests to OAuth endpoints
  • Content
  • The information elements or subset thereof to be transferred are to be negotiated between parties.
  • Information elements with the meaning defined herein to be transferred use the format and structure defined herein.
  • Additional information elements not defined herein are placed in extension elements as defined by the ESPI schema herein, use a namespace different from the ESPI schema herein, and are optional.
  • It is recommended that any additional information elements included in an implementation be submitted for consideration in future versions of ESPI.

4.3Test Groups

This section provides a grouping of logical functions from specifications and IFS that can be broken into a set of test groups

4.3.1Protocol

  • HTTP
  • HTTPS

4.3.2Fundamental

  • ServiceStatus
  • Authorization
  • Request
  • Authorize
  • Callback
  • Access

4.3.3Application

  • ApplicationInformation

4.3.4Batch

  • Subscription
  • Notification
  • Deliver (Push)
  • Receive (Pull)

4.3.5Online

  • Request/Reply

4.3.6Information

  • UsagePoint
  • MeterReading
  • ElectricPowerQualitySummary
  • ElectricPowerUsageSummary

4.4Test Purpose

< This section provides.>

4.5Additional Requirements

< This section provides an add-in where additional requirements are added to support a specific set of needs by an organization. An example might include an organization that has a set of specific business functions that where additional testing is needed due a regulatory requirement. The specified Additional Requirements should not impact the end-to-end test process and can be inserted with minimal impact on the specified test requirements and test process.>

4.6Testing Context and Methods

4.6.1Testing Context and Methods

Test Context and Methods Overview

Interaction Diagram x.291 Standard

4.6.2Test Steps

< This section provides.>

Step # / Ref. Message # / Logical Components / Detailed Step / Completed / Pass/Fail / Notes

4.6.3Test Sequence Diagram

< This section provides.>

5Testing Process

5.1Overview

5.2Policies and Principles

5.3Test Assessment

< This section provides.>

5.3.1Application Submission

< This section provides.>

5.3.2Test Environment Submission

< This section provides.>

5.4Test Preparation

< This section provides.>

5.4.1Test Plan

< This section provides.>

5.4.2Test Design

< This section provides.>

5.4.3Test Configuration

< This section provides.>

5.5Testing

< This section provides.>

5.6Test Results

< This section provides.>

6Feedback

6.1ITCA

< This section provides.>

6.2User Group

< This section provides.>

Page 1 of 19