Rational Software
ClassicsCD.com
Measurement Plan
Version <1.0>
ClassicsCD.com / Version: <1.0>ClassicsCD.com Measurement Plan / Date: 05August 2002
<document identifier>
Revision History
Date / Version / Description / Author05 August 2002 / 1.0 / Initial Version / Lou <Project Manager>
Table of Contents
1.Introduction
1.1Purpose
1.2Scope
1.3Definitions, Acronyms, and Abbreviations
1.3.1ACWP
1.3.2Base measure
1.3.3BCWP
1.3.4BCWS
1.3.5CPI
1.3.6CV
1.3.7Derived measure
1.3.8Information need
1.3.9Measurable concept
1.3.10Measurable construct
1.3.11Measure (noun)
1.3.12Measure (verb)
1.3.13Measurement
1.3.14SPI
1.3.15SV
1.4References
1.5Overview
2.Management Goals and Subgoals
3.Measures
3.1Schedule and Progress Measures
3.1.1Project Tasks Complete
3.1.2Late Tasks
3.1.3Requirements Status
3.1.4Use Case Status
3.1.5Change Request Status
3.1.6Change Requests by Priority
3.1.7Test Case Status
3.2Resource and Cost Measures
3.2.1Earned Value versus Budget Cost
3.2.2Work Hours
3.2.3Schedule Performance Index (SPI)
3.2.4Cost Performance Index (CPI)
3.2.5Schedule Variance (SV)
3.2.6Cost Variance (CV)
3.3Product Size and Stability Measures
3.3.1Lines Added, Modified, and Deleted
3.3.2Requirements
3.3.3Requirements Churn (Revisions)
3.3.4Use Cases
3.3.5Use Case Steps
3.4Product Quality Measures
3.4.1Defects by Severity
3.4.2Age of Defects
4.Primitive Measures
4.1Template for a Microsoft Project Measures
4.2Template for Defect Measures
4.3Template for Enhancement Request Measures
4.4Template for Feature Requirement Measures
4.5Template for Use Case Requirement Measures
4.6Template for Requirements Churn Measure
4.7Template for Test Case Executed Measures
4.8Template for Test Case Planned Measures
4.9Template for Test Case Planned Measures
5.Annexes
5.1Data Transformations
5.1.1SPI, CPI, Cost Variance, Schedule Variance
5.1.2Age of Defects
5.1.3Test Case Results
Measurement Plan
1.Introduction
The primary objective of a measurement program is to generate information that provides insight into project information needs so that project managers, as well as any other management stakeholders, can make informed decisions based on objective data.
1.1Purpose
The purpose of this Measurement Plan is to document the measurements that the ClassicsCD.com project will collect and report so that management stakeholders can make informed decisions.
1.2Scope
This plan is limited to the ClassicsCD.com project and is not intended to infringe upon the detailed process/procedures used within the development team.
1.3Definitions, Acronyms, and Abbreviations
For the purpose of this Measurement Plan, the following definitions, acronyms, and abbreviations apply.
NOTE: Many of the definitions listed have been adopted from ISO/IEC 15939 [1] as well as from the Practical Software Measurement (PSM) [2].
1.3.1ACWP
Actual cost of work performed
1.3.2Base measure
Measure defined in terms of an attribute. A base measure is also fundamentally independent of other measures
1.3.3BCWP
Budgeted cost of work performed. Also called earned value.
1.3.4BCWS
Budgeted cost of work scheduled
1.3.5CPI
Cost performance index
1.3.6CV
Cost variance
1.3.7Derived measure
Measure that is defined as either function of one or more values of base measures.
1.3.8Information need
Measures are defined and implemented according to the information need of a project decision makers.
1.3.9Measurable concept
An idea about the entities that should be measured in order to satisfy an information need.
1.3.10Measurable construct
A measurement construct specifies exactly what will be measured and how the data will be combined to produce results that satisfy the information need.
1.3.11Measure (noun)
Variable to which a value is assigned as the result of measurement.
1.3.12Measure (verb)
To make a measurement.
1.3.13Measurement
Set of operations having the object of determining a value of a measure.
1.3.14SPI
Schedule performance index
1.3.15SV
Schedule variance
1.4References
- [1] ISO/IEC 15939:2002, “Software Engineering – Software Measurement Process.” Geneva, Switzerland, 2002.
- [2] McGarry, John, Card, David, Jones, Cheryl, Layman, Beth, Clark, Elizabeth, Dean, Joseph, Hall, Fred, Practical Software Measurement: Objective Information for Decision Makers. Boston, MA: Addison-Wesley, 2002.
- [3] Royce, Walker, Software Project Management: A Unified Framework. Boston, MA: Addison-Wesley, 1998.
1.5Overview
Measures should be evaluated in terms of the added value they provide to a project or an organization. They should only be deployed where the benefit can be identified. Measures not used in the management process should be “dropped.”
All projects are chartered with specific objectives. These objectives are typically defined in terms of system capabilities, resource budgets, schedules and milestones, quality, and business and system performance targets. Success is determined by how well the project team achieves these objectives.
Measurement planning usually begins with a project manager or project stakeholder identifying an information need to support project decision-making. Data that helps satisfy the defined information need can be obtained by measuring the elements or entities produced within the project. From an information need, you can then derive the entities that should be measured – this is called the measurable concept.
Here’s an example of a measurable concept: The project manager is concerned that iteration milestones are slipping and believes that scope creep could be a problem. Visibility into progress can be identified as an information need. There are many different factors that can help/hinder progress. One such area is functional stability. Functional stability is identified as a measurable. The measurement construct specifies exactly what will be measured and how the results will satisfy the information need. For our example, the measurement construct could be “requirements churn.”
The rest of this document identifies the management goals or information needs as well as the measurement concept and measurement construct (what to collect).
The measurement process and terms identified in this document are derived from the Measurement Information Model documented in the international standard ISO/IEC 15939, “Software Measurement Process,” and the guidelines outlined in the “Practical Software Measurement” (PSM).
2.Management Goals and Subgoals
The goal of a measurement program is to provide objective data so that the management, development and test teams can make informed decisions based on the information need. Therefore, a successful measurement program consists of first identifying the information needs.
The following information needs have been identified for the ClassicsCD.com project:
- In the previous two releases, delivery was late by three to four months. These schedule problems seem to “creep” up on the project team. Furthermore, each team manages its own schedule using Microsoft Project and reports its schedule status separately. The program manager wants to track the schedules more closely so that problems can be identified earlier, rather than near the end of a release. Information Need: Determine whether tasks for a given release will be completed on time.
- In the previous two releases, labor cost and expenditures exceeded the planned budget by over 10%. These figures were not determined until a financial analysis was performed after each release. To better identify budget problems, the program manager has mandated that each team track the budget using Microsoft Project. The program manager, as well as the team managers, would like a way to track earned value daily through trend and indicators so that they can pro-act instead of react. Information Need: Ability to track earned value through trends and indicators.
- It was determined that the likely cause of schedule slippage and budget overrun was largely due to scope “creep.” The feedback received from users resulted in numerous changes to existing requirements an the addition of new requirements. Information Need: Ability to track requirements churn or revisions so that the managers can determine the impact.
- After the first two releases, customers filed a high number of priority defects. It was speculated that the fact that the development team went through four redesigns of the software contributed to both the schedule slip and compromised software quality. Information Need: Ability to determine the amount of software rewrite or churn. Information Need: Ability to determine whether change requests are being resolved in a timely manner. Information Need: Ability to assess whether test coverage is adequate and whether tests are being successfully run.
3.Measures
Based on the information needs, we can refine the measures using the following table of Information Category, Measurable Concepts, and Measures:
Information Category / Measureable Concepts / MeasuresSchedule and Progress / Work Unit Progress / Tasks Complete
Late Tasks
Requirements Status
Use Case Status
Change Request Status
Change Requests by Priority
Test Case Status
Resource and Cost / Financial Performance / Earned Value Progress
SPI
CPI
Schedule Variance
Cost Variance
Work Hours
Product Size and Stability / Physical Size and Stability / Code Churn (Number of Lines Added, Modified, and Deleted)
Functional Size / Requirements
Requirements Churn (Revisions)
Use Cases
Use Case Steps
Product Quality / Functional Correctness / Defects by Severity
Age of Defects
These are the minimum set of measures that will be tracked on the ClassicsCD.com project.
3.1Schedule and Progress Measures
3.1.1Project Tasks Complete
Information Need / Determine whether planned tasks are completed in a progressive manner.Information Category / Schedule and Progress
Measurable Concept / Work Unit Progress
Indicator / Trend of Number of Tasks Complete
Goals / The goal of this measure is to ensure that tasks are being progressively being completed.
Decision Criteria / This measure is used to track the number of tasks completed in a given milestone. The number of tasks completed should approach zero the closer you get to the milestone. This shows that progress is being achieved.
Derived Measure / Number of Tasks Complete
Measurement Function / The number of tasks completed is derived from the Microsoft Project Collection. A task is considered to be complete if the Actual Finish date is set.
Base Measure / Number of Tasks.
Attributes / Actual Finish date.
Responsibilities / Rational ProjectConsole will be used to automatically collect the base attribute, Actual Finish, from Microsoft Project 2000. It is the responsibility of the project manager to ensure that the MS Project plan is kept up-to-date. A data transformation will then be automatically executed by ProjectConsole to determine for each collected task whether the Actual Finish date is set. If it is set, then the task is considered complete.
3.1.2Late Tasks
Information Need / Determine whether planned tasks for a given release will be completed on-time.Information Category / Schedule and Progress
Measurable Concept / Work Unit Progress
Indicator / Trend of number of late tasks
Goals / The goal of this measure is to ensure that tasks are being completed in a timely manner.
Decision Criteria / This measure can be used to track the number of tasks completed in a given milestone. The number of tasks completed should approach zero the closer you get to the milestone. This shows that progress is being achieved.
Derived Measure / Number of Late Tasks
Measurement Function / The number of late tasks will be derived from a collection from Microsoft Project. A task is determined to be late if the Actual Finish date is not set and the current date is greater than the Baseline Finish date. If the Actual Finish date is set, then task is late if the Actual Finish date falls after the Baseline Finish date.
Base Measure / Number of Tasks
Attributes / Actual Finish Date
Baseline Finish Date
Responsibilities / Rational ProjectConsole will be used to automatically collect the base measure and base attributes, Actual Finish and Baseline Finish dates, from Microsoft Project 2000. It is the responsibility of the project manager to ensure that the MS Project plan is kept up-to-date. A data transformation will then be automatically executed by ProjectConsole to determine for each task whether the task is late (defined above).
3.1.3Requirements Status
Information Need / Determine the progress of requirements definitionInformation Category / Schedule and Progress
Measurable Concept / Work Unit Progress
Indicator / Trend of Requirement Status
Goals / The goal of this measure to ensure that requirements development or definition is progressing in a timely manner.
Decision Criteria / For the ClassicsCD.com project, requirements progress through various stages (or status). These stages are: Proposed, Approved, Incorporated, and Validated. As you near a milestone, the number of requirements should be in a certain state. For example, at the end of inception, you would expect all requirements should be in the Approved state. Otherwise, the inception phase is not complete.
Derived Measure / N/A
Measurement Function / N/A
Base Measure / Number of Requirements by Status
Attributes / Requirement Status
Responsibilities / Rational ProjectConsole will be used to automatically collect the base attribute, requirement status, from Rational RequisitePro. It is the responsibility of the project team to ensure that the RequisitePro project is kept up-to-date.
3.1.4Use Case Status
Information Need / Determine the progress of use case definitionInformation Category / Schedule and Progress
Measurable Concept / Work Unit Progress
Indicator / Trend of Use Case Status
Goals / The goal of this measure to ensure that use case development or definition is progressing in a timely manner.
Decision Criteria / For the ClassicsCD.com project, use cases progress through various stages (or status). These stages are: Proposed, Approved, Incorporated, and Validated. As you near a milestone, the number of use cases should be in a certain state. For example, at the end of elaboration, you would expect all use cases should be in the Approved state. Otherwise, the elaboration phase is not complete.
Derived Measure / N/A
Measurement Function / N/A
Base Measure / Number of Use Cases by Status
Attributes / Use Case Status
Responsibilities / Rational ProjectConsole will be used to automatically collect the base attribute, use case status, from Rational RequisitePro. It is the responsibility of the project team to ensure that the RequisitePro project is kept up-to-date.
3.1.5Change Request Status
Information Need / Determine the progress of change request resolutionInformation Category / Schedule and Progress
Measurable Concept / Work Unit Progress
Indicator / Trend of Number of Change Requests by Status
Goals / The goal of this measure to ensure that change request resolution is progressing in a timely manner.
Decision Criteria / For the ClassicsCD.com project, change request resolution progress through various stages (or status). These stages are: Submitted, Assigned, Open, Resolved, and Closed As you near a milestone, the number of change requests you plan to close should reach a targeted number. Or another way to look at this is the number of open change requests should approach 0.
Derived Measure / Number of Change Requests by Status
Measurement Function / N/A
Base Measure / Number of Change Requests by Status
Attribute / Change Request Status
Responsibilities / Rational ProjectConsole will be used to automatically collect the base attribute, change request status, from Rational ClearQuest. It is the responsibility of the project manager to ensure that the ClearQuest change request process is followed on the project and the ClearQuest database is updated by developers and testers to reflect change requests being resolved and closed.
3.1.6Change Requests by Priority
Information Need / Evaluate product release readiness by tracking the number of change requests by resolved by priorityInformation Category / Schedule and Progress
Measurable Concept / Work Unit Progress
Indicator / Trend of Number of Change Requests by Priority
Goal / The goal of this measure is to determine release readiness by tracking the progress of change requests resolved by priority. Particular attention is given to Priority 1 and 2 change requests.
Decision Criteria / This measure is used to track progress. High priority change requests are those defects and enhancement requests that are must be included in a release. As the release or milestone approaches, the number of high priority change requests still open should decrease while the number of change requests closed should increase.
Change Requestscan be classified by the following priority values:
Priority1 – Resolve Immediately
Priority2 – Give High Attention
Priority3 – Normal Queue
Priority4 – Low
Derived Measure / N/A
Measurement Function / N/A
Base Measure / Number of Change Requests by Priority
Attribute / Change Request Status
Change Request Priority
Responsibilities / Rational ProjectConsole will be used to automatically collect the base measures, number of defects as well as the attributes, Status and Priority, from Rational ClearQuest. It is the responsibility of the project manager to ensure that the change request process is followed on the project and that the ClearQuest database is updated by developers and testers to reflect change requests being resolved and closed.
3.1.7Test Case Status
Information Need / Determine the progress of test case executionInformation Category / Schedule and Progress
Measurable Concept / Work Unit Progress
Indicator / Trend of Test Cases Planned, Executed, Passed, and Failed
Goals / The goal of this measure to ensure that test case planning and execution is progressing in a timely manner.
Decision Criteria / For the ClassicsCD.com project, test cases are planned and then executed. The number of test cases executed that passed and failed are then tracked. As you near the end of a milestone, the number of test cases passed should approach the number of test cases planned and executed. The number of test cases failed should approach 0.
Derived Measure / N/A
Measurement Function / N/A
Base Measure / Number of Test Cases Planned, Executed, Passed, Failed
Attributes / Test Case Planned
Test Case Executed
Test Case Result
Responsibilities / Rational ProjectConsole will be used to automatically collect the base measures, number of test cases planned, executed, passed, and failed, from Rational TestManager. It is the responsibility of the project team to ensure that the test cases are executed so that the TestManager database is up-to-date.
3.2Resource and Cost Measures
3.2.1Earned Value versus Budget Cost
Information Need / Determine how far along the project is in the development – the amount of work completed.Information Category / Resources and Cost
Measurable Concept / Financial Performance
Indicator / Trend of Earned Value (BCWP versus Budgeted Cost)
Goals / The goal of this measure is to show the progress of development in terms of the amount of work performed versus the budgeted cost.
Decision Criteria / As the project progresses and work is completed, the amount of work is measured in earned value or budgeted cost of work performed (BCWP) . This value can then be compared with the budgeted cost to determine how far along the project is in the development. As the project progresses, the earned value or BCWP should approach the budgeted cost.
Derived Measures / N/A
Measurement Function / N/A
Base Measure / BCWP
Budgeted Cost
Attributes
Responsibilities / Rational ProjectConsole will be used to automatically collect the base attributes, BCWP and the budgeted cost, from Microsoft Project 2000. It is the responsibility of the project manager to ensure that the MS Project plan is kept up-to-date.
3.2.2Work Hours