November 1, 2013

Performance metrics are necessary, yet they seem hard to define and apply throughout a project's life cycle. Let's look at a few and delve a little deeper.

Technical Performance Measurements are defined at the start of a program—the planned progress of selected technical parameters. They provide a warning to systems engineers of technical problems and an assessment of the impacts of proposed changes in system performance. They assist in decision-making (performance or resource tradeoffs) through comparison of actual vs. projected performance.

TPM parameters that especially need to be tracked are the cost drivers on the program, those that lie on the critical path, and those that represent high technical risk items. System performance requirements are testable and measurable.

Key Performance Parameters

KPPs are considered the most essential for successful mission accomplishment. KPPs represent those capabilities or characteristics so significant that failure to meet the threshold value of performance can be cause for the concept or system selected to be reevaluated, or the program to be reassessed or terminated. Each KPP has a threshold and an objective value.

Trade studies typically may tradeoff everything, except a KPP. In other words, KPPs are non-trade-able. They flow from the operational requirements. They are documented in the Operational Requirements Document, the Capability Development Document, or the Capability Production Document.

Key System Attributes

KSAs are attributes considered most critical or essential for an effective capability but not selected as KPPs. KSAs provide decision makers with an additional level of capability prioritization below the KPP. A KSA does not have to be related to a KPP and there is no implication that multiple KSA’s equal a KPP.

Measures Of Effectiveness

MOEs are a measure of how well an operational task or task element is accomplished through using a system. The data used to measure the effect (mission accomplishment) comes from the use of the system in its expected environment.

MOEs often have green/yellow/red limits established early in the program and are tracked until they turn green or are demonstrated to be unachievable. The MOE can be stated either as a question or a declarative statement. The MOE is often stated in terms of accomplishment gained per cost incurred.

Measures of Performance

MOPs are components, or subsets, of MOEs. The degree-to-which a system performs is one of a number of possible measures of how well a system's task is accomplished. MOPs can be accumulated to assess an MOE that is not directly measurable. Several MOPs may be related to the achievement of a particular MOE.

MOPs supply supporting data to determine MOE status as it evolves over the life of a program. Together they determine what constitutes successful operations for the system. MOPs and MOEs are derived from requirements or the Concept of Operations. Their selection should be based on their ability to discriminate between levels of good performance.

An excellent source for more information on this subject can be found in the Technical Measurement Guide (Garry Roedler and Cheryl Jones Dec 2005 INCOSE-TP-2003-020-01).