developing performance measure- KPI

Developing Performance Measures
Taking Utah state government to a new level of performance calls for an outcome-oriented management strategy. Focusing on outcomes will direct management attention toward performance and will help strengthen the connection between state employees and their customers.
State Agencies are not required to follow a particular planning methodology or process. Figure 1 is a graphical representation of one method of developing performance measures and is presented for your consideration. A more complete explanation of planning terminology and specific examples is found in the remaining sections of this document.
Fundamental Principles of a Performance Measurement System
The tasks of allocating resources and assuring effective services at reasonable cost are significantly facilitated by the availability of meaningful and accurate performance information. The development of an enhanced measurement system is based on the following principles:
— That the measurement system be Results-Oriented. (Focus principally on outcomes and outputs).
— That the measurement system be Selective. (Concentrate on significant indicators of performance)
— That the measurement system be Reliable. (Produce data that are accurate and consistent over time).
— That the measurement system be Useful. (Provide information which is valuable to both policy and program decision-makers).
— That the measurement system be Accessible. (Ensure the periodic and systematic disclosure of results achieved through agency efforts).
Clearly, the overall value of a measurement system is dependent upon the quality of individual measures. Focusing more on the results than effort will require significant changes in the way we do business.
The use of outcome, output, input, and efficiency measures are integral to assessing agency performance in achieving stated goals, objectives, and strategies. Because of the need to clearly establish what is being shown by a measure and to document for audit purposes the calculations upon which reported performances is based, measure development must necessarily include an easy-to-understand, detailed definition of each measure.
The following section more clearly defines each of the components of a performance measurement system and provides development guides for each step in the process.
Definition of Planning Terminology
The mission of the agency identifies what the agency is, why it exists, and its unique contribution. Placed at the beginning of the agency's strategic plan, the mission provides the foundation for the rest of the process.
The mission describes the overall role of the agency as it relates to the state as a whole and is the common thread binding the agency's organizational structure and its activities. The mission of the agency may link several state functional areas depending on the unique nature of the agency. All agency employees should be able to identify their specific working relationship to this defined mission. The mission must be clearly understandable to the public and should at a minimum answer the following questions:
· Who are we as an organization and whom do we serve?
· What are the basic purposes for which we exist, and what basic actions are we established to accomplish?
· What makes our purpose unique?
· Is the mission in harmony with the agency's enabling statute?
A mission succinctly identifies agency proposes, distilling from enabling statues or constitutional provisions the most important reasons for an agency's work. In developing the mission, agencies should also examine other relevant sources, e.g., board policies and program descriptions. The mission should generally be no more than one or two sentences in length.
EXAMPLE: Hypothetical mission for a Department of Environmental Quality.
The mission of the Department of Environmental Quality is to safeguard human health and quality of life by protecting and enhancing the environment

The vision is the statewide basis for all other elements of the strategic planning process. It declares what the state's leaders envision for the future of Utah. Visions form the dominating themes that unify state functions. Functional goals and agency strategic plans should be developed in harmony with the statewide vision.
The development of agency goals is one of the most critical aspects of the strategic planning process. Goals chart the future direction of the agency. The goal development process begins to focus the agency's actions toward clearly defined purposes. Within the scope of the stated mission and utilizing the external/internal assessment, goals specify where the organization desires to be in the future. Goals are issue-oriented statements that reflect the realistic priorities of the agency.
Goals should be client-focused, address the primary external and internal issues facing the organization, and be easily understood by the public. Although there is no established limit, the number of goals the agency may develop should be kept to a reasonable number in order to establish the agency's direction and provide a unifying theme for programs and activities. They are shown in the strategic plan in an approximate priority order, beginning with those of the greatest importance and impact.
During goal development, the agency should begin identifying performance indicators to measure accomplishment. The formation of agency goals should include, but is not limited to, an analysis of the following questions:
— Are the goals in harmony with the agency's mission and philosophy statements and will achievement of the goals fulfill or help fulfill the agency's mission?
— Are the goals derived from an internal/external assessment and do they reflect responses to customer needs?
— Do the goals provide a clear direction for agency action?
— Are the goals unrestricted by time? Do they reflect agency priorities?
EXAMPLES: Hypothetical goals for a Department of Environmental Quality
"We will improve air quality in Utah, especially in major urban areas where air pollution is the greatest."
"We will aggressively enforce all environmental standards and ensure swift, fair, and effective enforcement for violators of these standards."
"We will educate the general public and industry about the importance of preserving the environment through pollution prevention, recycling, and source reduction."
"We will respond effectively and efficiently to federal mandates while aggressively pursuing federal funding for Utah 's environmental programs."
In contrast to goals, which are broad general statements of long-range purposes, objectives are specific, quantified, and time-based statements of accomplishments or outcome. Objectives represent the extent to which agency goals will be achieved at the end of the time period covered by the strategic plan. An agency's objectives should be derived directly from its stated goals which imply a priority for resource allocation. Objectives emphasize the results the agency is aiming to achieve. Outcomes are tied directly to objectives.
The development of objectives aids decision-making and accountability by focusing on issues and the accomplishment of outcomes. They should clearly quantify the specific results the agency seeks to achieve during implementation of the plan and should be easily understood by the public.
A focused external/internal assessment is necessary to establish objectives. The formation of agency objectives should include, but is not limited to, an analysis of the following questions:
— Does each objective describe an outcome in terms of specific targets and time frames? Is each objective realistic and attainable?
— Do the objectives relate to results or outcomes instead of internal processes?
— Are the objectives logically connected to a particular goal and the external/internal assessment?
— Although no limit is set, there must be at least one objective for each stated goal. They are shown in the strategic plan in an approximate priority order, beginning with those of greatest importance and impact. Generally, each objective should be no more than two sentences in length.
EXAMPLES: Hypothetical objectives for a Dept of Environmental Quality
— To reduce levels of criteria air pollutants to attain federal standards by 1995.
— To reduce air toxins emissions by 50% between 1992 and 1998.
Strategies are specific courses of action that will be undertaken by the agency to accomplish its goals. While goals indicate what the agency wants to achieve, strategies indicate how those goals will be achieved. Strategies are action-oriented rather than procedural in nature and are directly linked to output measures.
To develop strategies, the agency determines how best to achieve the results intended by the goals. More than one strategy may be needed for accomplishing each goal. In choosing strategies, the costs, benefits, and anticipated consequences of alternative courses of action must be evaluated by the agency. Strategies may, and probably will, cross programs, activity, or division lines. Questions to consider in developing strategies include, but are not limited to, the following:
— If this strategy (or strategies) is (are) implemented, can we assume that the goal will be reached?
— What are the anticipated costs and benefits of each strategy?
— Do we have the authorization to take the action outlined in each strategy? Is it legal and practical action?
— Do we have the necessary resources to implement this strategy (or strategies)?
Strategies must be easily understood by the public and should be generally no more than two sentences in length.
EXAMPLES: Hypothetical strategies for a Department of Environmental Quality
— Implement EPA's post-1987 ozone policy and expand monitoring network to monitor progress.
— Research, develop, and implement a comprehensive Air Quality Program.
— Implement a project to determine the extent of exposure of Utah citizens to toxic chemical in the environment and the impact on their health from these exposures.
Performance Measures
Performance measures are tools or indicators of the state's actions in achieving a given goal or objective. Performance measures can generally be divided into output measures, outcome measures, input measures, or efficiency measures.
Output measures are tools used to measure performance of the agency in implementing its strategies and to evaluate the efficiency of the implementation of the strategic plan.
In developing and selecting key output measures, the following questions should be addressed:
— Is the output reliably measurable? Will it measure the same thing over a period of time? Will the data used in the measure be available on a continuing basis?
— Is the output measure directly related to the agency's strategies?
— Does the output measure show the quantity of work performed? Can the measure be stated in unit cost terms?
— Is the output measure clear? Are the terms used generally accepted and defined? Will the measure be easily understood by those who are not familiar with the subject?
Outcome measures are tools used to evaluate performance of the agency in achieving its goals and to measure the effectiveness of the implementation of the strategic plan.
In developing outcome measures, the following questions should be addressed:
— Is the outcome measure directly related to the agency's goals?
— Is the outcome reliably measurable? Will it measure the same thing over a period of time? Will the data used in the measure be available on a continuing basis?
— Does the outcome measure show what change (difference) the agency's action will have on the target group or problem? What information is needed to tell whether the expected change is occurring? Is this information available or must we settle for proxy indicators such as output measures.
— Can the agency gather data for the outcome measure without incurring excessive costs or undertaking cumbersome procedures? Could sampling techniques or other more cost-effective alternatives be used to collect the data?
— Is the outcome measure clear? Are the terms generally accepted and defined? Will the measure be easily understood by those who are unfamiliar with the subject?
— Will the outcome measure enable a decision to be made or lead to a valid conclusion concerning the agency's action?
Specific examples showing the difference between outcome measures and output measures include the following:
— Percentage of high school graduates (output measure) is not the same as the percentage of students with a certain level of mastery of subjects (outcome measure).
— The number of patients treated and discharged from a state mental hospital (output measure) is the not the same as the number of discharged patients who are capable of living independently (outcome measure).
— The number of vaccinations given (output measure) is no an indicator of the reduction of the incidence of the disease (outcome measure).
The print-version of this article contains actual program performance measures developed by various departments and agencies of the State of Texas and adopted by the legislature as part of the General Appropriations Act of the State of Texas for 1993. (Open the document here).
Glossary of Terms
Strategic Planning is a long-term, future oriented process of assessment, goal setting, and decision-making that maps an explicit path between the present and a vision of the future; that relies on careful consideration of an organization's capabilities and environment, and leads to priority-based resource allocation.
Vision is an inspiring picture of a preferred future. A vision is not bound by time, represents global and continuing purposes, and serves as a foundation for a system of strategic planning. A statewide vision depicts an ideal future for the people of Utah and the contributions that state government can make to that end.
Agency Mission is the reason for an agency's existence. It succinctly identifies what the agency does, and why and for whom it does it. A mission statement reminds everyone- the public, the Governor, Legislators, the Courts, and agency personnel-of the unique purposes promoted and served by the agency.
Functional Goals are the general ends toward which the state directs its efforts. Functional goals address the primary issues facing the state within broad groupings of interrelated state concerns. Functional goals are founded on the statewide vision and may involve coordination among several agencies with similar functions.
Agency Goals are the general ends toward which agencies direct their efforts. A goal addresses issues by stating policy intention. They are both qualitative and quantifiable, but not quantified. In a strategic planning system, goals are ranked for priority. Goals stretch and challenge an agency, but they are realistic and achievable.
Objectives are clear targets for specific action. More detailed than goals, objectives have shorter time frames and may state quantity. An objective is achievable, measurable, and sets the direction for strategies. A single goal may be subdivided into multiple objectives.
Strategies are methods to achieve goals and objectives. Formulated from goals and objectives, a strategy is the means of transforming inputs into outputs, and ultimately outcomes, with the best use of resources. A strategy reflects budgetary and other resources.
Performance Measures are indicators of the work performed and the results achieved in an activity, process, organization, or program. Performance measures can generally be divided into outcome measures, output measures, input measures, or efficiency measures.
Inputs are the resources that an agency uses to produce services, including human, financial, facility, or material resources (e.g., number of dollars expended or tons of material used).
Outcomes are the quantified results, or impacts, of government action. Progress is assessed by comparing outcomes to objectives through the use of measures. Outcomes are the effects-both intended and unintended-of the agency outputs on a particular population or problem area. Outcomes are not outputs: an output is the quantity of a service or good produced; an outcome is the result, or impact, of the output.
Outcome Measures are tools, or indicators, to assess the actual impact of an agency's actions. An outcome measure is a means for quantified comparison between the actual result the intended result.
Outputs are the goods and services produced by an agency (e.g., number of students trained or miles of roads repaired).
Output Measures are tools, or indicators, to count the services and goods produced by an agency. The numbers of people receiving a service, or the number of services delivered, are often used as measures of output.
Efficiency Measures are indicators that measure the cost, unit cost or productivity associated with a given outcome or output.