At the request of TRB’s state representatives, a special effort was made during the 1997 field visits to determine the current status of the use of performance measures within the state DOT programs. The following is a summary based on interviews with individuals involved in the use of performance measures within the various DOT offices. More information can be found in National Cooperative Highway Research Program Synthesis 238, Performance Measurement in State Departments of

Transportation, which includes detailed descriptions of current practice in Florida, Minnesota, Oregon, and Pennsylvania. Two other related TRB reports are NCHRP Report 357, Measuring State Transportation Program Performance, and Transit Cooperative Research Program Synthesis 6, The Role of Performance-Based Measures in Allocating Funding for Transit Operations. Ongoing research includes NCHRP Project 8-32(2), Multimodal Transportation Planning—Development of a Performance Based Planning Process.

A mix of cautious optimism and reasoned skepticism describes the general attitude of states regarding the usefulness of performance measures in improving their programs and activities. On the one hand, many states see the potential value of monitoring performance in different program areas across the state, as well as among subdivisions within the state, such as DOT districts. On the other hand, there is considerable concern regarding the use of performance measures to rank states against each other, as well as the potential use of these rankings by federal agencies to determine federal funding for individual states.

This concern is not without basis. Direct and meaningful comparisons among states are unlikely because of the inability to account for such factors as differences in climate and urbanization. Further, there remains considerable uncertainty over which factors are most relevant and whether the data needed to measure those factors are available. While these concerns also apply within a given state, expanding the comparison to a national level increases the uncertainty by an order of magnitude.

Nevertheless, nearly all states are moving toward greater use of performance measures. Yet even among the states that are fully committed to these measures as a central aspect of their management approach, most are in the early developmental phase. Perhaps the greatest current need is for sharing of information among the states, as well as from other organizations, regarding different approaches and lessons learned. This information exchange is taking place through various activities of the American Association of State Highway and Transportation Officials, TRB, U.S. DOT, and others, but a great deal remains to be done in this regard as additional states expand their performance measurement efforts. One state suggested cooperative efforts among groups of states having similar characteristics.

Some state DOTs have no choice. The state legislature or the governor, often in response to or as part of a demand for greater external accountability, mandates the use of performance measures for performance-based budgeting and oversight. One state legislature requires a formal report when progress falls 5 percent or more below a target level. Often the requirements are placed on all state agencies, not just the DOT. In fact, in one state the DOT-initiated approach was used as a model for extending the use of performance measures to other state agencies. As an aside, it is worth noting that the U.S. Army Corps of Engineers based its performance measurement approach on that of the Oregon DOT. Clearly, some states are playing a leadership role in this area.

Some state DOTs started using performance measures in support of ISTEA-mandated management systems, and continue to maintain these systems—though no longer mandated—for their own purposes. The most cogent reason for using performance measures was found among state DOTs that have implemented a comprehensive and aggressive management approach—termed a strategic plan, business plan, quality initiative, or the like—aimed at improving the department’s efficiency, productivity, and image with a strong customer focus. Indeed, such efforts appear to be the driving force behind the increased attention being given to performance measures. There is often an emphasis on greater public involvement in DOT programs, as well as on partnering with other agencies and organizations. With leadership from top DOT officials, these initiatives permeate the organization, resulting in a new outlook on doing business. Some high-level managers have direct on-line access to the information systems used for these initiatives so they can personally monitor progress.

What is most impressive in these states is that performance measures are being used not so much to identify poor performance as to encourage improved performance in a nonthreatening, positive way. To illustrate, one state uses the term "critical success factor indicators" instead of "performance measures." These new initiatives are not totally successful in the states that are trying them, nor are they fully supported by all levels within the organization, nor should all states adopt the same approach. Yet these initiatives are moving the DOTs in the right direction.

To facilitate increased sharing of information on performance measures among the states, TRB staff collected the names of contact people in each state in the performance measurement area and distributed this list to all the states. Much remains to be done, however.

First, there is a need to define what items are to be measured and how. The measures need to serve multiple users, including (1) program unit staff in their monitoring of progress, (2) top-level management in their decision making and monitoring of goal attainment, (3) legislators and governors in the budgeting process, and (4) the public. Program units use the more traditional measures of physical condition, such as miles of resurfaced highway. For the other users, greater attention is being given to outcome measures that can be directly related to goals and to transportation system performance, such as aircraft on-time arrivals and departures, improved mobility, and reduced air pollution. Measures currently in use are well documented (see TRB Synthesis 238 and NCHRP Report 357, for example), but more experience in their application is needed before agreement can be reached on what is most useful in supporting state programs.

Perhaps even more important, there is a need to establish benchmarks against which performance can be measured. These benchmarks must be realistic, that is, achievable, and they must be meaningful, that is, related to decision points. Given the need for continuing reassessment and revision as experience is gained, this task of establishing benchmarks will never be completed, yet it probably represents the greatest current need with regard to advancing the process.

The individual states and AASHTO are obviously the key players. Federal agencies, along with organizations such as TRB and the research community, also have a role in information dissemination, as well as in the conduct of research to support such needs as the identification of best-practice benchmarks and the development of process and procedural guidelines. Institutional concerns related to the use of performance measures are addressed in the next section.