Chapter 6: Decision Science Technologies

Although progressive transportation planners and project managers have concluded that public involvement is an essential element of effective planning and project development, most traditional public involvement processes focus on gathering preferences and opinions on controversial issues from the public but do not adequately consider how this information will be used in the specific decisionmaking processes. Stakeholders often feel their input goes into a "black box;" it is difficult for them to know if or how they have affected decisions and whether a decision is apt to "stick." This leads to frustration, polarization of interests, and political gamesmanship and ultimately results in the failure of many infrastructure improvement plans or projects.

Internal to transportation agencies, organizational or schedule pressures often cause managers to lose patience and jump to solutions (finding the quick fix). Other managers delay or avoid decisions, mistakenly feeling they are reducing personal risk. Still others make decisions based on their own risk preference rather than that of their organization. Although it takes courage to make hard choices under pressure, without a systematic process and helpful tools, many managers realize they risk making wrong decisions with potentially severe adverse consequences.

Informal decisionmaking methods generally involve elements of compromise (making peace with all affected interests), fairness (everyone gets something), and exercise of government authority (trust us; we’ll represent your interests). Often these informal approaches are successful, but just as often they do not withstand the intense public scrutiny associated with resource allocation or alternative selection and lead to political wrangling and project delays.

Decision science technologies can help transportation agency staff define problems, manage expectations, identify an appropriate range of alternatives, clarify information needs, identify and quantify uncertainties and their impacts on the decision, avoid decision traps in evaluating alternatives, and ensure meaningful involvement of stakeholders. Application of decision science technologies is advantageous for technical analysis as well as public outreach processes and generally assists in creating a credible and auditable decision process.

Four decision science technologies are applicable to transportation decisionmaking:

• Multiple attribute utility analysis
• Prioritization
• Risk analysis
• Optimization

Multiple attribute utility analysis technology is an approach used to evaluate and select alternatives based upon multiple attributes or criteria. This technology allows for the management of multiple objectives, the quantification of objectives, and the illustration of trade-offs. This technology is typically applied when multiple stakeholders with multiple issues are required to select one alternative.

Prioritization technology results in a ranking of competing alternatives based upon objective criteria and specified constraints. This technology is primarily used to prioritize multiple activities or projects and illustrate explicitly that the maximum benefit is being derived from the investment.

Risk analysis technology is an approach designed to determine how risk contributes to decision success and how to manage that risk. An example of an application of this technology is deciding when to proceed with a project to minimize the cost, risk, and uncertainty related to a parallel project.

Optimization technology is the development of an optimal system solution based on the comparison of multiple variables. This technology may be applied to determine traffic timing elements at a complex intersection.

Each of these technologies is supported by a host of tools, both hard tools such as computer models, and soft tools such as facilitation and conflict resolution.

Some decisions, such as searching for a low-cost alternative, depend on achieving a single objective. Many projects, however, require balancing a number of often competing objectives. Furthermore, project objectives are not usually considered equally important. In order to select the best strategy or alternative, decisionmakers often need to compare alternatives that provide different types and levels of benefits. Multiple attribute utility analysis (MUA) is designed to address these types of projects.

MUA allows decisionmakers to sort out these conflicts and make complex trade-offs. Decisionmakers develop an evaluation expression (typically, a weighted average of project objectives) to measure the relative goodness of each strategy. All choices and alternatives are evaluated against this objective measure of project performance. This evaluation expression provides a common means of comparison between often widely divergent strategies.

MUA proceeds through a series of defined steps:

This is the overall substance, or purpose, of the evaluation. It is that which is to be accomplished by making a decision. This clarifies what is included and excluded from the scope of the evaluation. Framing the problem clarifies what can be a difficult issue in complex problems.

Fundamental objectives are the most basic elements of the problem. Fundamental objectives may be further characterized by the development of subcriteria, which ultimately produces an objectives hierarchy. The objectives hierarchy presents a simple, graphical representation of the important elements. It is critical that the key decisionmaker(s) understand and approve the fundamental objectives.

Once the objectives are fully developed and the decisionmaker(s) agree that they fully represent the important issues in the problem, performance measures are required to determine how well alternatives perform against the objectives.

Performance measures may be quantitative or qualitative, depending upon the objective. A quantitative, or "natural," scale can be measured directly. Using a car as an example, fuel efficiency can be measured in units of miles per gallon. The decisionmaker can obtain data to assess performance against the objective. Similarly, storage space can be measured in units of cubic feet. On the other hand, style is subjective. The performance of an alternative against this objective depends upon opinion and cannot be researched from a consumer report, as can fuel efficiency and storage space.

One of the most important features of MUA that distinguishes it from other forms of analysis is the creation of performance measurement scales for attributes that are difficult to measure. Each performance measure can be arithmetically transformed to a scale of zero-to-one. The zero-to-one scale shows a linear relationship between cost and utility. This means that increasing cost from $1,000 to $1,500 is as important as increasing cost from $1,500 to $2,000. The two incremental changes are of equivalent utility. However, many scales are nonlinear. Changes along the scale have different degrees of importance. Utility functions are defined by the perspective of the decisionmaker(s). They are created by consideration of incremental changes along the performance measurement scale.

Alternatives are the actions that may be taken to accomplish objectives. A well-considered problem framing includes a complete set of alternatives. Care must be taken not to exclude or overlook alternatives that might meet the stated objectives.

Alternatives are often the first components identified in problem. As soon as a need or problem is identified, alternatives come to mind. Typically, alternatives are identified, then the attributes are compared. It is important to re-examine alternatives generated this way after the objectives hierarchy is well defined. This allows the important elements of the problem to define the alternatives, instead of the other way around. This creates "decision opportunities" from "decision problems." Sometimes this reconsideration identifies alternatives that may not have been considered previously.

Based on the value system of the decisionmaker(s), some objectives may be more or less important than other objectives. For example, loss of an ecosystem may be more important to a particular decisionmaker than the cost to protect that ecosystem. Obviously, different stakeholders faced with the same problem may have different underlying value systems, and, therefore, may have a different sense of what is most important in the given problem.

This leads to the concept of "weighting" objectives. Assigning weights to objectives is a subjective exercise based on the values of the stakeholder(s). Weighting is done after the performance measures have been developed, so stakeholders can include in their consideration the extent to which the full set of alternatives vary in performance.

The weighting process should not be interpreted as a determination that some things are important, and others are not. Any attribute that is in the objectives hierarchy has been determined to be fundamentally important to the decision analysis. Otherwise, it would have been excluded as irrelevant to the decision analysis. The process of weighting simply confirms that some fundamental objectives may be more, less, or equally important as others. Lesser-weighted objectives are still a factor.

Rating, or scoring, alternatives is the process by which the performance measurement scales are applied to the alternatives.

The results of any decision analysis are best regarded and applied as decision aids. Results should inform rather than dictate the decision. The analysis provides a way of organizing and comparing complex information. To the extent the decisionmaker(s) believe that the structure of the model represents the important issues in the problem, the weights and performance measures are appropriate, and the scores are accurate, they may be confident in the results.

Scores are usually subject to some degree of uncertainty, which should be addressed in the analysis. In addition, it is valuable to evaluate the model for sensitivity to weighting. If the results of the model do not change unless there are substantial changes in weights, then the decisionmaker(s) may be confident in the results.

MUA is a universal technology. It can be applied to almost all delivery phases for two reasons:

• High likelihood of the need for prioritization at most steps in the process
• Flexibility with application of the large set of tools that make up this technology

The geographic scale applicability is also very universal for the MUA technology. The structure the technology provides and the wide availability of tools are the reasons this technology may be applied at any scale.

Newberg-Dundee Project: This project in the Portland, Oregon, area provides an example of a corridor planningmajor investment study application of MUA. The study involved an alternatives analysis to more clearly define the transportation problem in the area, explore a broad range of multi-modal and multi-corridor solutions, and select alternatives for more detailed environmental review. The alternative analysis phase included a feasibility analysis to consider environmental, land use, and permitting issues that would need to be resolved during implementation of any solution to the transportation problem.

I-580 Freeway Extension Project: MUA was applied during the conceptual design phase of the I-580 Freeway Extension in Reno, Nevada. Criteria developed by the Project Stakeholders Working Group were used to evaluate alternative alignments within the preferred corridor established by a previous EIS as well as tradeoffs among various design features. The preferred alignment and design features emerging from the MUA process are being used as a basis for preliminary design and right-of-way acquisition.

MUA has the ability to provide significant benefit to transportation project planning, designing, and implementation. In particular, MUA can be very beneficial with identification of fatal flaws, reduction of litigation potential, and time savings because of the identification of stakeholder issues early in the process.

MUA also will have a significant benefit to resources management. Specifically, MUA will be very instrumental with improving the understanding of trade-offs and potential impacts. The same is true of data manipulation. MUA provides a very clear and simple structure for collecting, organizing, and transmitting data.

MUA and its associated tools is a very simple technology to implement. The technology is currently available and has been used successfully on many projects throughout the country. The costs associated with training for the use of the technology are very low.

The only integration issue associated with the technology is the desire for transportation agency staff and leadership to use more structured decision technologies. Successful use of this technology as part of a public involvement process requires the sponsoring agency to be very clear about the assumptions and framework of the particular plan or project in question and very willing to be open about the policy and objectives of the agency in carrying the plan or project forward. Those who are responsible for making the final decision, as well as incremental decisions along the way, must be identified at the outset and must be willing to make commitments for the agency. Recommendations stemming from a structured decision process such as MUA generally represent a broadly supported conclusion and are therefore difficult to ignore or countermand without loss of agency credibility.

Most transportation agencies make prioritization decisions on a daily basis. But when a large, complex organization must cope with increased budget constraints, or the phasing of projects, this presents a tremendous challenge for decisionmakers. Often, they must balance projects with multiple, competing objectives that differ in cost, schedule, benefits, duration, and risk. Managers are required to make difficult value trade-offs to generate the most benefit from the investments that are available. Decisionmakers must combine these values and opinions with hard cost and technical data and select the optimal portfolio of projects to meet budget constraints and explain/defend their decisions to external stakeholders. Increasingly complex and contentious budgeting processes require a rigorous approach to establish funding criteria and trade-offs.

The prioritization technology is designed to meet these needs. It incorporates technical project information, as well as cost and schedule data, and merges it with the values and concerns expressed by key decisionmakers and stakeholders. When a consistent measure of benefit is established, decisionmakers can proceed to optimize the selected project portfolio or budget, selecting projects that most closely match program goals and stakeholder views. The result is a ranked, prioritized listing of projects that demonstrates:

• The optimal mix of projects given a budget constraint
• A list of the criteria used in making the decision
• A summary of the weights (value trade-offs) used to evaluate alternatives
• The cost and schedule impacts of the chosen portfolio
• Sensitivity analysis, showing how certain criteria contributed to the overall scores
• "What if" analysis, testing different alternatives and future scenarios

The specific steps involved in the construction and implementation of the prioritization technology are outlined below. Many of these steps are similar to those defined in the MUA technology.

Like other complex projects, properly framing the decision problem and determining the analytical results that will address this problem are the critical first steps. This requires an understanding of the bounds in which the prioritization takes place and those stakeholders that must be involved.

A value hierarchy is simply a graphical representation of organizational objectives and the performance criteria used to evaluate achievement of these objectives. A well-constructed value hierarchy has several characteristics. In particular, a value hierarchy should contain objectives that are fundamental, nonredundant, and independent to ensure mathematical validity of priority calculations. Fundamental objectives are those that define the basic elements of the decision body. These requirements ensure that the benefits assigned to objectives in the value hierarchy are not double-counted and are additive.

Beyond fundamental objectives, a value hierarchy also will display performance criteria for each fundamental objective. Although certain fundamental objectives inherently define performance criteria, this component of the value hierarchy ensures a clear articulation of how fundamental objectives are accomplished.

The redefined set of candidate projects are prioritized according to the extent to which they contribute to achievement of the utility’s fundamental objectives. This requires establishing measures of performance for each fundamental objective hierarchy. These measures may employ "natural" or "constructed" scales. Natural scales are used where direct measures or data on project performance are available. Constructed scales, in contrast, are used when direct measures of project performance are not available. These scales, usually in the form of a narrative description of performance with reference to specific criteria, must provide precise, unambiguous definitions of project performance. Both forms of performance scale must identify the full range of project performance and collectively define the basis for evaluation and prioritization of the candidate projects.

Part of the performance measurement process is the explicit recognition that benefits often do not accrue linearly. Establishing a utility function allows for the conversion of the performance data to a measure of project benefit.

Although each of the objectives identified in the value hierarchy is "fundamental," each is not equally important, nor are the criteria used to define accomplishment of each objective. Accordingly, weighting of objectives and criteria is necessary to properly reflect organizational values. Weighting requires particular attention to several details to retain the mathematical validity of prioritization results.

Though all prioritization processes, formal or not, the process of developing and weighting objectives and criteria may result in revisions to the candidate project listing. In addition, this process may serve as an effective project screening vehicle in that it will require that candidate projects be evaluated to determine if sufficient information on the project is available to conduct a meaningful evaluation. It also can serve to facilitate redefinition of projects or construction of work packages that are more consistent with organizational objectives.

Each project, work package, or budget item is scored on its performance on each of the objectives included in the value hierarchy. Because objectives are independent and nonredundant, these measures of benefit may be added across objectives to yield a measure of total benefit. Projects are ranked based on the ratio of this benefit measure to project costs. Although care must be taken in the use of these rankings for project selection, these results provide clear guidance on the relative merits of candidate projects.

This information may be presented in a variety of formats to aid in ultimate decisionmaking. Perhaps the most useful of these formats is a benefit/cost curve, which plots project rankings by cumulative benefit versus cumulative costs. This curve provides a visual representation of project rankings, identifying projects that are clear winners and losers and those that offer relatively marginal benefits per unit of cost. Additionally, project selections given a specified budget constraint may be depicted by the distribution of funding across major categories. Finally, program scheduling is shown through listings of annual project funding recommendations.

These results can and should be tested through a variety of sensitivity analyses that help verify weightings and examine the relative importance of each objective to project rankings. These sensitivity analyses also frequently illustrate substantial agreement across different decisionmakers on numerous projects within a particular budget constraint, despite significantly different swing weightings.

Prioritization is a universal technology. It can be applied to almost all project delivery phases for two reasons:

• High likelihood of the need for prioritization at most steps in the process
• Flexibility with application of the large set of tools that make up this technology

The geographic scale applicability is also very universal for the prioritization technology. The structure the technology provides and the need for prioritization at all levels are the reasons this technology may be applied at any scale.

Prioritization is being applied to many programming processes throughout the country but probably has not been used in many applications designed to improve consideration of environmental issues in transportation decisionmaking. Many agencies informally prioritize projects without the use of a structured framework offered in this technology.

Northeast Area Transportation Study (NEATS), Sacramento, California: NEATS’ purpose was to identify and prioritize potential City of Sacramento transportation projects that would improve access and circulation within the study area. A structured decision process, with a Stakeholder Working Group (SWG), was used to involve the stakeholders, communicate concerns, and ultimately agree to the project priorities. The year-long study resulted in the SWG reaching consensus, recommending a ranked list of transportation projects within the NEATS study area, and creating a funding and implementation strategy for top-ranked projects. Environmental factors were explicitly included in the evaluation process.

Use of the prioritization technology results in many benefits, including:

• Better-quality decisions, because it develops "apples to apples" comparisons and increases understanding of value trade-offs.
• More effective use of resources, because it clarifies the value of more information and saves time and iterations.
• Improved defensibility, because it documents assumptions, makes decision rationale explicit, and reduces the role of advocacy.
• Enhanced credibility, because it includes the full range of stakeholder values, links objectives with actions, demonstrates expected return on investment, and promotes consensus.

Integration of the prioritization technology is slightly more difficult than the MUA technology. Application to the development of State Transportation Improvement Plans and Regional Transportation Plans would likely require major revamping of politically driven processes that have been in place for decades. Data requirements for candidate projects would be more rigorous than that required in many of the existing prioritization processes and would require the same level of data detail for all projects being ranked against each other. The limitations outlined above for application of MUA in a public outreach context also apply here.

Managerial complexity is increased by trying to maintain competitiveness and improving organizational credibility simultaneously. First, competitiveness and credibility are tightly linked: changes that affect one often affect the other. Second, neither can be managed in isolation, so managers must simultaneously juggle both organizational and analytical issues. Risk is the common link between these objectives. Because both contribute elements of project risk, sound risk management is usually one of the most effective means to balance these competing objectives.

For most organizations, coping with project risk can be a daunting task. It is difficult to know how and when to address certain risks, and in some cases whether the risk-reduction activities are worth the investment of resources. As a result, many managers avoid analyzing these uncertainties. However, risk is inherent in any type of project; failure to analyze these risks leaves the organization susceptible to a large measure of uncontrolled events and negative outcomes. In essence, avoiding risk can lead managers to "bet the ranch" on a daily basis.

It is not a question of how managers can completely eliminate project uncertainty: they cannot. The most they can do is reduce uncertainty and its impact on projects. To do so, managers must understand how uncertainty contributes to project outcomes, how to prioritize risk-reduction activities to maximize benefit, and how much effort to invest in risk reduction versus other business activities.

Risk analysis is an approach to decisionmaking that is well suited for problems where uncertainty may cloud the identification of a preferred strategy from multiple alternatives. It has been used widely in the business environment for several decades and is becoming more widely applied in the areas of environmental and resource management.

Risk analysis also proceeds through a series of defined steps that are very similar to the steps in other decision science technologies. The risk analysis process consists of:

• Problem framing
• Definition of alternatives
• Selection of a single evaluation criterion
• Identification of influences
• Identification of costs and uncertainties
• Risk profile creation
• Risk management strategies

Framing the problem should always be the first step. This is the overall substance, or purpose, of the decision to be made. It clarifies what is included and excluded from the scope of the evaluation.

Alternatives are the actions that may be taken to solve the problem. In risk analysis applications, these alternatives may be in the form of decisions to be made, strategies that could be pursued, or specific project alternatives.

Risk analysis is designed to analyze risk and provide evaluations to minimize and manage risk. In most circumstances the single criterion used in these assessments is cost. All issues, concerns, and influences would therefore be reduced to costs and associated probability of those costs occurring.

The next step is creating a model that captures the drivers that influence the alternatives. Drivers can be known elements, such as regulatory requirements or capital costs, and influences may be unknowns such as public support or transaction costs.

With identification of the influences, the next step is to populate these influences with the costs and associated probabilities of these costs. If cost is the evaluation criterion of choice, all influences must be reduced to the costs that they will force upon the alternatives.

The result of this assessment is the creation of a risk profile for each alternative. A risk profile is a graphical representation of the possible total costs and the probability of occurrence of these costs for each alternative. These profiles are created from the probabilistic occurrence of all the influences upon the alternatives. This information will provide a snapshot to which risk decisions may be made. The model also produces information to determine the level of influence drivers have upon the alternatives.

The final step is the application of the information from the model. The result will be a decision that creates a cost and associated risk that fits within the bounds of the decisionmakers’ level of comfort. Means to minimize the costs and associated risks will also result.

Risk analysis is applicable in most phases, but it does not lend itself to public involvement, description of existing conditions, or problem identification.

The geographic scale applicability is also very universal for the risk analysis technology. When uncertainty and risk related to cost are unknown, which can be true for all geographic scales, risk analysis can help understand and manage these risks.

Sound Transit’s High Occupancy Vehicle (HOV) Design Project, Renton, Washington: The options of the HOV design rely upon the decisions made by Washington DOT regarding the improvements to I-405. The HOV design will ultimately have to link to the final alternative selection for the I-405 improvements. Sound Transit was uncertain of the direction of the mainline improvements. Therefore, they were uncertain if they should commence with the HOV design within this uncertainty. However, if Sound Transit waits for the decision regarding mainline improvements, they run the risk of increasing the cost of the HOV design and construction. This is a risk analysis issue. The risk analysis decision science technology is designed to capture what is known, add the bounds of uncertainty, and generate a decision that minimizes the risk, and ultimately cost, for the agency. Sound Transit decided to move forward with the preliminary evaluation despite the uncertainty around the I-405 project.

In terms of improving consideration of environmental issues in transportation decisionmaking, risk analysis might be usefully applied to evaluate the potential success of various environmental mitigation strategies within a specific project context.

Right-of-Way/Real Estate Acquisition: The risk analysis process also can be applied to decision issues within a project. For example, a common concern in selecting project alignments, particularly in older or industrial areas, is the potential for subsurface contamination and the financial risks associated with purchasing and owning contaminated property (e.g., right-of-way) and of contacting hazardous materials during facility construction or operations. Risk analysis could be performed to determine the expected financial risk associated with purchasing, mitigating, and/or managing this liability. Assessment of human health and ecological risks would be one component of this overall financial risk analysis. Typical choices in this situation would be between "fixing" the problem (e.g., cleaning it up and securing release from financial or legal liability) or avoiding the problem by designing the facility around it (keeping in mind potential long-term risks, such as leachate movement from a dirty to a clean site). Each option has costs in time and money. A risk analysis process can be used to capture the variables associated with each decision and translate them into probability and financial criteria that can be compared.

Risk analysis may be able to provide significant cost and schedule savings to a project. Understanding and managing the risk of variables could have a significant reduction in work duplications and the uncertainty associated with costs.

Resources management influence may be equally significant. Applying the risk analysis technology can have significant influence on managing time and money. This translates into the better allocation of resources.

Risk analysis is one of the very few methods designed to manipulate uncertainty and risk. The tools that are used with this technology provide the structure to understand the uncertainty and risk of projects.

Risk analysis requires more education and commitment than the two previous decision science technologies. Although these transaction costs are moderate, its specific value to increasing consideration of environmental factors in transportation decisionmaking are less clear than in the other decision science applications.

Optimization technology is body of tools that provides the ability to understand, describe, and influence complicated systems. Organizations and processes can be modeled as an integrated system, allowing managers to improve processes, develop "what if" capabilities, and sort out complex relationships.

Optimization is a graphical, visual decision support process using simulation to model the functions of an organization. It tests ideas and strategies and, through analysis of process design, provides insight into how to improve. It can address all types of problems, including strategic, managerial, and technical issues.

A transportation application might be, How do I optimize throughput at an intermodal facility and design for future flows?

The steps in an optimization technology application include:

• Diagram or flowchart
• Determine how a system behaves
• Experiment with changes
• Provide instant, graphic feedback
• Produce critical performance measurement

Optimization can integrate information between various decision support tools, providing a clear view of process mechanisms of a complex system to create an insightful visual model.

Optimization is applicable in most phases, but it does not lend itself to public involvement or permitting.

The geographic scale applicability is also very universal for the optimization technology. Optimization is applicable to system modeling, whether it be broad, multi-state systems or a traffic signalization system.

Systems Modeling of a New Highway Facility with Other Existing Facilities: The optimization model would simulate the introduction of new traffic flows and the association with other transportation modes. This type of optimization is used frequently in the transportation industry.

Modeling the Flow of Pollutants from the Introduction of a New Transportation Facility: Optimization can model the ecosystem in which a pollutant may be introduced. The mitigation scenarios can then be modeled to understand the potential impacts and the value of each mitigation component. This is particularly applicable to a situation involving a sensitive species or sensitive area. The optimization modeling can mimic the ecosystem, model different scenarios, and demonstrate impacts and mitigation issues. Many models are used to assess the impacts of transportation projects and assess mitigation potential. However, the application of optimization in the assessment of environmental impacts is not widely used.

Of the four decision science technologies, this is the most difficult benefit to define. If the optimization tools are applied to a very complex situation, the results can be astounding. However, there are few applications of this technology.

The integration costs of optimization are the highest of all the decision science technologies. The extent of current application is fairly low because transaction costs to use this technology are fairly high.

The table below summarizes the applications and advantages and disadvantages of some decision modeling tools. These tools are designed to assist decisionmakers in managing expectations, solving problems, avoiding classic decision traps, and coaching leaders to successful decisions. Because a decision can be derailed at any step along the way, this set of tools has been developed to help identify traps and unproductive behavior and to resolve them before they happen. These tools also will help provide structured support, documentability, continuity, and defensibility in the technology approaches.