Number E-C002, July 1998|
FUTURE DIRECTIONS IN ROADSIDE SAFETY
COMMITTEE ON ROADSIDE SAFETY FIGURES
John F. Carney III, Chairman
Richard B. Albin|
Robert F. Baker
Maurice E. Bronstad
James E. Bryden
Owen S. Denman
Arthur M. Dinitz
Don Jay Gripne
William W. Hunter
Malcolm D. MacDonald
King K. Mak|
Charles F. McDevitt
Richard G. McGinnis
John W. Melvin
Robert A. Mileti
Richard D. Powers Robert Quincy Malcolm Ray Randa Radwan Samaha
|Larry A. Scofield Robert K. Seyfried Rudolph Kenneth Shearin Jr. Dean L. Sicking Kenneth Stack Roger L. Stoughton Rod J. Troutbeck Thomas Turbell William H. Wendling|
D.W. "Bill" Dearasaugh, Jr., TRB Staff Representative
|This circular is posted by the Transportation Research Board as a service to the transportation community. The information in this Circular was taken directly from the submissions of the authors and the sponsoring committee; it has not been edited by the Transportation Research Board|
IIA highway and facility design
Transportation Research Board|
National Research Council
2101 Constitution Avenue, NW
Washington, DC 20418
FUTURE DIRECTIONS IN ROADSIDE SAFETYJ. Erik Jonsson Center of the National Academy of Sciences
July 27-30, 1997
This workshop was intended to promote a dialogue on future directions in roadside safety among government officials, industry representatives, researchers, academics, and others. The strategies for improving roadside safety evolving from the efforts under NCHRP Project 17-13, Strategic Plan for Improving Roadside Safety, were presented. The need for improved standards for roadside safety features was the focus of the workshop. Workshop participants had the opportunity to raise questions about current standards, assess their relevancy, and propose improvements. The results of this workshop will provide the basis for considering research efforts to be undertaken under NCHRP Project 22-14, Improvement of the Procedures for the Safety-Performance Evaluation of Roadside Features.
Daniel W. Dearasaugh
Transportation Research Board
FORMULATING A STRATEGIC PLAN FOR IMPROVING ROADSIDE SAFETY
John F. Carney, III
Worcester Polytech Institute
TRANSLATING STRATEGIES INTO ACTIONS
Richard G. McGinnis
IMPLEMENTATION AND RESEARCH NEEDS
Kenneth S. Opiela
NCHRP, Transportation Research Board
UPDATING PROCEDURES FOR THE CRASHWORTHINESS EVALUATION OF HIGHWAY SAFETY FEATURES
Charles D. Sanders
WORK PLAN FOR UPDATING PROCEDURES FOR SAFETY PERFORMANCE EVALUATION
Texas Transportation Institute
DEFINING AND MEASURING RELEVANCY OF EVALUATION PROCEDURES
Lindsay I. Griffin, III
Texas Transportation Institute
REVIEW OF TEST MATRICES AND CONDITIONS
Roger P. Bligh
Texas Transportation Institute
OTHER SAFETY PERFORMANCE EVALUATION CRITERIA
Maurice E. Bronstad
Dynatech Engineering, Inc.
EFFICACY OF ALTERNATE EVALUATION TECHNIQUES
Dean L. Sicking
University of Nebraska
EVALUATION OF ROADSIDE FEATURES TO ACCOMMODATE VANS,
MINI-VANS, PICKUP TRUCKS AND 4-WHEEL DRIVE VEHICLES
Hayes E. Ross, Jr.
Texas Transportation Institute
FHWA PERSPECTIVE ON FULL-SCALE CRASH TESTING
James H. Hatton, Jr.
Federal Highway Administration
FHWA PERSPECTIVE ON FULL-SCALE CRASH TESTING
Richard D. Powers
Federal Highway Administration
TESTING AGENCY VIEWS
University of Nebraska
TESTING AGENCY VIEWS
Wanda L. Menges
Texas Transportation Institute
HARDWARE INDUSTRY PERSPECTIVE
Owen S. Denman
Energy Absorption Systems
HARDWARE INDUSTRY PERSPECTIVE
HARDWARE INDUSTRY PERSPECTIVE
Dean C. Alberson
Safety Quest, Inc.
SUMMARY OF BREAKOUT GROUP DISCUSSIONS
Group A- Relevancy of Evaluation Procedures (Moderator- Lindsay I. Griffin, III)
Group B- Test Matrices and Conditions (Moderator- Maurice E. Bronstad)
Group C- Full-Scale Crash Testing (Moderator- Owen S. Denman)
Group D- Evaluation Criteria (Moderator- Richard D. Powers)
Group E- Processes for Certification (Moderator- Don J. Gripne)
APPENDIX A-- PARTICIPANT LIST
I would like to welcome everyone on behalf of the Transportation Research Board. Your attendance and participation are greatly appreciated.
People talk about seeking excellence and striving for improvements all the time. Whenever I hear those words, I think of how difficult they are to achieve. The TRB Committee on Roadside Safety Features (A2A04) is unique because it has been effective in meeting these goals. Continual improvement has been noted in increased attendance at meetings with a high level of interest. You are truly on the cutting edge of technology, have contributed to improved and increased funding for NCHRP projects, have strong leadership, and benefit from active subcommittees.
You may know that every third year each committee at TRB submits a triennial self evaluation. Dr. John Carney, the committee chair, prepared a self-evaluation and circulated it through the committee for review. When it was submitted to the TRB Group on Design and Construction of Transportation Facilities, A2A04 was the only committee at TRB that received a perfect rating on their triennial evaluation. This reflects the excellence of the committee.
This workshop deals with future directions in roadside safety. The results of NCHRP Project 17-13, Strategic Plan for Improving Roadside Safety, provide many suggestions for future directions. This project is winding down and it is providing some very useful information to TRB Committee A2A04, researchers, and highway safety professionals. One example is the new NCHRP Project 22-14, Improvement of the Procedures for the Safety-Performance Evaluation of Roadside Features. This project addresses the need for updated and improved procedures for evaluating the crashworthiness of roadside hardware given increased concerns about safety. This contract was awarded to the Texas Transportation Institute and work on the project has just begun.
This is a very appropriate time in the life of this project to seek input that will assure procedures that will serve future needs. It is hoped that this workshop will provide a lot of useful input into this venture, which will eventually result in changes, improvements and modifications in NCHRP Report 350, Recommended Procedures for the Safety Performance Evaluation of Highway Features. It is important to note the international interest in this subject. Testimony to this fact is the many guests at this workshop from other parts of the world.
A review of the strategic planning efforts for roadside safety will be presented in three parts. First, I will summarize the process undertaken over the last two years that led to the strategic plan. The five missions that are the basis of the plan will then be described. Many of you were present at the Park City workshop last summer during which this strategic plan was subjected to a thorough review. Efforts have continued to build on the goals, objectives, and actions based upon the feedback gathered at the meetings. Second, Dick McGinnis will discuss in more detail the objectives, and actions believed necessary to achieve the goals. Last, Ken Opiela will discuss the research needs and implementation activities that have been derived from the plan.
The roadside safety problem is a very serious one. We have a million highway crashes associated with the roadside every year in the United States. It accounts for one third of the highway fatalities, results in societal costs exceeding $60 billion, and clearly is a major cause on injury and death. It is a very complicated problem, because there are so many variables or external influences. These include:
In the last few years there has been increased interaction with General Motors to provide dialogue with the vehicle manufacturing sector relative to roadside safety. The vehicle fleet keeps changing and there is a need to work towards improved compatibility of vehicles and roadside features. Traffic demands keep going up and driver behaviors are changing. It is interesting to note that a couple of weeks ago Congress had a hearing on driver behavior and the director of NHTSA coined the phrase, "road rage." There have been a series of articles in the newspapers in the last few weeks talking about mild mannered people who get behind the wheel of a car and turn into monsters. This is a serious problem that can have an impact on roadside safety. Environmental concerns, higher traffic speeds, an aging driver population, and the other issues make it hard to get our arms around the roadside safety issue.
The U.S. has done very well in highway safety when you consider that the number of vehicle miles have increased exponentially and yet fatalities are down in the low 40,000s instead of 59,000, like they were some 20 years ago. But 42,000 people are dying a year in this country, which is totally unacceptable. To address the roadside safety problem it is necessary to consider all aspects - the roadway, the driver, and the vehicle. This was the motivation for starting this strategic planning project.
The first meeting on this project took place here exactly three years ago. The purpose of the meeting was to identify the issues. Transportation Research Circular 435 Roadside Safety Issues was the result of that meeting. The logistics for a project were discussed after the Task Force 13 meeting in Charleston. The first NCHRP 17-13 panel meeting took place in May of 1995 where a decision was made to solicit additional issues and employ a facilitator to develop the strategic plan. At the 1995 summer meeting of this committee at the National Academy's facility in Irvine, California, members of the committee discussed issues further and developed research needs in roadside safety. A facilitator led the panel through a lot of nitty gritty work to lay the groundwork for the strategic plan at that meeting. Then, in January of 1996, at the TRB Annual Meeting, a briefing for top officials was held. The problem was the blizzard, and if you didn't come into Washington on the Saturday, you didn't get there. This was a disappointment, because the briefing didn't get the attendance that had been expected. In May, back in Washington, the panel tried to draw outsiders into the discussions-state police, advocacy groups, public works officials, and others- to get their perspectives on the problems and approaches to solve the roadside safety problem. In Park City, Utah, the 1996 A2A04 Summer Meeting was devoted to refining the goals and objectives of the strategic plan. As a result of these efforts a comprehensive strategic plan has been completed and efforts are underway to document it.
The panel came to consensus on the vision for the strategic plan-a highway system where people do not pay with their lives when vehicles inadvertently leave the roadway. In this system, drivers rarely leave the road. When they do, the vehicle and the roadside work together to minimize harm.
For those of you who have never gone through the trauma of developing a strategic plan- and it is traumatic- these are the elements around which the plan is structured:
Mission 1: Increase the awareness of roadside safety and support for it.
This paper will concentrate on providing a more detailed example to explain the process used to take a mission of the strategic plan and then narrow it down to specific actions. Obviously the main objective of the whole plan is reduce the number and severity of roadside accidents.
The strategic plan has the five missions. The first mission is focused on the need to create an awareness among the public about roadside safety, to help decision makers understand the problem, and to get the needed financial and political support to address the problem. Second, there is a need to develop the data and analysis procedures that are needed to fully understand the problem, monitor changing conditions, and come up with solutions. The third step is to keep the vehicles on the roadway by promoting smooth traffic flow, good maintenance, and proper driver behavior. If that can be done, then the problem of roadside accidents is solved. Practically speaking, it isn't always possible to keep the vehicle on the road, so, mission four is to keep the vehicle upright or keep it from hitting a harmful object, if it does go off the roadway. Finally, if rollover crashes cannot be avoided, then there is a need to minimize the injuries and the fatalities through effective design of the roadside and vehicle.
These missions, from an overall perspective, are logical, but they do not provide clear directives on actions that can be taken or provide a basis for measuring the effectiveness of efforts to address the roadside safety problem on each of these fronts. The structure used in formulating the strategic plan is based on missions, goals, objectives, and actions. The mission is a broad statement of an area to attack-like keep the vehicle on the roadway. Then within each mission there are specific sets of substrategies of how to approach that problem. These are reflected in goals which highlight the desired outcomes of the efforts. Below each goal, are more specific objectives. These are more detailed definitions of the approaches to achieve a particular goal. Finally, there are the things that specific agents or groups can do to address the problem-the activities or actions.
In the strategic plan, actions can be categorized into four different areas - research, operations, policy, and training. Research activities are obvious. These are activities oriented to getting the answers to questions about impacts or the need to get more basic information. The operations activities cover those things that can be done now; agencies just have to do it. It may take resources, but it can be done in the short term. Policy activities relate to changing legislation or policies within agencies to effect change more broadly. Finally, training activities are needed to train the engineers or other people in the DOTs and the public–the drivers–to actually behave more safely.
To demonstrate the detail in the strategic plan, I chose to concentrate on mission three– keep the vehicle on the roadway. An iterative process was used to define the goals, objectives, and actions under this as well as other missions.
There are three fundamental aspects that have to be considered in addressing the roadside safety problem— the roadway, the vehicle and the driver. These three aspects served to provide the basis for formulating the details of the strategic plan. The goals defined under mission three reflect these aspects:
The next step in the process involved defining objectives for each of these five goals. The objectives that were defined for the Mission 3 goals are:
Goal 3.2 - Improved traffic operating environment that reduces the occurrences of events contributing to loss of vehicle control and roadside encroachment.
Under Goal 3.1 there are three objectives that focus on design approaches to reducing the probability of a vehicle actually leaving the highway. Developing better tools for the highway designers to analyze and incorporate safety represents one aspect of keeping vehicles on the road. Since such tools do not exist, the research at FHWA to develop the interactive Highway Safety Design Model (IHSDM) is a step in that direction. The next objective is to improve design policies and guidelines in order to keep the vehicle on the highway. Recent research has found that highway designers do not explicitly consider safety when following current design guidelines. A key reason for this is some rather significant gaps in understanding the relationship between design elements and their safety impacts. There are practices, such as safety audits or reviews, that attempt to assure that new road designs are safe, but more can be done to improve guidelines, tools, and policies.
The fourth goal under mission three is aimed at the vehicle-based systems to keep vehicles on the road. Two different objectives have been identified. The first objective is associated with the prospect of smarter vehicles that will have reliable vehicle lateral guidance systems. These systems would automatically keep vehicles on the road. The second objective is to give more information to the driver to help him or her keep the vehicle on the road. These might be improved signing, marking, or delineation treatments, or audible warnings that you are getting too close to the edge of the road or advance warnings of road narrowing or curves. Some of these things that are not available yet, but are expected to be in use over the next ten years.
The fifth goal is oriented to the driver. Three objectives have been defined for improving driver performance. The three objectives here address changing behaviors, policies, and driver education relative to roadside safety. Clearly, changing behaviors like the aggressive driver is not easy. Changing the driver’s behavior could be considered a training effort, but there is also research needs associated with it. Similarly, policy changes, legislation, and better enforcement or seatbelt laws can play a role in improving roadside safety.
Under the goal 3.3, the maintenance aspect is addressed. There are three objectives which indicate how maintenance can contribute to improving roadside safety. The objectives address maintaining the roadway (e.g., preventing shoulder edge drop-offs), maintaining the vehicle (e.g., replacing defective head lamps), and minimizing the number and duration of work zones which can be considered special cases of the roadside.
Then finally, in goal 3.2, there are two objectives. The first relates to the overall operating environment of the roadway, vehicle, and driver. Through effective management of the roadway operating conditions it is possible to reduce the number of vehicles going out of control and potentially leaving the roadway. Improvement in signing, lighting, and other aspects of delineation and guidance can be used for these purposes.
Through the efforts of the panel and many other professionals who participated in workshops or provided written comments on the strategic plan another level of detail evolved. For each objective a series of actions has been defined. These represent the specific elements of work or research that can contribute to meeting specific objectives and goals. It is important mention that the strategic plan has more than 350 action items. Full implementation would require millions of dollars to implement. This implies the need to do a cost effectiveness analysis, to see which ones of these actions should have the highest priority. This level of detail allows the entire plan to be stratified in several useful ways, and including:
There is not time to address all of the actions under mission three. It is possible to get a sense of this level of detail by looking at the actions under Goal 3.1 related to improving highway design procedures. The actions indicate a need to have a better understanding of the relationship between the effects of geometry and roadside safety. These are:
This would allow a more consistent design process. Research actions are needed to improve hazards identification, and to integrate safety audits, and other procedures that could be used to enhance the design process and assure that designs are effectively implemented. The actions also suggest that 3- or 4-dimensional visualization technologies will provide a driver's perception of the roadway to help designers assess the effectiveness of designs, traffic controls, and other features that affect roadside safety. Other actions address the need to convey new procedures to the actual people designing the systems. There must be simultaneous actions to get the policies changed, so that all this new information and procedures get used by the highway agencies; that sometimes can be a very difficult and slow process.
The present version of the plan has been through seven iterations starting from the very broad missions to the specific actions just described. It has been a tedious process with input from many professionals that led to this plan comprised of 5 missions, 25 goals, 121 objectives, and 358 action items. There has been constant editing, consolidation, and reorganization, but the panel is satisfied that the strategic plan comprehensive and useful. The remaining challenges will be to implement the plan, set priorities for individual elements, and continually update the plan to reflect changing conditions.
IMPLEMENTATION AND RESEARCH NEEDS
Kenneth S. Opiela
TRB, Senior Program Officer
In an effort to address the roadside safety problem, the National Cooperative Highway Research Program assembled a distinguished group of experts and charged them with the task of identifying ways to improve roadside safety. In their deliberations, they formulated a vision, missions, goals, objectives, and actions for improving roadside safety as has been described by the previous speakers. It is important to note that considerable effort has gone into the development of the strategic plan. The project panel, several consultants, and many others, including those from very different backgrounds and diverse perspectives on safety, have contributed to content and organization of the strategic plan through numerous meetings and workshops. It has been estimated that over 2.5 person years of time was contributed in meetings and consultant efforts for the plan. The strategic plan has reached a level of considerable detail as a result of the incremental build, review, and revise process that was utilized.
The mere existence of a strategic plan does not assure that roadside safety will be improved. Further, neither the NCHRP nor its expert panel have the authority to implement such a plan. Recognizing these facts, the panel has attempted to develop recommendations for future directions in roadside safety using the strategic plan. It was recognized that the plan can be thought of as a comprehensive list of strategies or actions that can contribute to improved roadside safety. Some of these strategies are regularly being done, while others may be possible but not getting adequate attention. There are, of course, those things that cannot be done because relationships are not understood, agencies are not communicating, data is lacking, or systems have not been developed. Based upon these distinctions, the panel has been pursuing the development of implementation recommendations and lists of research needs.
The five fundamental missions and the associated goals of the strategic plan are presented in Table 1. The subdivision of the missions and goals into objectives, actions, and needs allows the roles of various agents to be highlighted, action agendas formulated, and needs for research identified. The following paragraphs highlight some of the actions that can be effective, the implementation efforts that are necessary, and areas where research is needed.
Translating the Vision into Actions
The efforts to identify actions in the strategic plan provided the basis for defining specific activities that could be undertaken to improve roadside safety. Summaries of these actions by discipline provide an action agenda that can reinforce continuance of current activities or to highlight other things that can be done. Lists of action items are being developed for the planning, design, operations, maintenance, and administrative functions of DOTs. In addition, the strategic plan is providing a basis for identifying roles for law enforcement, driver training, construction, vehicle design, and other related disciplines. These action plans provide a basis for systematic review of an agencies efforts to address roadside safety problems.
The details provided in the strategic plan also provide the basis for further analyses. First, decomposition of the strategic plan also reveals areas where synergy can be achieved through the coordination of actions. It is also possible to isolate all actions that can be directed to specific problems for program development purposes. For example, crashes with trees are known to be a serious problem. By tagging all elements of a plan that have direct relationship to an issue, it is possible to recognize the necessary aspects of a program to address that issue. The strategic plan also permits interaction analysis to provide the foundations for the establishment of coalitions, linking of activities, and finding places where multiple purposes can be achieved. It can facilitate the consideration of the trade-offs between actions or agents to determine the most effective.
The NCHRP effort identified many actions for addressing roadside safety problems. Some examples of important actions are:
Obviously, some of these actions can be undertaken immediately, at little cost and others will take time and larger amounts of money to implement. There is a strong consensus, however, that these and other actions can lead to major reductions in the deaths and injuries that are occurring on the roadside. These actions and others are described more fully in the report being prepared by the NCHRP.
During the efforts, it was recognized that highway safety encompasses a very broad range of organizations including:
Each of these groups has its own specific areas of expertise and concern which sometimes complement each other and other times work against each other. The primary purpose of a strategic plan for roadside safety is to form a framework to unite all these different organizations in coordinated action for improving the roadside.
Implementation of the strategic plan will require addressing the resource, procedural (process), and administrative issues listed below through coordinated efforts with the various organizations:
The comprehensive review of the strategic plan to improve roadside safety led to the identification of areas where research was needed. Important research needs by mission are described below.
Mission 1 - Increase the awareness of roadside safety and support for it.
Mission 2 - Build and maintain the information resources and analysis procedures.
Mission 3 - Keep vehicles from leaving the roadway:
Mission 4 - Keep vehicles from overturning or striking objects on the roadside when they do leave the roadway.
Mission 5 - Minimize injuries and fatalities when overturns occur or objects are struck in the roadside.
Roadside safety researchers have played a significant role in reducing fatalities and injuries in roadside collisions during the past several decades. In the past many problems could be solved with the intuition and judgment developed by researchers during 20 years of crash testing and observing accidents in the field. Many of the problems that remain are the difficult ones that have defied solution for many years. Maintaining the current level of safety in the face of increased travel demand, lighter vehicles, and strained public funding resources will require a bold strategic plan that coordinates the efforts of all agencies involved and maximizes the effectiveness of future research and development efforts. Roadside safety is a huge problem that can be significantly reduced.
Summary & Conclusions
The efforts under the NCHRP Project are believed to have laid the groundwork for further effort and importantly the development of a coordinated approach to improving roadside safety. More effort is needed, particularly to assure that all perspectives are considered and the most cost-effective options are selected. With the resources available for this project it is hoped that a more detailed strategic plan can be developed and momentum promoted towards the implementation of the strategic plan. The next step in the process will involve a series of meetings in April and May of this year to add detail to the strategic plan, particularly relative to defining goals, objectives, actions, and research needs. It is envisioned that three working meeting will be held with 15 to 30 persons representing the various interested organizations. These meetings will focus on particular missions develop the appropriate additional elements under each. A national meeting is planned for late summer or fall during which the entire strategic plan will be presented and reviewed. Working sessions at this meeting will assess the completeness and feasibility of the strategic plan and attempts will be made to prioritize the research needs, build consensus, and promote a champion for the strategic plan for improving roadside safety.
A great deal has been accomplished in improving the effectiveness of roadside safety hardware during the past several decades. The always-changing vehicle fleet and highway environment do not allow the roadside safety community the luxury of complacency. There are significant challenges ahead in improving roadside safety. These challenges can only be met by openly discussing difficult issues as they emerge and focusing the efforts all those with an interest in roadside safety on coordinated action.
Table 1 - Missions and Goals of the Strategic Plan
1 - Increase the awareness of roadside safety and support for it.
2 - Build and maintain the information resources and analysis procedures.
3 - Keep vehicles from leaving the roadway.
4 - Keep vehicles from overturning or striking objects on the roadside when they do leave the roadway.
5 - Minimize injuries and fatalities when overturns occur or objects are struck in the roadside.
UPDATING PROCEDURES FOR THE CRASHWORTHINESS EVALUATION OF HIGHWAY SAFETY FEATURES
Recently, NCHRP Project 22-14, Improvement of the Procedures for the Safety-Performance Evaluation of Roadside Features was initiated with the Texas A&M Research Foundation as the contractor. This effort continues the tradition of TRB involvement in the development and updating of crash testing procedures, but it also follows one of the important research needs noted in the strategic plan for improving roadside safety. These procedures provide the essential basis for certifying that new roadside devices are crashworthy. They also provide the performance requirements that are important in the design and development of innovative roadside safety features. Interestingly, this was one of those topics that came up in one of the early strategic plan meetings (Irvine, 1996) and was formulated as a research problems by the participants of that meeting. The research problem statement found its way into the NCHRP program and it was selected as a project to be funded under the FY '97 budget. The project was officially started on March 1st of this year. The project panel met last year, wrote the request for proposals, evaluated proposals, and selected the contractor from the several proposals received. At first, a lot of people ask why an update is needed since NCHRP Report 350 just came out. It is important to note that NCHRP 350 was published in 1992. It was adopted (by reference) as the FHWA standard in 1993, but since the compliance date for safety hardware was set for August 1998, the report has not gotten much attention -- until lately. Further, the updating process for NCHRP Report 230, the predecessor document, took six years from the time they first started talking about it. It takes quite a long time to systematically review a technical document and prepare an update, but it takes longer and longer as the procedures become more comprehensive and complicated. The evolution of the process clearly indicates growing complexity. The first procedures for crash testing (Highway Research Circular 482) took about six months to develop and prepare and it was one page in length. As hardware safety testing evolved, it took two to six years to update every time. So, it is the right time to begin looking into updating NCHRP Report 350. One of the things that needs to be emphasized is that the current research under Project 22-14 is not intended to produce the updated document. It is intended to look at specific needs for an update and to identify the research that may be needed to provide a sound basis for the update. Frankly, the funds available do not provide enough money to update Report 350. The objectives of this research defined by the panel are to 1) evaluate the relevance of procedures for the safety performance evaluation of highway features, 2) assess the needs for updates to the NCHRP Report 350 document and recommend strategies for implementing them. It is assumed that this project may identify specific needs for immediate updates to NCHRP Report 350. Project funds have been reserved to provide resources for the production of an interim update, if necessary. It is also assumed that a full update will eventually be needed and additional funding will be necessary at that time. When the panel first met, the members started with a long list of concerns about NCHRP Report 350. The following is not an all inclusive list of those concerns and it is not in any particular order. It does, however, indicate some of the things that the panel will think about for the update. These include:
Recently, NCHRP Project 22-14, Improvement of the Procedures for the Safety-Performance Evaluation of Roadside Features was initiated with the Texas A&M Research Foundation as the contractor. This effort continues the tradition of TRB involvement in the development and updating of crash testing procedures, but it also follows one of the important research needs noted in the strategic plan for improving roadside safety. These procedures provide the essential basis for certifying that new roadside devices are crashworthy. They also provide the performance requirements that are important in the design and development of innovative roadside safety features. Interestingly, this was one of those topics that came up in one of the early strategic plan meetings (Irvine, 1996) and was formulated as a research problems by the participants of that meeting. The research problem statement found its way into the NCHRP program and it was selected as a project to be funded under the FY '97 budget. The project was officially started on March 1st of this year. The project panel met last year, wrote the request for proposals, evaluated proposals, and selected the contractor from the several proposals received.
At first, a lot of people ask why an update is needed since NCHRP Report 350 just came out. It is important to note that NCHRP 350 was published in 1992. It was adopted (by reference) as the FHWA standard in 1993, but since the compliance date for safety hardware was set for August 1998, the report has not gotten much attention -- until lately. Further, the updating process for NCHRP Report 230, the predecessor document, took six years from the time they first started talking about it. It takes quite a long time to systematically review a technical document and prepare an update, but it takes longer and longer as the procedures become more comprehensive and complicated. The evolution of the process clearly indicates growing complexity. The first procedures for crash testing (Highway Research Circular 482) took about six months to develop and prepare and it was one page in length. As hardware safety testing evolved, it took two to six years to update every time. So, it is the right time to begin looking into updating NCHRP Report 350. One of the things that needs to be emphasized is that the current research under Project 22-14 is not intended to produce the updated document. It is intended to look at specific needs for an update and to identify the research that may be needed to provide a sound basis for the update. Frankly, the funds available do not provide enough money to update Report 350. The objectives of this research defined by the panel are to 1) evaluate the relevance of procedures for the safety performance evaluation of highway features, 2) assess the needs for updates to the NCHRP Report 350 document and recommend strategies for implementing them. It is assumed that this project may identify specific needs for immediate updates to NCHRP Report 350. Project funds have been reserved to provide resources for the production of an interim update, if necessary. It is also assumed that a full update will eventually be needed and additional funding will be necessary at that time.
When the panel first met, the members started with a long list of concerns about NCHRP Report 350. The following is not an all inclusive list of those concerns and it is not in any particular order. It does, however, indicate some of the things that the panel will think about for the update. These include:
Since most of the participants here don't represent state DOTs, it is appropriate that state DOT concerns be expressed. The DOTs are concerned about the costs of roadside safety hardware. About two years ago, the FHWA told the DOTs not to install breakaway cable terminals anymore. In Illinois, it was decided that products tested to NCHRP Report 350 would be used instead. This, of course, generated flak throughout the state DOT because of the increased costs. It is a big issue in a state like Illinois, where the DOT has 17,000 miles of roadway under direct supervision. A backlog of about 2,800 miles of roadway that needed repairs existed, but couldn't be funded. That backlog is anticipated to grow to around 4,000 miles over the next five years to some extent as a result of higher costs to meet safety requirements. It is not easy to convince the director that increased spending on guardrail end treatments is needed. The director doesn't get a lot of phone calls complaining about a outdated guardrail end treatments, but he gets a lot of phone calls about potholes. Looking at the Illinois in-service inventory, one notes roadside hardware treatments that have been in use for years. The DOT has a stockpile of components, maintenance people know how to put them in place, and designers know where they are effective. The need to meet a higher requirement all of a sudden renders these things outdated. Is there a need to include in the testing procedures a caveat for devices that have acceptable, long-term in-service performance. Is there a need to provide for less stringent in-service testing? In some cases, there is hardware in Illinois that didn't pass 230, and doesn't stand a ghost of a chance of passing the Report 350 criteria. Yet, the accident data indicates that it is working. There is also some hardware that passed 230, but it didn't work which in-service performance evaluations identified. There is a need to include acceptance criteria for hardware with effective in-service performance without crash testing. The states need a way of accepting products that have been used for years and that have been proved with experience. It seems rational to assume that no state would continue to put in hardware that was known to be killing people.
Reviewing the roadside hardware standards from the various states -- I used to work in the standards department and I have standards books from about every state in the United States -- one will note that there are more than 200 guardrail designs. Agencies forced to change their basic designs will face other difficulties. For example, Illinois is a steel post, steel block-out state. There are probably millions of posts and block outs installed. The wood industry has approached the department a couple of times about coming in. The DOT has said that they will accept wood post and wood block outs. Selling the alternative to contractors is another matter. The contractors argue that they know how to install steel posts and steel block outs and are equipped to install them. Changes will be costly to them. Other considerations are whether the test labs can complete testing to meet the 1998 date. Higher costs can be expected in other areas as well. For example, temporary concrete barriers are used extensively in highway construction projects across the country. It hasn't passed Report 350 yet. What are we going to do with all of the non-standard concrete barriers which because of their durability would be expected to get many more years of use in construction? These are major issues to the state DOTs.
The panel determined that an important aspect of this project is the relevance issue. The term often gets mentioned, but there is no universally accepted definition. The panel probably spent more time discussing relevance and trying to define relevance than on any other part of the work statement. The definitions and thinking changed frequently during the meeting. But the biggest concern of the panel that remained throughout was, when a vehicle leaves a roadway and hits a barrier, is it effective. All the crash tests and all the data isn't any good if lives are not saved and injuries reduced. After that comes the question of whether the prescribed crash tests relate to the types of hits that are occurring and at the levels of severity assumed. Good data is limited, but essential to providing these insights.
We need to address the relevance issue based on available data at this time, because there is a lot of data that is needed. Part of what we are hoping to get out this effort is how to get at that data. What research is needed.
Under Project 22-14, the contractor is undertaking a work plan that will involve the following efforts:
There is a little bit of money in our budget for a possible interim update if the panel identifies something that really needs to get out there now. This should help the industry people to relax a bit relative to the potential of changing requirements.
WORK PLAN UPDATING FOR PROCEDURES FOR SAFETY PERFORMANCE EVALUATION
King K. Mak
Texas Transportation Institute
The NCHRP Project 22-14, "Improvement of the Procedures for the Safety Performance Evaluation of Roadside Safety Features," research effort is divided into two phases and 10 tasks. The project started in March of this year and is proceeding as planned through the first phase of the work. The task elements are described below.
Task 1 involves a literature review and contacts with safety professionals. This meeting was planned as part of the effort to solicit inputs on updating needs and the relevance of the testing guidelines from more than the academic or research side. It is expected that the participants of this meeting will have the opportunity to provide insights and specific information, and openly propose issues that should be considered in the updating process. These inputs will be combined with the findings of the literature and the feedback from other contacts with professionals at the Federal Highway Administration, the state DOTs, universities, and industry to provide a comprehensive list of potential updating topics. These will be incorporated into an interim report to the panel from which they will pick specific items to be addressed in the second phase of the project that will lead to recommendations for future updates.
Task 2 attempts to develop a prototype methodology to assess the relevance of testing procedures. There is not enough money to fully address this issue, so it has been proposed that first a definition of relevance be established and then viable measures of it be determined. The research team has tried to deal with the definitions of relevance and how it can be measured to stimulate discussions during this meeting. It is believed that relevance measures are critical to ascertaining the need for a test outlined in the test matrix as well as to provide a means to identify emerging safety problems with roadside features. After the meeting, the research team will attempt to devise a prototype methodology that can be used in the future to address this issue.
Task 3 will focus on the assessment of the updating needs. A number of concerns have been identified, including:
The research team will look at these topics on a preliminary basis try to provide background materials, assess the impacts and updating needs, and make recommendations in an interim report.
Task 4 will involve a meeting with the project panel. The interim report will serve as the basis for detailed discussions at a panel meeting to determine the importance of the issues and to set priorities for changes to be incorporated in an update to NCHRP Report 350.
Task 5 will develop more detailed procedures for determining relevance. The results of the pilot studies and discussions of the interim meeting will allow a more in-depth review of means to measure relevance.
Task 6 will begin the process of updating by focusing on specific issues preparing "white papers" on these subjects. These white papers will present relevant background material, offer draft versions of the proposed changes to Report 350, provide commentary on the rationale for the changes, and assess the expected impacts on topics ranging from costs of testing to highway safety.
Task 7 will try to come up with some strategies for updating the NCHRP Report 350. These strategies will consider various scenarios of timing and scope for the updates. In essence, the panel will look at the items that need to be updated and determine which should be addressed right away. The project has funds for an interim update, if that is determined to be appropriate. A time table for undertaking further analyses and research, including crash testing, will also be formulated to assure a sound basis for the full update of the procedures.
In addition, updating issues not covered under the current contract will be prioritized. The research team will outline those topics that should be addressed prior to the full effort to update NCHRP Report 350. These efforts may involve further research efforts, crash tests, and/or integration and validation of findings from other research.
Task 8 will focus on the preparation of a second interim report summarizing the findings of Tasks 5 through 7. This report will provide the panel another opportunity to consider updating issues in more detail.
Task 9 will involve an outside review to solicit feedback on the proposed updates to NCHRP Report 350. It is not clear how this review will be handled. It might be possible to use the next summer meeting of A2A04, or it may be distributed for review and comment via the mail. The intent will be to provide parties another chance to offer arguments about the nature and need for proposed updates to the procedures for safety performance evaluation.
Task 10 will complete the efforts under the current project and document these efforts and the findings in a final report.
The planned schedule for Phase one, which is tasks 1 through 4, should be completed by September 1997. Basically, position papers have already been drafted. Inputs from this meeting will be incorporated into the position papers, and these will be put together into a report by September. The project panel will be provided the interim prior to an interim panel meeting before the end of 1997. In the interim panel meeting, the project panel will address differences of opinion, confirm or alter the topics that are considered important for an update by the project staff, and establish consensus on the priorities for addressing the issues. It is expected that efforts to develop the white papers will be underway in early 1998. The critical (second) review will begin sometime in mid- to late 1998 with the expectation that the project will end in May of 1999. Since the need for an interim update will not be determined until the interim meeting, there is no schedule for when that might occur. It is, however, expected that a full update should be planned to occur by 2002, which would be ten years after the publication of NCHRP Report 350.
In order to define and measure the relevancy of crash-testing procedures, we basically need to know three things about those procedures:
Let me begin by distinguishing between reliability and validity. In the shot pattern shown in figure 1, we see that the shooter is off target. More importantly, we see that the shot pattern is scattered all over the target.
FIGURE 1: UNRELIABLE SHOT PATTERN
There is no consistency, no reliability to the shot pattern.
In the second figure, the shot pattern is much tighter than in the first. The shooter is still off target, but at least the shooter is consistently off target. The shot pattern is reliably concentrated in a small area.
FIGURE 2: RELIABLE SHOT PATTERN
In the third figure, the shot pattern is again consistent, but in this figure the pattern is also concentrated in and around the bull’s eye. This pattern is valid. Validity in this case means that the shooter is actually hitting that which he or she intends to hit, the bull’s eye.
From just these three figures, it should be clear that "validity implies reliability," but reliability does not necessarily imply validity. If a given crash test is repeated many times, and if the results of each test are consistent, then the test is reliable, though not necessarily valid.
RELIABILITY OF TEST PROCEDURES
Reliability, as the term might be applied to crash testing, can actually be divided into two categories: intra-facility reliability and inter-facility reliability.
With sufficient funding it would, obviously, be possible to assess directly the intra-facility reliability of a particular crash test procedure by replicating a given test several, say, five or ten times. To the extent that all of the tests yield the same or similar results, the reliability of the test procedure is supported. [Unfortunately, when roadside appurtenances are tested for compliance with a set of guidelines, very few (if any) replications of successful crash tests are ever run. Once an appurtenance passes a test, that test is not typically repeated.]
In similar manner, the inter-facility reliability of a given crash test procedure could be supported through "round robin" trials in which several crash test facilities all conducted the same crash test. To the extent that all participating facilities produced the same (or similar) results and arrived at the same conclusions, the inter-facility reliability of the testing procedure would be supported.
As a practical matter, I doubt that funds will be forthcoming to test the reliability of current crash testing procedures. In the absence of funds to directly assess the intra- and inter-facility reliability of crash testing procedures, some estimates of the reliability of existing crash testing procedures might nonetheless be undertaken indirectly.
To indirectly assess the reliability of crash test procedures, we would start by identifying those factors that are fairly critical to the reliability of the tests (i.e., those factors that impact the outcomes of individual tests). Some of the factors impacting crash test reliability are shown in table 1.
|Table 1: Factors Impacting Crash Test Reliability|
üInstrumentation and Data Reduction
üDocumentation and Reporting
üVariables Not Accounted for in NCHRP Report 350
If we are going to have reliable crash tests, instrumentation is a major area of concern. Are the outcomes of individual crash tests impacted by differences in transducers, telemetry systems, on-board recording systems, filters, etc. And if these instruments are out of calibration, how is the reliability of the tests threatened?
When we reduce crash-test data, at least across organizations, different protocols may be employed. These different protocols may add to the variance in our problem, to the variability in the perceived outcomes of crash tests.
Documentation and reporting do not directly impact the reliability of crash tests, but if we are ever to understand the differences in crash-test outcomes, we have to have reports that are well documented and fully explained. NCHRP 350 does a pretty good job of outlining the documentation and reporting requirements for crash tests, but I think this area is deserving of more attention.
What about the test vehicles themselves? Is there any variance there? Some vehicle parameters are certainly specified in NCHRP 350: mass, wheel base, and so forth. But, we do not have a requirement for bumper height, suspension characteristics, optional equipment, inertial properties, or some measure of the frontal profile of the vehicle. These are characteristics that are left at variance—characteristics that can affect the outcomes of these tests.
Under the provisions of NCHRP 350, engineers are left with options for some tests. One example: the testing of break away support structures at 35 and 100 kilometers per hour (tests 360 and 361). These tests can be run so that the striking vehicle impacts the test structure "dead center" or midway between the centerline of the striking vehicle and the left- or right- outside edge of the vehicle. This option gives the investigator some latitude while again allowing more variance into our problem.
What about subjective evaluation? NCHRP 350 defines some outcome measures fairly well. Occupant impact velocity and ride down acceleration, for example, are both well defined that are mathematically quantified. But what about occupant compartment penetration or deformation, debris, and vehicle stability? It seems to me that we need some good operational definitions for these more subjective measures of test outcome. We do not define these terms nearly as explicitly as we might.
Test installations: NCHRP 350 gives some tolerances for conducting tests (e.g., speed and angle) but relatively little on the test installation itself. In some cases these discrepancies from the nominal (intended) height, width, strength, etc. may impact the outcome of the test.
Finally, there are variables (e.g., soil compaction, moisture) that are unaccounted for in NCHRP 350. When these variables are not accounted for, as in the previous examples, variance (unreliability) can be introduced into our problem. Two identical guardrails run under the same test conditions (except for the characteristics of the soil in which the guardrail posts are embedded) may yield very different test results.
In summary, if we want to run more reliable crash tests in the future, there are a number of issues that need to be addressed. But, it should be understood that by enhancing the reliability of our crash-test procedures, we are not guaranteed a relevant crash test procedure. Reliability is a necessary condition for a relevant crash test procedure, but it is not a sufficient condition.
VALIDITY OF TEST PROCEDURES
Test validity might be defined as "the degree to which a test actually measures that which it purports to measure." If a crash test procedure is valid, that procedure is somehow predictive of what should occur under similar circumstances in real-world crashes. When we run crash tests according to the procedures outlined in NCHRP 350, we assume that there is some correlation between test outcomes and the injuries sustained in similar, real-world crashes. The question might then be asked: are the crash testing procedures and acceptance criteria established in NCHRP valid? If they are valid, then appurtenances installed pursuant to those standards should promote reductions in injuries in real-world collisions.
There is substantial "face validity" for the testing procedures in NCHRP 350. The test protocols and acceptance criteria appear to be predictive of the outcomes of real-world crashes, but, the protocols and test criteria of NCHRP 350 have not been tied to real-world crash experiences. In fact, it would be very difficult to experimentally correlate the procedures and acceptance criteria established in NCHRP 350 with the outcomes of real-world crashes for several reasons, including those shown in table 2.
|Table 2: Problems in Validating Crash Testing and Evaluation Guidelines||
The dependent variables in crash tests include a number of qualitative and quantitative measures (e.g., occupant impact velocity, ride down acceleration, vehicle overturn, occupant compartment penetration, etc.) that, in essence, are surrogates for injury—the dependent variable of interest in real-world crashes. Undoubtedly, there is some correlation between the crash test criteria established in NCHRP 350 and real-world injury, but how good is that correlation? The answer is we do not really know. If the occupant impact velocity criterion in NCHRP 350 were lowered, and if appurtenances were designed to meet this more rigorous criterion, to what degree would crashes on this "new and improved" appurtenance result in fewer and less severe injuries? If the ride down acceleration criterion in NCHRP 350 were lowered, by what degree would new appurtenances manufactured to meet this enhanced requirement also serve to reduce the frequency and severity of injuries to motorists striking the new appurtenance? While we do not now have good answers to the questions just posed, there is little likelihood that we will ever be able to carry out the kinds of experimental studies that might be able to answer these questions.
Many factors affect the severities of injuries sustained in motor vehicle crashes involving roadside appurtenances: the type of crash (i.e., was the vehicle yawing or pitching when it struck the appurtenance, was the impact with the front quarter panel, the front, or the side of the vehicle), the type of vehicle in which the motorists were riding, their ages, whether or not they were belted, whether or not their vehicle had an air bag that deployed during the crash, and so on. Because so many factors (independent of the quality of the appurtenance) affect crash injury, it would be difficult to tease out these extraneous factors and express crash injury as a function of the provisions in NCHRP 350, or any other standard or guideline, for that matter.
Further complicating our problem is the fact that many appurtenances that are "unacceptable" under NCHRP 350 are not being fielded. Ethics aside (which is not being proposed), if we could randomly install some new appurtenances (e.g., crash cushions) that were "woefully inadequate," "almost acceptable" and "completely acceptable" under the provisions of NCHRP 350, we might experimentally discern a correlation between NCHRP 350 provisions and crash injury. But with a truncated range of appurtenances making their way into the highway environment on a non-random basis, correlations between NCHRP 350 provisions and crash injury have been, and will continue to be, difficult to establish experimentally.
Which brings us to the issue of accident data. Police-level accident data are highly subjective, often erroneous, and typically lacking in details on appurtenance crashes (e.g., was the guardrail struck on end or at some point throughout the length of need). Accident data bases that have been assembled through in-depth investigations are somewhat more reliable and complete, but the numbers of available crashes in these data bases (for a given appurtenance) are relatively few. In short, the crash data that might be needed to experimentally validate crash testing procedures are not available.
Even if crash data were available and sufficiently reliable for purposes of evaluating the provisions of a set of crash testing procedures, it should be recognized that "reporting thresholds" can seriously impact the evaluation of crash-phase safety devices. It has been known for some time, that as reporting thresholds are raised, the estimated effectiveness of crash-phase safety devices is reduced (see, for example, Griffin 1990). "Successes"—non-injury crashes that do not make their way into the data base—are not accounted for in estimates of effectiveness.
Finally, to introduce one last impediment to the experimental evaluation of the validity of crash test procedures, it should be recognized that many appurtenances are not installed and maintained "as tested." If a longitudinal barrier that has been found acceptable under the provisions of NCHRP 350 is installed on a down slope on the outside of a horizontal curve in weak soil, and then found to be inadequate (i.e., vehicles striking the barrier are going over or through it with consequential injuries to vehicle occupants), should we conclude that NCHRP 350 is invalid? No. The real-world barrier installation is so discrepant from the staged installation for crash testing that any real-world injury outcomes sustained in impacts with this barrier do not speak to the validity of the test procedures. By the same token, barriers that are ill maintained may deviate so greatly from the articles that were tested that injuries sustained on the ill-maintained article do not imply invalidity in the testing procedures.
In lieu of an experimental assessment of the validity of a set of crash testing procedures, we might instead clinically review real-world operational data and real-world crash data to shed some light on crash testing procedures. Such clinical analyses might be particularly useful in considering:
In order for an appurtenance to be qualified under NCHRP 350, it must pass several tests in a prescribed crash matrix, under pre-defined test conditions. Are these test matrices representative of what is happening in real-world crashes? Are the test conditions—the vehicles, the speeds, and the angles—in keeping with real-world experience? For guardrails, there are about seven different tests in the test matrix. On the basis of collected real-world crashes, are there too many different tests included in the matrix? Too few? Are we failing to run some tests that might have the potential to better predict injuries in real-world crashes?
The underlying philosophy of crash testing is that the tests should be run under the "worst practical conditions." If we were to deviate from this philosophy, if we were to run some tests under conditions more typical of those observed in real-world collisions, what would happen? Should we give some thought to running some tests on "mid-sized" vehicles, at different speeds, different angles, etc.?
In 1996, speed limits were raised on many highways throughout the United States. Travel speeds have increased. What are the implications of these higher speeds? Should we raise the test speeds for longitudinal barrier testing? Reduce the angle of impact? Real-world data—crash data and operational data—should be reviewed in trying to answer questions such as these.
How about the evaluation criteria used in crash testing? The two major areas of concern here are structural adequacy and occupant protection. When we consider the structural adequacy of guardrails, we are concerned about guardrail penetration and whether or not the impacting vehicle overturns. When we consider occupant protection, we are concerned about occupant impact velocities, ride down accelerations, and occupant compartment deformations and penetrations—all of which presumably correlate with injury.
Clinical evaluation of real-world crash data can help us to validate the test evaluation criteria used in NCHRP 350. How common are penetrations of guardrails built to NCHRP 350 standards? How often do vehicles overturn in crashes involving these rails? When vehicles do not penetrate the guardrails and when the vehicles do not overturn, how severely are the occupants injured? With increasing use of seat belts and ever greater proportions of the vehicle fleet being equipped with air bags, are our occupant impact velocity and ride down acceleration criteria too conservative? These are questions worth considering in the light of real-world data.
CONSEQUENCES OF TEST PROCEDURES
While recognizing the limitations on the reliability of crash tests as they are currently performed, and the impediments to assessing the validity of existing crash testing protocols and acceptance criteria, we are lead back to the question: are we improving highway safety (i.e., are we reducing injury frequency and severity) through established crash testing procedures? Or, in slightly different terms, as we have upgraded our crash testing procedures over the years (moving, say, from NCHRP 230 in 1981 to NCHRP 350 in 1993) what sort of marginal reductions in injuries have we experienced because of these upgrades? And, at what cost?
There are really two basic questions to be addressed in assessed the consequences of a new or proposed set of guidelines for crash testing:
To begin to address the question of marginal benefit, we might consider a non-experimental methodology that was employed by Wilson and Savage in 1973 to evaluate the effectiveness (i.e., the benefits) that might be obtained from the installation of air bags in passenger cars. Note that this study was undertaken in 1973, before air bags were commercially available in the United States.
In their study, Wilson and Savage studied 706 fatalities in 1967-1972 passenger cars. (Approximately 90 percent of these fatalities were unbelted.) "A jury of four engineers, whose backgrounds included experience in the design, development, and testing of both active and passive restraint systems, were chosen to conduct this study." During their deliberations, the jury reviewed detailed information on each of the 706 fatally-injured vehicle occupants (e.g., age, health, height and weight, etc.) and the circumstances surrounding the crashes in which they were involved (e.g., interior compartment intrusion, crash severity, interior loose objects, etc.). On the basis of the information gained from these 706 fatalities, the jury sought, among other things, to estimate the percent reduction in fatalities that would accrue to air bags, when not used in conjunction with a belt. They estimated that air bags alone should reduce fatalities by about 18 percent. [It might be noted in passing that a recent study from NHTSA (Kahane, 1996) estimates that air bags reduce fatalities by 13 percent.]
Using the jury system employed by Wilson and Savage, it should be possible to estimate the likelihood that a person who was injured or killed in a collision with a non-conforming appurtenance (e.g., a non-conforming guardrail) would have survived (or sustained lesser injuries) if he or she had collided with a guardrail meeting the criteria established in a new or proposed set of crash testing guidelines. Although a "jury evaluation" falls far short of the rigor of a true experimental assessment of the worth of a new or proposed set of crash testing guidelines, it may be a practical compromise for assessing the marginal benefits that would accrue to a new or proposed guideline.
Estimating the marginal costs of a new or proposed set of crash testing guidelines should be considerably easier than estimating marginal benefits. It will be assumed that a jury of engineers drawn from industry and from the crash testing fraternity can estimate the added costs that would be incurred in manufacturing, installing, and maintaining appurtenances built to a higher set of performance standards.
These two sets of estimates (the estimated marginal benefits and costs that would accrue to a new or proposed set of crash testing and evaluation guidelines) would define the relevance of those guidelines.
Griffin, L.I. "Estimating the Effectiveness of Occupant Protection Safety Devices from State Accident Data," NHTSA Conference on the Collection and Analysis of State Highway Safety Data, San Diego, California (February-March 1990).
Kahane, C.J. "Fatality Reduction by Air Bags: Analyses of Accident Data through Early 1996," (DOT HS 808 470). National Highway Traffic Safety Administration, U.S. Department of Transportation, August 1996.
Michie, J.D. "Recommended Procedures for the Safety Performance Evaluation of Highway Appurtenances," NCHRP 230, Transportation Research Board, Washington, D.C. (1981).
Ross, H.E., D.L. Sicking, R.A. Zimmer, and J.D. Michie."Recommended Procedures for the Safety Performance Evaluation of Highway Features," NCHRP 350, Transportation Research Board, Washington, D.C. (1993).
Wilson, R.A. and C.M. Savage. "Restraint System Effectiveness—A Study of Fatal Accidents", General Motors Automotive Safety Engineering Seminar, Warren, Michigan (June 1973).