The Problem section justifies the need for the project. The initial justification of the project is generally only the beginning in the understanding of the research problem. All research activities should be couched within “Further understanding the problem or problems.” This concept provides the “Big Picture” that justifies and unites all the activities and purposes of the research project. Without this concept, research programs run the high risk of failure from investing in solutions searching for problems, or allocating resources to problems that are already well understood. Problems needing research should be redefined and better understood as a matter of course throughout a successful research project. Without constant refinement of the research problem itself, finding effective solutions and implementable recommendations is elusive.

Researchable problems may be discovered in the problem statement phase. Some initial problem descriptions may not turn out to be researchable. Some may not be true problems. Many requests may be referred to operations or planning units for assistance. If a problem is thought to be worth research investment, hopefully, after further discussion, literature syntheses, surveys, pilot studies, experiments, development, etc. the problem can be better understood, and a useful result can still be offered. It is for these reasons that some successful research projects employ multiple lines of investigation, hoping that at least some of them will be useful for different aspects of the problem.

The problem section typically consists of one to three paragraphs describing the problem, and explaining why it should be solved. The section should include background information on current practices and why they are deficient. It is important to provide information on the magnitude and extent of the problem. For example, whether the problem is a serviceability issue or a safety issue, and whether it occurs at the local, regional or national level. Wherever possible, it is useful to provide baseline data, e.g. number of accidents, or cost of failures. It is often appropriate to place limits on the scope of the project by indicating what form the solution should take, e.g. a specification, report, test method, design procedure, computer program, or a piece of equipment. A very useful test of whether a problem can be solved by research is that, if the form of the solution can be defined, the problem is researchable.

The purpose of the Objective section is to state very clearly what products are expected from the research. The ability to define products that will resolve the problem, are attainable, and can be implemented, has a major impact on the likelihood of success.

Key words are used for indexing, and also for conducting a literature search to determine if related research is underway elsewhere. The section on related work provides an opportunity for the proposer to identify other work that is in progress or proposed in other research programs. Frequently, a research project will build upon the results of a project completed recently. It is unlikely that the results of the project will have been recorded in electronic databases, and would not be discovered by reviewers unless the proposer draws attention to the study.

Urgency/Priority provides an opportunity for the proposer to explain why the research needs to be initiated soon, and the consequences of a delay.

In the NCHRP process, problem statements include the cost, but not the duration of a project. Cost is required to estimate the benefit-cost of projects during the project selection phase. Other considerations, including cost, are discussed in the section ‘Request for Qualifications and Proposals’. Some agencies do include an estimate of time required for the work in the problem statement. This also provides an indication of the anticipated scope of the work, especially if fieldwork is envisaged, and permits annual budgets to be prepared for multi-year projects.

The User Community section is intended to identify groups who would benefit from the research and be affected by implementation of the findings. The Implementation section is necessary to identify how the results will be implemented. It is essential that an implementation strategy be developed before research work is initiated. This allows those who will be affected by the implementation to make necessary plans and ensures that, if successful, the results of the research will be in the form required by the users. The statement of effectiveness describes the anticipated state-of-the-practice after the results of the research have been implemented. In other words, it describes the benefits of undertaking the research so that those involved in the selection of projects can estimate the benefit-cost ratio. Where possible, the anticipated benefits should be expressed in the same units as the baseline data in the Problem section.

Because of the size of the program, and its large constituency, the NCHRP uses a two-stage process for developing problem statements. The first step is the documentation of ideas. These are then processed by NCHRP and FHWA staffs to provide feedback to the submitters in the form of a literature search, and identification of work in progress and related problem statements. It is the responsibility of the submitter to determine whether to prepare a second stage proposal, which must contain all the sections, described above. An example of a completed research problem statement is given in Figure B-1.

Figure B-1: An Example of a Research Problem Statement

^{Project Selection}

There are a number of ways of ranking individual projects for the purpose of developing the research program. In many agencies, the responsibility rests with an advisory committee, comprising representatives of the client groups. Typically each member of the committee will review all the research problem statements independently, discuss them with colleagues, and rank them according to some or all of the criteria given in Chapter Three. The ranking may be complex, involving weighting the criteria, or it could be simple three (0,1,2 or low, medium, high) or four (0,1,2,3, or no need, low, medium, high) point ranking. This simple approach works well when there are a large number of problem statements to consider, and the size of the program is such that most will not be selected. In situations where only a small number of problem statements are to be considered, a ranking procedure involving a greater number, and weighting, of the selection criteria may be justified.

^{Requests for Qualifications or Proposals}

As noted in Chapter Three, a Request for Qualifications (RFQ) is normally used in one of two situations:

1. When the work activities are known and the sponsor wishes to hire qualified people to complete the work.

2. As the first step before issuing a Request for Proposals (RFP).

An example of the first situation is given in Figure B-2. The RFQ provides an introduction describing why the work is being undertaken and the specific duties of the contractor. It also provides details of the format required for the response, selection criteria, time and schedule, and administrative details.

The format of an RFQ issued to screen respondents prior to issuing an RFP is very similar to that used in the first situation. The RFQ normally states the nature and scope of the work in general terms, and requests recipients to describe the qualifications and availability of key personnel, facilities, and prior experience in related work. It is common to limit the number of pages for the response.

An example of a Request for Proposals is given in Figure B-3. The RFP is an amplification of the research problem statement, with the addition of a schedule and administrative details. Because the NCHRP utilizes project panels, comprising individuals who are themselves technical experts in the field of study, RFP’s usually provide details of the tasks which the panel views necessary to complete the research successfully. Other research programs are less specific, and define the problem and solution desired, without stating the approach that is expected.

Figure B-2: An Example of a Request for Qualifications

Figure B-3: An Example of a Request for Proposals

^{Research Proposals}

Most sponsors have specific requirements for the content and format of proposals. This ensures that all the required information is provided. Review of the proposals is simplified if a standard format is used because the reviewers know where to find specific items. The NCHRP requires that the following information be presented in order:

1. Cover

2. Summary Page

3. Table of Contents

4. Research Plan

5. Qualifications of the Research Team

6. Accomplishments of the Research Team

7. Other Commitments of the Research Team

8. Equipment and Facilities

9. Time Requirements

10. Itemised Budget

11. Co-operative features (if appropriate)

12. Appendices (if appropriate)

Complete details of content and format are provided in the booklet ‘Information and Instructions for Preparing Proposals’. The following is an excerpt of the requirements (items omitted are self-explanatory or satisfy administrative requirements).

Research Plan

The research plan shall detail completely the prosecution of the research, including the submission of an acceptable final report. The plan ultimately becomes a part of the contract by reference of the proposal; therefore, it should describe, in a specific and straightforward manner, the proposed approach to the solution of the problem described in the project statement. It should be concise, yet include sufficient detail to describe completely the approach to the solution of the problem. Research methodology shall be described in sufficient detail to permit evaluation of the probability of success in achieving the objectives.

The research plan shall be subdivided into the following sections:

Introduction

The introduction to the research plan should provide a concise overview of the proposer’s approach to conducting research. It should describe the manner in which the expertise and experience of the proposed team will be used in the research, and the application of special data, facilities, contacts or equipment should be presented. The Introduction should highlight the linkages of the proposed team’s capabilities to the project tasks and the manner by which the proposed plan will satisfy the objectives.

Research Approach

This section will be used to describe how the objectives will be achieved through a logical, innovative, and rational scientific plan. The plan shall describe each phase or task of the research to be undertaken.

Anticipated Research Results.

The research plan for each proposal shall contain specific statements describing the anticipated research results. The results are expected to be presented in terms of the language and working tools of the practicing engineer or administrator so as to be immediately applicable to practice. Consequently, there must be specific statements of the manner in which the desired results would be reported; e.g., mathematical models, design techniques, field or laboratory test procedures, or recommendations for changes in AASHTO, FHWA, or standard highway specifications. If the nature of a project is such that it is known initially that the results will not be amenable to immediate implementation into practice, the research plan must include recommendations for the additional work necessary to reach the implementation stage.

Applicability of Results to Highway Practice.

The research plan shall include a section captioned “Applicability of Results to Highway Practice” that clearly describes how the anticipated research results can be used to improve highway engineering practices. Clearly indicate the expected audience for the research results. The writer should present the strongest case possible to convince the reviewer that the results of the research will be practical and implementable.

Qualifications of the Research Team

Name. Address, telephone number, and pertinent background information must be provided for the principal investigator bearing primary responsibility for the project. The same information is required for other research team members participating to a significant degree. The proposals must describe how the research team members’ academic, industrial, and/or research experience relate to the project to be undertaken.

Accomplishments of the Research Team

Proposals shall contain a summary of the past accomplishments (“track record”) of the research team in the same, or closely related problem area of the project to be undertaken. This summary is to include full particulars concerning all known instances of application to practice of the agency’s research results. If no such knowledge exists, it should be so stated.

Other Commitments of the Research Team

Proposals shall contain a listing of current organization and personnel commitments to other work in sufficient detail to indicate that the organization and all of the individuals assigned to the proposed project will be able to meet the commitments of the proposal. Man-hour commitments and percentage of time committed to other work for each member of the proposed research team shall be specified.

Equipment and Facilities

This section shall include a description of the facilities available to undertake the research and an itemization of the equipment on hand considered necessary to complete the research. In the event that use of the facilities or equipment is conditional, the conditions should be described. In the event that certain facilities or equipment are considered necessary to undertake the research but are not on hand, that fact should be presented. The proposer should identify any arrangements that will be made to borrow or rent necessary equipment. In the case where it is contemplated that additional equipment will be purchased under project funds, be certain that the budget item “capital equipment” indicates this.

Time Requirements

The time required to complete the research project shall be specified. Proposals will not be rejected if the proposed time does not match the time specified in the project statement. However, the agency must justify any difference. In addition, a schedule shall be included that shows each phase or task of the work, when that phase or task will begin, how long it will continue, and when it should end. The timetable should clearly delineate the points in time where project deliverables and reports are planned.

Itemized Budget

The estimated cost for the project should be based on the proposed performance period. The budget shall reflect phase and/or task costs. Lump sum estimates are not acceptable; budgets shall be itemized in accordance with the following cost categories where appropriate:

(a) Salaries and Wages. Each employee to participate in the performance of the project shall be identified by name, with role, level of effort, and cost presented in the format specified in the terms of Figure B-4.

(b) Borrowed Personnel. Reimbursement to other employers for salaries and wages paid by them to their employers released for, and directly engaged in, the performance of the subject research, plus federal and state payroll taxes and related employee benefit plan costs.

(d) Subcontract. Costs for services deemed necessary for performance of a portion of the research.

(e) Capital Equipment. Items with a value in excess of $500.00 per article or assembly required for the conduct of the research.

(f) Materials and Services. Materials, supplies, and other articles, including the cost of processing, testing, rental of apparatus and equipment from others, preparing, editing and reproducing reports, including the final report.

(g) Communications and Shipping. Long-distance telephone calls, postage, freight, etc.

(h) Travel. Transportation costs plus reasonable actual subsistence expenses

(i) Employee Benefit Plan Costs and Payroll Taxes. Costs of group insurance and employees’ pension and retirement plans federal and state payroll taxes for employees working directly on the subject research.

(j) Overhead. An allowance for overhead costs determined in accordance with the research agency’s usual method of accounting and generally accepted accounting principles.

Figure B-4: NCHRP Requirements for Tabulating Level of Effort in Proposals

Cooperative Features

If assistance in the form of personnel, data, or equipment is required from other agencies, public or private, describe the plans for obtaining such help or information. In the case where such cooperative features play an important part in the conduct of the research, a letter of intent from agencies agreeing to provide cooperative features should be included in the proposal.

Appendices

The appendices may include such things as statements concerning previous work on this problem or related problems, abstracts of related projects. A bibliography or list of references, or descriptive brochures or materials describing the agency’s organization and capabilities in general terms. Any other material not specifically mentioned previously and felt to be relevant for purposes of the proposal may be included as an appendix.

^{Review of Proposals and Contractor Selection}

The selection criteria used by different programs are very consistent, and typically consist of:

1. Understanding of the Problem - Has the investigator grasped what the problem and issues really are and not just reiterated the contents of the request for proposal?

2. Research Approach - Has the investigator stated clearly a research methodology with “whys”, “whats”, “hows”, and “whens” in sufficient detail, in terms of both work and budget, to assess the probability of success and monitor progress against prescribed tasks and milestones?

3. Staffing - Are those who will work on the project identified and their capabilities documented by reference to qualifications, experience and publications? Is the role and responsibility of each contributor clear, and is the contributor available for the time committed in the proposal?

4. Facilities and Equipment - Does the research team have access to all the facilities required to complete the work?

5. Application of Results - Does the proposal provide a realistic appraisal of the prospects for satisfying the objectives? Will the results be reported in a practical manner and is a feasible implementation plan provided?

Forms often provide space for the reviewer to make notes or comments, in addition to entering a score. This is very useful in enabling the reviewer to recall the strengths and weaknesses of individual proposals when they are discussed in committee, sometimes weeks after the review was made. The form used for the initial rating proposal by panel members in the NCHRP is shown in Figure B-5. Weights are assigned by the individual reviewer. In most cases, the greatest importance is attached to the caliber of the research principals and the adequacy of the research approach.

Figure B-5: An Example of a Form for the Initial Rating of Proposals

Task 1 would be described as in the previous example. Task 2 will probably consist of sub-tasks, each comprising a series of experiments. The experiments should be described in detail. This allows the principal investigator to ensure that the necessary people, equipment and laboratory facilities will be available to perform the work, and to calculate the cost of the work. This will usually be an iterative process that involves determining how many experiments can be performed within the available budget. It is essential that the principal investigator be confident that the experiments being performed will be sufficient to satisfy the objectives for the task. It is also good practice to identify a strategy to be followed if the experimental work is not completely successful, e.g. are there other test procedures that could be used? The work plan should describe clearly what would have been accomplished before Task 3 begins, the way in which the work in Task 3 will be influenced by the results obtained in Task 2, and the manner in which the results will be reported. The activities in Task 3 should be described in the same way as for Task 2, except that they will be more tentative because it is likely that many details cannot be finalized until results from Task 2 become available. Because it is a plan that is being written, it is inevitable that less detail can be provided for the later activities. One of the key roles of the principal investigator is to recognize when changes in the work plan are necessary, and to modify the plan accordingly. The good researcher will maintain flexibility in the work plan to deal with setbacks, and to seize opportunities, while planning in sufficient detail to estimate resources, and to make arrangements for activities which require a long lead time.

In the example given, Task 4 consists of developing a design for a field study to verify the results of Tasks 2 and 3 under real world conditions, not performing the field work itself. The specific details will depend on the results of Tasks 1 to 3, but the plan should contain the following sections:

· hypothesis to be tested on field studies

· criteria for selection of the study site

· potential sites

· equipment and other facilities required

· data collection and analysis procedures

· staffing plan

· schedule

· budget

The final report prepared in Task 5 should provide full documentation of the problem definition, experimental methods, results, conclusions and future activities, including implementation. Further information on the content and preparation of reports is given in Chapter Five.

In the section describing the project team, the team members should be listed, along with the responsibilities and time commitment for each person. This information is required to ensure that each person will be available to work on the project when required, and to establish costs. The key members of the team should be identified, but for junior positions where the work could be done by one of a number of technicians, it is sufficient to identify the person by rank, e.g. senior technician @ 200 hours, 3 junior technicians @ 300 hours each. The key members must be identified because if a substitution becomes necessary, management and the sponsor will need to satisfy themselves that the replacement has the necessary credentials and experience to perform the research. Curriculum vitae are not required in a work plan.

Equipment and facilities are listed for the same reason as personnel, i.e. to ensure that they will be available when needed and that all the costs are captured in the budget. This also forces the researcher to verify that all the equipment necessary to perform the planned experiments is available, or identify equipment and supplies that must be purchased.

If the research approach, project team, and the equipment are documented in sufficient detail, calculation of the budget for the project is relatively straightforward. Other items such as travel, overhead and profit must, of course, also be included.

A detailed schedule for the work is required to establish a completion date and to provide a mechanism for management to measure the progress of the work. It is also needed to identify when resources will be needed, and to calculate cash flow requirements. For projects that involve no more than about 10 tasks, and about an equal number of sub-tasks, a chart similar to that shown in Figure B-6 is perfectly adequate. Using the second study described above as an example, the chart shows the start and completion dates for each sub-task, and the anticipated progress of each sub-task, as a cumulative percentage. It also shows the interrelationship of the tasks and sub-tasks because it is clear which activities are underway concurrently, and which are sequential. The schedule allows one month for the review of the interim report, and two months to review the final report. The anticipated progress on the overall project can be calculated from a knowledge of the amount of effort required for each task and sub-task. The amount of effort required to complete each sub-task, expressed as a percentage of the effort required to complete the project, is shown in the third column. Taking the end of June 1996 as an example, the anticipated overall completion is:

Task 1 + 25% of sub-tasks 2a, 2b, and 2c

= 10% + 0.25 x (16% + 16% + 16%)

= 22%

By plotting the values from the row “Overall Completion” in Figure C-6 against duration of the project, a chart of anticipated progress is developed, as illustrated in Figure B-7. Using the same technique, planned expenditures can be plotted, as shown in Figure B-8, from knowledge of the budget and cash flow projections for each sub-task. Figures C-6 to C-8 are used to measure the progress of the research as described in the section “Execution of the Research”.

The Gantt chart provides a powerful visual representation of the work schedule and quite large research projects, typically up to $1million budget, can be managed in this way. Larger projects, especially those involving many semi-autonomous contributors, or requiring access to expensive equipment that is used on many projects, may require a more sophisticated work schedule to be developed. This can be done by preparing a critical path analysis or using computer software designed for project management.

The final section of the work plan, Implementation, describes how the findings of the research will be implemented. This enables the researchers to involve those who will be responsible for implementation in the research, and to allow the plans for the implementation activities to be developed while the research is in progress so that there is no delay between completion of the research and implementation of the findings.

Figure B-8: Anticipated Expenses

^{Execution of the Research}

As noted in Chapter Three, technical progress and financial expenditures must be tracked, and monitored, throughout the period of execution of the research. Charts similar to those illustrated in Figures B-6 to B-8 are suitable for this purpose. As an example, Figures B-9 to B-11 illustrate the status of the project described in the previous section as it might have been reported in December 1996. The data indicate that there were delays in beginning Task 2, but, except for work on the performance tests (Task 2c), the project was back on schedule.

Where a greater level of detail is required, as for example, where different budget codes must be reconciled, comparisons between planned and actual expenditures can be made by category, e.g. salaries, travel, and equipment. In this way, variations can be detected early in the process, and steps taken to bring the project under control.

Figure B-9: Completed Progress Schedule for December, 1996

Figure B-10: Overall Progress to December, 1996

Figure B-11: Expenditures to December, 1996

^{Implementation and Dissemination of Research Results}

An NCHRP study of technology transfer methods and implementation practices included a survey of transportation agencies and an analysis of case studies (10). Much of the information in this section is based on the data collected in the study, and from which the recommendations presented in Chapter Three were developed. Implementation “success” was defined in terms of timeliness, effectiveness, and scope of use. The study examined success from three perspectives:

· Characteristics of the research results

· Characteristics of the implementing organization

· Characteristics of the implementation process

From each perspective, the relative importances of factors that promote implementation, and those which act as barriers to implementation were rated, by participants in a workshop, on a scale of 1 to 5. The results are given in Tables B-1 to B-3 (the data were obtained from figures contained in reference 10). Barriers to implementation have been expressed on a scale of -1 to -5 (-5 being the greatest barrier).

Characteristics of Research Results

Some attributes of the research output can impede implementation. Most obviously, if the research does not meet the needs of potential users, the results will not be put into practice. Also, if users do not see evidence that a new product or process has been adequately tested, they may not want to be guinea pigs.

On the other hand, research results are more likely to be put into practice rapidly and effectively, if the researchers have accounted for real-world situations. In many cases, pilot or demonstration projects are useful steps toward full implementation. Their purpose is to evaluate the new products or processes under operational conditions, and make any necessary improvements, before full implementation on the in-service system. This is important to reduce the risk of failures. Even when full-scale implementation is adopted, the product or process may be designated experimental for a prescribed period to ensure that there is a systematic evaluation of the how well the changes are working.

Site selection for demonstration projects is much more complex than it might appear to be. The objective is to evaluate the new product or process under conditions, which are representative (with respect to site conditions, difficulty of work, contractor expertise, time of year) of the anticipated future applications while minimizing the risk of delay or failure. Frequently the work must be incorporated in contracts already awarded, or about to be tendered. Each has its disadvantages. Changes to existing contracts are often expensive and the contractor may not have the necessary expertise. Contracts not yet awarded are subject to shifts in award schedules, often resulting in projects being moved from one construction season to another. Rarely will there be an ideal candidate and the selection must be based on compromises between ideal and available conditions. The longer the time available to plan a demonstration project, the greater are the chances of locating site meeting a larger number of the selection criteria.

Table B-1: Factors Pertaining to the Characteristics of Research Results

Table B-2: Factors Pertaining to the Organization

Table B-3: Factors Pertaining to the Implementation Process

Characteristics of the Organization

The most important institutional barriers to implementation include organizational inertia, risk-averse behavior, management discomfort with change, and inadequacy of resources. As noted in the section describing the characteristics of research results, new products and processes entail a degree of risk. Some will prove not worth their cost, and some may not function effectively. The organization must be prepared to encourage staff to innovate, and to support them even when new ventures are not successful. This includes a commitment on the part of senior management to provide resources for implementation efforts and the willingness of senior officials to champion innovation.

Characteristics of the Implementation Process

This method involves obtaining the opinions of clients, often through technical advisory committees or project panels. Like expert opinion, client opinion is most commonly sought when problem statements or proposals for research projects are reviewed. Surveys are a useful method of obtaining client or user input in situations where there are numerous users, or the users are widely separated geographically.

Cost-Benefit Methods

Cost-benefit analysis seeks to assess the project in terms of the economic and social benefits generated for a particular “referent group” (e.g. an agency, a region or society as a whole) as well as the economic and social costs incurred by the referent group to carry out the research and implement the results. The definition of the referent group is important because it defines the bounds for the analysis. There are a number of general principles associated with conducting cost-benefit analyses of transportation-related research projects.

· There are three main types of costs associated with a research project that must be taken into account in the analysis. These are the costs of generating the research results, the costs of introducing and supplying the results to the end users, and the costs incurred by the end users to implement the results.

· The benefits of the research project are valued at the price society is willing to pay for them. The assessment of benefits includes not only those for which prices are paid, but also benefits associated with, for example, reduced environmental damage, better air quality and improvements in health and safety, even though such benefits may be difficult to quantify.

· Once they have been identified, all the benefits and costs associated with the project are compared using a common measuring system of constant dollar values. Allowances can be made for future changes in relative real prices, but no adjustments are made for the future changes in the general price level as a result of inflation or deflation. All costs and benefits must be discounted to their present values through a social discount rate before they can be compared to determine the net benefit of the project to society. The selection of the discount rate is controversial, and has a major impact on the results of the analysis. The discount rate is intended to be the social discount rate, which represents the present generation’s weighting of benefits and costs to be borne by future generations. In the private sector, the discount rate is sometimes taken to be the difference between the prime interest rate and the rate of inflation.

^Introduction

^{Problem Statement Development}