^Appendix

_D

As noted in Chapter One, the result of the review of a researcher’s work by professional peers is a measure of one’s standing in the profession.  Peer review is the process commonly used by publishers to determine whether technical and scientific papers are worthy of publication.  This appendix describes briefly the process of peer review, the role of the reviewer and includes, as an example, the review forms used by the Transportation Research Board.

While most publishers and technical societies have developed their own procedures for the review of papers offered for publication, there are a number of common principles.  The process involves a detailed review of the paper by a minimum of two, and preferably three, professional peers knowledgeable in the subject of the research.  It is important that the reviewers be selected for their objectivity, especially when the findings reported are controversial.  In such cases, it is advantageous to increase the number of reviewers, and to select individuals who have different perspectives on the subject matter.  The names of the reviewers are not normally disclosed to the author.

Publishers are concerned about the quality of their journals and proceedings, and also about reaching the maximum number of readers within the defined audience.  The opinions of the reviewers are used to assist the publisher in deciding what to publish. The most important criteria are:

·                     Is the subject matter within the scope of the journal or conference proceedings?

·                     Is the subject of interest to a wide segment of the target audience?

·                     Is the research valid, and are the conclusions supported?

·                     Is the paper free from personalities, and special or commercial interests?

·                     Will the paper have lasting value?

·                     Is the paper clear, concise, and prepared in the required format?

Although scientific journals have formal peer review processes in place, this is not always the case with DOT’s. In general, DOT sponsored reports, documents, and research should be peer reviewed by a panel of experts, similar to what is done for professional journals. Technical Project Panels or Research Advisory Committees, composed of clients, topic experts, and influential technocrats do not necessarily contain scientific peers needed to conduct a quality review of sponsored research. One way to deal with this issue is to allocate a peer review period in all DOT sponsored research, where the DOT identifies experts on the particular topic to conduct the review. Another way to accomplish peer review is to require the contractor to identify a peer group, who the contractor will utilize in specified interim review periods. In general, peers who are compensated for their review time will conduct more detailed, thorough, and constructive reviews than will uncompensated reviewers. In addition, reviewers who remain anonymous to the contractor will generally provide more objective review comments—however sometimes they are not as constructive.

Reviewing the work of peers provides an opportunity to contribute to the improvement of the scientific and technical literature.  It also carries many obligations: to the publisher to provide an objective and timely review, to the author to be thorough and courteous, to oneself to be professional, and to the scientific community to support the integrity of the peer review process.

The reviewer should think of his or her role as providing assistance to the publisher, who must decide on publication, and to the author, in terms of making improvements.  Reviewers should not hide behind the cloak of anonymity to offer gratuitous, sarcastic or condescending comments.  Do not insult the author’s professional ability or language skills.  Do not make comments that are so general that the author would not be able to respond, e.g. “The paper needs to be rewritten.”  Be courteous and constructive.  A good rule is not to write anything that you would not be comfortable saying in a face-to-face meeting with the author.  Be specific in your comments, and explain deficiencies in polite language.  Be prepared to support your statements with examples.

It is helpful if comments are recorded in three categories: primary, secondary and editorial.  Primary comments are those which the reviewer thinks are mandatory, i.e. changes that must be made before the paper is considered acceptable for publication.  Secondary comments comprise suggestions by the reviewer, but changes resulting from the suggestions are left to the discretion of the author.  Editorial comments point out where corrections need to be made to spelling, grammar or sentence structure.  In making a final recommendation to the publisher, reviewers should remember that they are not being asked whether they agree with the content of a paper.  They are being asked to offer a professional opinion on the validity of the research, and whether the conclusions follow from the findings.

Both the publisher and author appreciate a through and constructive review.  Researchers strive to improve and, when the reviewers act professionally, the peer review process can be a very positive experience.

The review procedures followed by the Transportation Research Board are typical of those used by technical and professional societies.  Reviewers are asked to complete two forms (shown in Figures D-1 and D-2) according to the review criteria included in Figure D-2.  The purpose of the review is to determine the suitability of the paper for presentation, and for publication.  Form A is used by TRB to determine the disposition of the paper; Form B is used to convey the reviewer’s comments to the author.

Transportation Research Board                                               TRB Paper No. ______________

National Research Council

PAPER REVIEW: FORM A

Author(s)_________________________________

Title of Paper_________________________________

Committee________________________ Chair________________________Phone________________________

Reviewer________________________ Phone________________________ Date________________________

REVIEWER: The care and thoroughness of your review are the foundations of the quality of TRB

Records and the TRB Annual Meeting. Please take the time and effort in your review to serve the

author(s) and TRB well. Please contact the committee chair immediately if the topic is outside your

area of expertise or if you cannot meet the deadline for review comments. Please fill out and return two copies of Form A (and the manuscript if it contains review comments) to the committee chair, who will forward one copy to the TRB staff representative. Do not send a copy of this form to the author(s).

The following information will be used to reach decisions on presentation and/or publication. Please

include detailed comments on Form B regarding specific flaws that must be corrected.

Rating for Presentation: ___ 1. Excellent ___ 2. Good ___ 3. Fair ___ 4. Poor

Rating for Publication in a TRB Record: ___ 1. Excellent ___ 2. Good ___ 3. Fair ___ 4. Poor

Publication Recommendations:

___ 1. Publish as is or with minor modifications

___ 2. Request major revision and re-review

___ 3. Reject

Is paper an award candidate? ___ 1. Yes ___ 2. Perhaps ___ 3. No

COMMENTS:

NOTE: Adobe Acrobat Reader fill-in versions of this form are available at

http://www4.nationalacademies.org/trb/annual.nsf

Figure D-1:  Transportation Research Board Paper Review Form A

Transportation Research Board                                               TRB Paper No. ______________

National Research Council

PAPER REVIEW: FORM B

Author(s) _________________________________________________________________

Title of Paper_______________________________________________________________

REVIEWER: Please provide detailed comments on this form. Return three copies of Form B (and the

manuscript if it contains review comments) to the committee chair, who will forward one copy to the TRB

staff representative. The committee chair will send the corresponding author a copy of this form or a

summary of information submitted by all reviewers; in no case will the reviewers be identified.

Attach additional pages to provide specific comments that:

1) Support your Form A recommendations;

2) Explain any “no” entries shown below;

3) Help the author(s) improve the paper;

4) Identify which flaws must be remedied; and

5) Indicate if you are aware of publication of this paper elsewhere.

Review criteria (presentation and publication):                                    YES    NO

1) Significant contribution to state-of-the-art or practice .................................        ____     ____

2) Content of the paper is original and timely .................................................      ____     ____

3) Coverage of the subject is complete and well organized ...........................         ____     ____

4) Data are valid and research methods are appropriate ...............................         ____     ____

5) Conclusions are valid and properly supported............................................        ____     ____

6) Paper is useful to practitioners....................................................................    ____     ____

7) Paper is useful to researchers ....................................................................    ____     ____

8) Free of sensitive statements advocating special interests,         

advertising, and government policies and programs ..................................                        ____     ____

Additional criteria for publication:

9) Abstract conveys the meaning of the paper ...............................................       ____     ____

10) Written in simple, concise, and effective language.....................................      ____     ____

11) Long-term value as a research reference or as a description of practice ..          ____     ____

12) Final report on the study (not interim or task reports).................................      ____     ____

13) Subject of interest to a large audience .......................................................    ____     ____

COMMENTS:

NOTE: Adobe Acrobat Reader fill-in versions of this form are available at

http://www4.nationalacademies.org/trb/annual.nsf

Figure D-2:  Transportation Research Board Paper Review Form B and Review Criteria