Instructions for Reviewers
Instructions to Referees
ITE publishes contributions in the following sections: Papers, Invited Articles, Notes, Cases, Games & Puzzles, and Reviews (textbooks, software, etc.), and you could be asked to referee a contribution in any one of these categories.
Once you receive a refereeing assignment for ITE, check your schedule to see whether you can complete the review within four weeks. If your schedule does not allow for this, contact the person who asked for the report and either negotiate a different schedule or ask for a reassignment of the paper.
Referees play significant roles in the "engineering" of an ITE paper. The primary role of an ITE referee is not helping make an accept/reject decision, but to help the authors improve their paper. The overriding concern is simple: "How can I increase the appeal and usefulness of this paper?" ITE is not primarily an archival journal; it is a journal for readers. Most authors do not publish in ITE for academic credit. Their goal is to impact OR/MS education. Our goal is to help them. Hence, we desire "developmental" reviews rather than mere judgmental ones. Nevertheless, we do want a recommendation. Typically there are four summary recommendations: publish as is (accept), minor revisions (conditional acceptance), major revisions (revise and resubmit), reject.
The recommendations and directions in the report are the most important if the referee recommends a major revision (a very common recommendation). The referee should clearly outline the shortcomings of the paper and offer advice on how to overcome them. Both sides (reviewers and authors) have to keep in mind that this is not a contract or an offer. Our job is to tell the authors how to get closer to perfection with the paper. It is up to them which recommendations they choose to follow and with how much vigour. After all, the authors are the ultimate decision-makers as to what their paper should contain, and we respect their priorities (but retain the right to make the publication decision). While we may well not require that authors follow every recommendation, we must identify those that are essential for publication (an example is an incomplete literature survey).
The first-level criterion for publication in ITE is simple: Does this paper help advance OR/MS education? (In other words, can some OR/MS educators deliver their material more effectively as a result of reading this paper?) The second-level criteria will depend on the nature of the paper. However, they are likely to include all or most of the following:
Given the lack of expertise in the OR community in publishing education-related articles, the referees have a significant role to play in judging what the final paper should contain. For example, a referee can recommend shortening a paper into a note or expanding a note into a full paper (where the distinction is based primarily on length-which should be proportional to content). Similarly, the referee may be able to identify a separate potential paper, or may notice that part of the paper can be built into a more interesting paper, or into a paper that has broader appeal. Instead of merely judging what has been submitted, it is important to evaluate the potential and offer direction to the authors.
While the referee report is most important in major revision recommendations, it is possible to provide a constructive report along with a rejection recommendation which may help the authors rewrite the paper for resubmission. The referee report can be important even for papers that are likely to be accepted-every paper can be improved.
The typical referee report will be one page in Word. In clear-cut cases half-page reports can be adequate, while in cases where there is potential but the paper requires a lot of work, longer reports are desirable. The referee can provide confidential comments to the AE and/or the Editor in addition to the comments in the referee report, which will be sent to the authors.
Instructions to Associate Editors
ITE publishes contributions in the following sections: Papers, Invited Articles, Notes, Cases, Games & Puzzles, and Reviews (textbooks, software, etc.). The Editor-in-Chief processes Invited Articles. Designated Area Editors deal with Cases, Games & Puzzles, and Reviews. Hence, these instructions are meant for Papers and Notes. (You may be asked to review a case, or a game by an Area Editor. You would then be acting as a referee, and not as an Associate Editor.)
When you receive an assignment from the Editor, check your schedule to see whether you can complete the review within six weeks. If your schedule does not allow for this, contact the Editor and either negotiate a different schedule or ask for a reassignment of the paper. You can also ask for a reassignment if you have provided the author feedback on the submitted article, or if you believe you are in a conflict of interest.
Before you send the paper out to referees, perform a first pass-a quick scan of the paper. Keep in mind that you have the authority to return a paper to the authors without referee reports (preempt-reject) if you notice a serious problem with the paper. In such cases, clearly outline the problem and, if possible, provide some guidance to the authors about how the paper could be improved or what might be a more appropriate outlet. This does not happen very often, but it does happen. I would recommend preempt-reject in cases where the authors have failed to follow a substantial portion of the instructions to authors or where the paper is not suitable for ITE.
Each paper and note should be peer-reviewed by two-to-four referees, depending on your level of comfort with the submission. We prefer that you do not act as a referee since this sends the author a skewed message with your opinion counting twice. You have to monitor the refereeing delays and in some cases you may have to complete your report with fewer referee reports than you had planned. If you feel uncomfortable about making a recommendation with the evidence on hand, consider getting a quick opinion from a colleague, another AE, or the Editor.
Please spend some time selecting your referees. We prefer a variety of opinions, and we should seek a diverse referee base. Diversify with respect to careers (academic vs. industry), university mission (research vs. teaching), with respect to faculties (business vs. engineering), with respect to seniority (senior vs. junior), and with respect to geography (North-American vs. non-North-American). You are encouraged to select junior faculty members (as well as Ph.D. students with some teaching experience). They are more likely to have the time and the energy to provide thoughtful and detailed reviews than senior faculty members burdened by numerous demands. Some papers (for example a paper on a better way to present X) should be refereed by textbook authors, while others (for example a survey or a tutorial on Y) may require expert opinion from an OR researcher. Papers that would be classified as traditional education research papers should be sent to at least one referee who is familiar with the relevant education research. While the web-based tool can automatically contact the referees you select, it is important to contact first-time referees by phone to make sure they understand the requirements and to increase the probability they will accept the task. The tracking tool will send regular reminders to referees. However, personal e-mail contacts may work better. Give the referees four weeks and send a reminder a week before the deadline (and another one, if necessary, on the deadline).
Once you have the referee reports, compose an AE report. The AE report should be more than a simple summary of the referee reports. The AE plays a very significant role in the "engineering" of an ITE paper. The primary role of an ITE AE is not helping make an accept/reject decision, but to help the authors improve their paper. The overriding concern is simple: "How can I increase the appeal and usefulness of this paper?" ITE is not primarily an archival journal; it is a journal for readers. Most authors do not publish in ITE for academic credit. Their goal is to impact OR/MS education. Our goal is to help them. Hence, we desire "developmental" reviews rather than mere judgmental ones. Nevertheless, we do want a recommendation. Typically there are four summary recommendations: publish as is (accept), minor revisions (conditional acceptance), major revisions (revise and resubmit), reject.
The AE report should summarize the paper as well as the referee reports. Then it should provide a recommendation. The recommendations and directions in the report are the most important if the AE recommends a major revision (a very common recommendation). The AE should clearly outline the shortcomings of the paper and offer advice on how to overcome them. Both sides (referees and authors) have to keep in mind that this is not a contract or an offer. Our job is to tell the authors how to get closer to perfection. It is up to them which recommendations they choose to follow and with how much vigour. After all, the authors are the ultimate decision-makers as to what their paper should contain, and we respect their priorities (but retain the right to make the publication decision). While we may well not require that authors follow every recommendation, the AE must identify those that are essential for publication (an example is an incomplete literature survey). Similarly, the AE should also identify which referee comments should not be followed.
The first-level criterion for publication in ITE is simple: Does this paper help advance OR/MS education? (In other words, can some OR/MS educators deliver their material more effectively as a result of reading this paper?) The second-level criteria will depend on the nature of the paper. However, they are likely to include all or most of the following:
Given the lack of expertise in the OR community in publishing education-related articles, the AEs have a significant role to play in judging what the final paper should contain. For example, an AE can recommend shortening a paper into a note or expanding a note into a full paper (where the distinction is based primarily on length-which should be proportional to content). Similarly, the AE may be able to identify a separate potential paper, or may notice that part of the paper can be built into a more interesting paper, or into a paper that has broader appeal. Instead of merely judging what has been submitted, it is important to evaluate the potential and offer direction to the authors.
While the AE report is most important in major revision recommendations, it is possible to provide a constructive AE report along with a rejection which may help the authors rewrite the paper for resubmission (assuming there is some potential). In such cases the paper may go through several rounds of refereeing before becoming publishable. The AE report can be important even for papers which are to be accepted with minor revisions-every paper can be improved.
It is important to keep in mind that the AE is not taking a simple vote from the referees to form a recommendation. The AE can make a recommendation that conflicts the opinion of the majority of the referees, as long as the report articulates the reasons and justifies the recommendation. The AE recommendation is more informed than the referee recommendations since the AE has the referee reports as well as his/her own opinion to form a recommendation. Note that the same principle applies to the decisions by the Editor. While the Editor is unlikely to go against the recommendation of an AE, the final publication decision rests with the Editor.
The typical AE report will be between one and two pages in Word. In clear-cut cases half-page reports can be adequate, while in cases where there is potential but the paper requires a lot of work, three or four page reports are possible. The AE can provide confidential comments to the Editor in addition to the comments in the AE report, which will be sent to the authors. The AE should send the names of the referees to the Editor for end-of-year acknowledgments.
It is important to keep your identity and those of the referees confidential. To delete user information from Word files, go to Tools-Options-User Information, and enter "Unknown User" as name and "UU" as Initials, before you save the document. You may want to cut-and-paste all referee reports into one document.
It is a good idea to send the AE report as well as all referee reports back to the referees. This will increase the transparency in the editorial process and help calibrate referee standards and expectations.
In cases of second or third round submissions, the AE should decide whom to send the paper for refereeing. If the paper has been revised significantly, then it should probably be sent to all first-round referees. The paper can be sent back to a subset of the referees depending on the contents of the first-round reports. For example, if Referee A found the first submission publishable while Referee B had some concerns, you may want to send the revision to Referee B only. Finally, in cases of minor revisions, you can review the paper yourself without sending it back to referees.