How to Revise a Research Paper and Respond to Reviewers

1. How to Revise a Research Paper and Respond to Reviewers Damon Chandler ECEN 6001, Spring 2009 Oklahoma State University

2. Outline • Understanding the review process • Digesting the reviews • Revising your paper • Communicating your revisionsto the reviewers and editor • Conclusions

6. The Need for Peer Reviews • Ensure quality, checking that no mistakes in logic have been made [Benos et al., 2003]

7. Submitted to Elsevier Health Series: Computer Methods and Programs in Biomedicine

9. The Need for Peer Reviews • Ensure quality, checking that no mistakes in logic have been made • Ensure that the work is original and significant • Ensure that the experimental methods are sound • Ensure that the results presented support the conclusions drawn • Ensure that no errors in citations to previous work have been made [Benos et al., 2003]

10. Outline • Understanding the review process • Digesting the reviews • Revising your paper • Communicating your revisions to the reviewers and editor • Conclusions

11. The Review Process • The paper is assigned to an editor • The editor selects reviewers • The reviewers are contacted and asked to review your paper • The reviewers review your paper • You receive the reviews, modify, and resubmit • The editor and the reviewers review your paper (again) • The editor makes his decision

12. An Invitation to Review Dear Dr. Chandler, As associate editor of the IEEE Transactions on Information Forensics and Security, I have received the above-referenced manuscript. I believe the topic is in your area of expertise, and hope that you are available to review it for me. The abstract is provided below for your reference. We ask that reviewers complete their evaluation within4 weeks of receipt.

13. IEEE Transactions Reviews • Ratings • Suitability of the topic • Content • Presentation • Overall Rating • Recommendation • Detailed Comments

14. IEEE Transactions Ratings I(a). Suitability of the topic: • Is the topic appropriate for publication in these transactions? • Is the topic important to colleagues working in the field?

15. IEEE Transactions Ratings I(b). Content: • Is the paper technically sound? • Is the coverage of the topic sufficiently comprehensive and balanced? • How would you describe the technical depth of the paper? • How would you rate the technical novelty of the paper?

16. IEEE Transactions Ratings I(c). Presentation: • How would you rate the overall organization of the paper? • Are the title and abstract satisfactory? • Is the length of the paper appropriate? • Are symbols, terms, and concepts adequately defined? • How do you rate the English usage? • Rate the Bibliography

17. IEEE Transactions Ratings I(d). Overall rating: • How would you rate the appropriateness of this paper for publication in this IEEE Transactions? • How would you rate the technical contents of the paper? • How would you rate the novelty of the paper? • How would you rate the literary presentation of the paper?

18. IEEE Trans. Recommendation

19. IEEE Trans. Detailed Comments • Please state why you rated the paper as you did in Sections I and II • This is the “meat” of the reviews • Typically, two paragraphs to multiple pages (some reviewers can get very detailed) • Your changes and responses will be based on these detailed comments

20. J. Optical Society Reviews • Review Criteria • Originality and significance • Accuracy and clarity • Appropriateness • Recommendation • Detailed Comments

21. J. Optical Society Review Criteria I(a). Originality and significance • Does the paper contain enough new material to warrant publication? • Does the paper contain enough significant/useful material to warrant publication? • Is the work placed in the proper context? • Are there adequate references?

22. J. Optical Society Review Criteria I(b). Accuracy and clarity • Is the scientific development sound? • Are the conclusions supported by the evidence? • Is the paper clearly written, and assumptions and procedures clearly stated? • Is the paper reasonably self-contained? • Are values given for important experimentalparameters? • Are the figures and tables effectively presented?

23. J. Optical Society Review Criteria I(c). Appropriateness • Is the paper appropriate for this journal? • Is there a more appropriate journal?

24. J. Optical Society Recommendation

25. J. Optical Society Detailed Comments • Sameformat as in other journals • This is the “meat” of the reviews • Typically, two paragraphs to multiple pages (some reviewers can get very detailed) • Your changes and responses will be based on these detailed comments

26. More on the Detailed Comments • Why the detailed comments? • Bordage, 2001: 151 research papers submitted to Research in Medical Education from 1997-1998 • 40% of papers received satisfactoryratings, but were still recommended for rejection • Detailed comments are a vital means of communication between the authors and reviewers

27. More on the Detailed Comments • Good detailed comments contain: • One-paragraph summary (this demonstrates that the reviewer understands the main points) • One paragraph on the positives • Multiple paragraphs on major negatives (or potential major negatives) • Listing of minor comments

28. More on the Detailed Comments • Damon’s Rule of Thumb: • Never write anything in a review that you wouldn’t say in person • Try to balance criticisms with encouragement: • “The author’s main conclusions are terrible.” • “I was impressed by the author’s novel experimental approach. However, I do not feel that the main conclusions are supported by the results.”

29. Summary of a Good Reviewer • The most helpful review is one that articulates the strengths of a paper while also assiduously identifying the limitations of the manuscript that can be addressed in a revision. • Nevertheless, even if a paper is well received overall, reviewer comments on manuscripts are commonly “negative.” [Roberts et al., 2004]

30. Outline • Understanding the review process • Digesting the reviews • Revising your paper • Communicating your revisions to the reviewers and editor • Conclusions

31. “During my editorship, no article submitted to the Psychological Bulletin has been accepted outright with no changes.” [R. J. Sternberg, Yale University]

32. Pushovers Demoters Number of Reviewers Zealots Assassins Mainstream Positive Review Reviewer’s Average Score on 10+ Papers Negative Review

33. Odds of Getting a Harsh Reviewer harsh reviewers Probability of Reviewer Positive Review Negative Review Reviewer’s Average Score

34. Odds of Getting a Harsh Reviewer

35. Digesting the Reviews Four-step process: • Read the reviews once, and then file them in a safe location • Don’t think about the reviews for at least a week (instead go skiing, golfing, etc.) • Read the reviews again • Discuss the reviews with your co-authors and create a plan-of-attack

36. Outline • Understanding the review process • Digesting the reviews • Revising your paper • Communicating your revisions to the reviewers and editor • Conclusions

37. Deciding What to Change • You must address all comments • You can’t pick-and-choose which comments to address • Even minor comments need to be addressed • Addressdoes not always mean change • You and your co-authors should decide what to change, and what to defend • Often, changing is the easiest route (demonstrates openness to suggestions)

38. Deciding What to Change • Change does not always mean revamp • Easy changes include: • Rewording • Adding extra references • Adding an extra paragraph, table, or figure • Adding an appendix • More difficult changes include: • Modifying your central hypothesis • Modifying your main algorithm • Redoing an experiment

39. Deciding What to Change • Always change technical errors • It’s the reviewer’s job to find these • Even minor errors can cast doubt • Always change errors in references • Skilled reviewers know the history better than newer authors • You don’t want to get off on the wrong foot with experts in the field by not citing the correct papers in the correct order

40. Deciding What to Change • Always change parts which yielded “I didn’t understand”-type comments • If the reviewer didn’t understand it, the readers might not either • The effort required to defend this point will be more than the effort required to change the paper • “I didn’t understand” is a polite way of saying “you didn’t explain clearly enough”

41. Deciding What to Change • Always change parts which are have been mentioned by multiple reviewers • If two or more reviewers make similar comments, the readers will likely have the same comments • Repeated comments stand out to the editor • The effort required to defend this point will be more than the effort required to change the paper

42. Revising the Paper • Divide the comments into two categories: • Easy changes • Difficult changes • Do the difficult changes first • This might take some time (especially if you need to repeat an experiment) • Easy changes might be eliminated • Consult with your co-authors on changes

43. Revising the Paper(Example of an Easy Change) Reviewer did not like the memoryless Gaussian assumption

44. Revising the Paper(Example of an Easy Change) Reviewer did not like the memoryless Gaussian assumption Quick fix: Use a distribution-dependent parameter

45. Revising the Paper(Example of an Easy, but Lengthy Change)

46. Revising the Paper(Example of a Difficult Change)

47. Outline • Understanding the review process • Digesting the reviews • Revising your paper • Communicating your revisions to the reviewers and editor • Conclusions

48. Communicating Your Changes • Letter to the editor • Summary of changes/defenses • Write this last • Letters to each of the reviewers • Responses to each comment • Write these first • Letter to typesetter (optional)

49. Letter to Editor(Opening Paragraphs) February 5, 2003 Dear Dr. Said: Enclosed please find documents which contain our responses to the reviewers of manuscript TIP-00290-2003 "Dynamic Contrast-Based Quantization for Lossy Wavelet Image Compression," D. M. Chandler and S. S. Hemami. We have modified the manuscript to directly address all of the comments of the reviewers. The modifications are detailed in the individual letters to the reviewers. (continued)

50. Letter to Editor(Summary of Changes) • To summarize our changes: • We have performed and added the results of a psychophysical experiment... (Reviewers 1, 2, and 4) • We have reworded our claims about the images produced... (Reviewers 2 and 4) • We have generalized our rate-distortion function to account for... (Reviewers 3 and 4) • We have added an appendix which shows the derivation of the approximation... (Reviewer 4) • (continued)