A typical weak paper

1. A typical weak paper • Short and naïve introduction – demonstrates lack of background research and of expertise from the authors • Methods / algorithms not very original – demonstrates lack of understanding of the state-of-the-art in the field • Results show operation of system on one example case – lack of systematic study demonstrates laziness and greatly reduces belief that research described is generally applicable • Short discussion limited to own research rather than putting work into perspective by comparing to previous studies – shows lack of knowledge of others’ work and reduces credibility CS 597

2. A typical weak paper: summary A weak paper is one where the authors describe work that is not very new, is not thoroughly validated, and is not properly placed in perspective with respect to previous work. Mostly, it is weak because… the authors have been lazy and have not done proper background research. CS 597

3. A typical strong paper • Comprehensive expert introduction – demonstrates extensive background research, mastery of the whole field, understanding of the important issues, and clear positioning of the research as new • Methods / algorithms are original – demonstrates good understanding of the state-of-the-art in the field, and expertise • Results include thorough quantitative validation – the authors seriously stand behind their research and make efforts to prove how it is new / different / better • Discussion puts work into perspective by comparing to previous studies – shows expertise and credibility by not being shy about comparing to other research CS 597

4. A typical strong paper: summary A strong paper is one where the authors demonstrate that they have complete expert understanding of the major open research issues in the field, and where they provide a convincing argument that they just professionally cleaned up one of those issues. Mostly, it is strong because the authors know what they are doing! And because they worked extremely hard on their research. CS 597

5. Introduction; Originality • Your paper needs to bring something new; • But research is nowadays a very active field, and chances are that something very similar to your contribution has already been proposed 20 years ago. • Again, don’t try to mask that fact, hoping that reviewers will be ignorant. On the contrary, use extensive literature research to frame your work comparatively to previous studies. • It’s ok to write: • “XXX et al. (1972) have proposed […], an idea which we here sought to further explore…” • “Our approach is related to that of YYY et al. (1995), …” • “We follow the hypothesis of ZZZ et al. (1938) and investigate …” CS 597

6. Discussion • Don’t try to hide weaknesses; readers and especially reviewers are not naïve and will explicitly ask you about those weaknesses if you don’t appropriately discuss them – in the end, you just waste time. • It’s ok to have in your discussion statements like: • “a clear limitation of our approach is that in its present from it cannot …” • “our study has not addressed the problem of…” • “better methods have been proposed for [component X] of our system …” • Just be honest, fair, objective. CS 597

7. Video compression for human viewers • Many optimizations are possible and some are already implemented by modern video codecs: • Texture complexity • Spatial and temporal masking • Importance of motion • Human contrast sensitivity function • … [e.g., Osberger et al., 1997/1998/1999] • But some remain which are more difficult to implement: • Importance of humans, faces, animals, … • Biological motion • Object size • Main actor vs. secondary actors • Man-made vs. natural objects • … CS 597

8. Purpose of the study To evaluate applicability of a biological model of visual attention to automatic prioritization of video Systematic study of 64 variants of the model Demonstration using MPEG-1 and MPEG-4 Comparison with human eye movements Validation using large collection of video clips Key defining characteristic of the work: do not attempt to develop models for high-priority content (e.g., human faces); instead, develop model of the neurobiology of human visual processing. CS 597

9. CS 597 Itti & Koch, Nature Reviews Neurosci 2001

10. Comparison to eye movements • 50 video clips, 0:06 to 1:30 each, over 25 minutes total • Outdoors day & night, parks, crowds, rooftop bar, etc • TV news, sports, commercials, talk shows, etc • Video games, test stimuli • Task: “follow main actors and actions, try to understand overall what happens in each clip. We will ask you a question about main contents. Do not worry about details like specific text messages.” • 8 subjects total, 4 to 6 subjects / clip (4.7 avg) • Normal corrected vision, ages 23-32, 3 females, 5 males, mixed ethnicities • ISCAN tracker • 240Hz recording, 9-point calibration every 5 clips • Affine + thin-plate-spline calibration transform [Bookstein, 1989] • 640x480 @ 60.27Hz doublescan, 33.185ms/movie frame CS 597

11. CS 597

12. CS 597

13. Outdoors scene (easy) CS 597

14. CS 597

15. Video game CS 597

16. CS 597

17. TV, news (worst clip) CS 597

18. CS 597

19. CS 597

20. 50 clips 46,489 frames CS 597

21. 4.7 subjects/clip 50 clips Average blur at human eye = 1/2 x average blur over entire display p<0.01 for every clip using continuous blur map, And for 48/50 clips using 3 discrete foveas 46,489 frames CS 597

22. MPEG1: (average) Foveated compressed file size = 51% unfoveated compressed file size MPEG4: (average) Foveated compressed file size = 46% unfoveated compressed file size 50 clips 46,489 frames CS 597

23. Conclusions • Bottom-up salience based on simple interacting center-surround operators may explain much more than we often expect. • Video encoding priority predicted by simple salience model highly significantly correlates with human eye fixations, under our specific task. • Significant compressed file size reductions were achieved by automatic foveation. CS 597

24. The paper • June 30, 2003: Submitted to IEEE Transactions on Image Processing • October 15, 2003: reviews received – 2 liked it a lot, one liked the idea but not the writing. Out of typical editorial verdicts: • Reject • Revise and resubmit / major revision required with re-review • Minor changes required, usually without re-review • Accept as is ours is “revise and resubmit” • October 22, 2003: revision submitted along with answers to reviews • January 20, 2004: 2 reviewers liked the changes and recommend publication as is, one still wants a minor change. Paper status now: “accepted pending minor revision” • January 22, 2004: second revision submitted with answers • January 24, 2004: paper officially accepted • January 28, 2004: final version of text, separate figures, separate bibliography, author bio, etc sent to publisher. • June 24, 2004: official acceptance date (what happened Jan-June??) • October, 2004: publication CS 597

25. The reviews: first a few points of general feedback Reviewer 1 Comments: I. REVIEW Please expand and give details in Section III. A. Suitability of topic 1. Is the topic appropriate for publication in these transactions? (X) Yes ( ) Perhaps ( ) No 2. Is the topic important to colleagues working in the field? (X) Yes ( ) Moderately So ( ) No (explain: ) B. Content 1. Is the paper technically sound? (X) Yes ( ) No (why not? ) 2. Is the coverage of the topic sufficiently comprehensive and balanced? (X) Yes ( ) Important information is missing or superficially treated. ( ) Treatment somewhat unbalanced, but not seriously so. ( ) Certain parts significantly overstresses. [etc etc…] CS 597

26. The review: then a written commentary III. DETAILED COMMENTS Please state why you rated the paper as you did in Sections I and II. If you have indicated that revisions are required, please give the author specific guidance regarding those revisions, differentiating between optional and mandatory changes. This is an interesting approach and it is quite well described. The methods are essentially sound. The one concern I have with the approach is the nature of the instructions given to the subjects during the eye movement experiment, namely to 'not focus on small details.' This may not _guarantee_ the (favorable) outcome of the comparison with the model but it sure stacks the deck in favor of the model. This is, in fact, acknowledged by the author on p. 12 where he states that 'humans often fixated small details (...) instead of the most salient actor...despite the instructions given'. In other words, even though the subjects were explicitly instructed to behave like the model they did not always do so! [etc etc…] CS 597

27. Let’s write an answer • Here we start with some general evaluation of the reviews: • Key points to address here: • Reviewers always want more details, figures, validations, tests, … • Editors always want a shorter paper overall CS 597

28. CS 597

29. Is that a good answer? • It does make a convincing argument • It is polite • It answers the question well • It is detailed enough CS 597

30. Is that a good answer? • NO – explaining to the reviewer why s/he misinterpreted your writing or what you intended to write or how s/he is missing some general knowledge in the field is not at all what we are after. • If a reviewer makes a comment, no matter how irrelevant, besides the point, due to careless reading, etc. it may look to you, you must address it in the manuscript. • Remember: If one out of 3 reviewers made that comment, one out of 3 readers are likely to make it. • So, your answer should focus not on answering the question/comment, but on explaining how you have modified the paper such that this comment/question is addressed in the paper. CS 597

31. Explain how you modified the paper • Document exactly where you modified it so that reviewers • and editor can check it CS 597

32. Note: you do not have to agree and comply with every Reviewer request, but you must demonstrate evidence That you have done something about every request. CS 597

33. If the reviewer missed something that was explained In the paper, readers may miss it too. Always modify The paper to address that problem. CS 597

34. CS 597

35. CS 597

36. CS 597

37. If you are going to not comply with a request, be sure To back your decision up with some serious data! CS 597

38. CS 597

39. CS 597

40. CS 597

41. CS 597

42. CS 597

43. CS 597

44. CS 597

45. Some of the revisions… CS 597

46. CS 597

47. Revising the text is easy, but how about figures/data? • When you submit a paper, be sure to make a full archive of: • All the source code of all programs used • All the raw data • All the processing steps in perl/shell scripts • All the result analysis steps in perl/shell/matlab scripts • Etc • Etc • And of course remember the first commandment: “thou shalt never manipulate data with your bare hands” • Most reviewers will ask for revisions, no matter how good a paper is. You must be ready to address them! CS 597

48. CS 597

49. CS 597