WHAT IS RESEARCH INTEGRITY?

1. Martyn Hammersley The Open University, UK [Personal website: http://martynhammersley.wordpress.com/] Seminar School of Applied Social Studies/School of Education University College, Cork November 2018 WHAT IS RESEARCH INTEGRITY?

2. An influential and important concept • The term ‘research integrity’ has come to be widely used in recent years, especially in official documents concerned with the governance of research. (See e.g. IUA National Policy Statement; UK Research Integrity Office and Universities UK Concordat; Select Cmte 2018)  • This development is closely associated with the rise of ethical regulation, and the sense given to the term ‘research integrity’ often reflects this. • However, the concept of integrity is older. It goes back to Aristotle (Macfarlane 2009).

3. The meaning of ‘research integrity’ • Generally speaking, it seems to be used to refer to such matters as: honesty (for example avoidance of plagiarism), declaration of conflicts of interest, and commitment to research rigour. • In these terms, it complements ‘research ethics’ (see Hammersley and Traianou 2012). • However, sometimes, research integrity, or researcher integrity, is treated as incorporating research ethics. (For a useful discussion and references, see Banks 2015).

4. Clarifying research integrity • I will treat ‘research integrity’ as an overarching concept, as focused on the central values that should guide research, and how they are to be interpreted. • Using the term in this way, an important distinction must be drawn between epistemic values (such as truth) and ethical values. • In this talk I will concentrate on epistemic values, since, while these are emphasised in most discussions of research integrity, their implications are rarely spelt out in detail.

5. A contentious notion • The official literature on research integrity treats what this term means as a matter of agreement, but in many respects it is not. • For instance, one of the usually mentioned requirements is rigour, but what this involves is disputed between quantitative and qualitative researchers, and indeed amongst advocates of different varieties of qualitative enquiry. • In light of this, you need to be aware that my arguments here about the implications of research integrity may be contentious.

6. The importance of epistemic integrity • It has come to be argued that we live in a ‘post-truth’ world (see Leith 2017), where factual claims are made-up or distorted by advertisers, governments and politicians, the tabloid press, online ‘news’ sources, etc. • Commentators who find particular evidence not to their liking raise spurious questions about it. • Given this, it is essential that researchers try their best to meet the requirements of epistemic integrity. But doing so is not easy.

7. Scandals The most extreme breach of epistemic integrity is the invention of data. A number of researchers have been accused of this. For instance: • The British psychologist Cyril Burt (see Tucker 1997) • The American anthropologist Carlos Castaneda (see de Mille 1978, 1990) • The Dutch social psychologist DiederikStapel (see Levelt et al 2012) • The Dutch anthropologist Mart Bax (see Baud et al 2013).

8. Scandals are the tip of the iceberg • While the sort of breaches highlighted in these scandals are extremely important, it is not helpful to think of research integrity solely as a matter of choosing between right and wrong. • As in life generally, what is involved is often more complex and difficult: there are frequently problems in deciding what it would be best to do in particular situations, and there may not be any right answer, even though there are definitely wrong ones. • Official accounts of research integrity often don’t take account of this complexity.

9. Procedural and phrónētic conceptions of integrity • A great deal of the official literature on research integrity conceptualises it as a matter of compliance with ‘good research practice’ – formulated to a large extent as a set of procedures, rules, or principles. • However, the older philosophical notion of integrity focuses, instead, on making wise judgments, balancing relevant value principles as situationally appropriate (see Hammersley 2017); i.e. one must be a reflective practitioner.

10. Potentially conflicting commitments • The most effective way of pursuing a piece of research may not be ethical. • Some interpretations of ethical requirements could make much social and educational research nearly, if not actually, impossible. • As already noted, there are conflicts within the epistemic field arising from different paradigms or approaches. • There can also be conflicts amongst ethical values.

11. Research: A demanding task • The task of research is to provide findings about a topic that are more likely to be true than information from other sources. • This requires achieving some threshold of likely validity, and producing evidence that justifies accepting the findings as true. • Decisions about how to carry out research effectively have to be made under conditions of considerable uncertainty. • Research is a communal not an individual task: aimed at contributing to collective knowledge.

12. Areas where issues of epistemic integrity arise • Relations with ‘stakeholders’: funding bodies, universities, research team members, thesis supervisors, gatekeepers in the field, etc. • Selecting research topics and formulating research questions. • Reviewing the literature. • Identifying cases for investigation. • Collecting and producing data. • Analysing data. • Writing research reports and ‘dissemination’.

13. ‘Stakeholders’ • Whose responsibility is it that the research is pursued effectively? Ultimately, it is the researcher’s. Learning and advice from others are essential, but sometimes external demands may need to be resisted. • The case of doctoral students and supervisors: who has the primary responsibility here? • Relevant issues: the student generally chooses the topic and approach; the supervisor usually knows more about the research field; the problem of ‘paradigmatic commitments’; the problem of ‘external’ commitments.

14. Research questions • Are the questions worth addressing: Are they important? Do we already know the answers? But note the problem of ‘obvious answers’: Lazarsfeld’s ‘trick’. • Are the questions answerable by research? Even if they are, can they be answered given the time and other resources available? This requires making a judgment about what threshold of likely validity can be reached. • Also, note that research questions can change over the course of inquiry.

15. Lazarsfeld’s trick • In a review of a famous study carried out during the Second World War, a sociologist, Paul Lazarsfeld, noted that many readers were likely to dismiss its findings as obvious. • He outlined these findings – for example, ‘better educated soldiers showed more psycho-neurotic symptoms than less educated ones’ – noting that these findings could be inferred from what is already well-known (such as ‘the mental instability of the intellectual’). • BUT… (see Gage 1991)

16. Reviewing the literature • Searching for and reviewing the literature adequately is an important obligation. • It reduces the risk that one is trying to answer questions that have already been satisfactorily answered. • The literature will also reveal some of the complexities surrounding the issues with which one is concerned, both theoretical and methodological. • It is important to respect the literature but also to interrogate it.

17. Scope for disagreement ‘It is unscientific and unethical to embark on new research without first analysing systematically what can be learned from existing research’ (Chalmers 2005:469; emphasis added)

18. Identifying cases for study • One obligation here is to be clear in defining the cases being investigated. This is not always easy, because their boundaries may be fuzzy, and there may be cases within cases. • A second requirement is to think carefully about the relationship between cases selected and research questions, both open to change. • Usually some sort of generalisation is built into research findings, so how do the cases studied provide for this? How well do they do this? How many cases are needed?

19. Collecting and producing data • What data are required from the case(s) • What methods are to be used? There is an obligation to consider the range of available methods. It is also necessary to pay attention to the various ways that any particular method can be used. • There can be a tension between understanding the data, on the one hand, and interrogating and checking it, on the other. But both are required.

20. Analysis • Creativity versus scholarly caution. • What can be reasonably inferred from the data that is relevant to the research questions? • What method can best produce these inferences? • There is a responsibility to recognise and counter potential bias. • For instance, there is a need to resist sentimentalism, on the one hand, and to avoid an overly sceptical attitude, on the other.

21. The question of evidence In analysis as in reviewing the literature, careful distinctions need to be made. So, for instance, ‘there is no evidence for X’ can mean: • I have seen no evidence for it; or • I have searched for evidence, but found none • The evidence suggests not-X

22. Writing research reports • It is necessary to strive for sufficient clarity of argument. For instance, distinguishing between conclusions and guiding assumptions. • There is a danger of distorting or plagiarising sources (whether previous literature or data) • Sufficient information must be provided about how the research was done, and about potential threats to validity. • Sufficient documentation is required of the evidence on which the conclusions were based. • Is the task to present findings or to persuade?

23. Conclusion • Research integrity is very important, especially in a ‘post-truth age’. • However, it is not easily achieved. The notion of ‘moral luck’ (Williams 1981). Imperfection to some degree is the norm. Do one’s best! • There must be honesty about the failings of the research, but it is also necessary to recognise its value and strengths. • Here, as elsewhere, balanced, reflective judgment and discussion are necessary.

24. Some challenging questions • Is it legitimate to claim, in proposals and access negotiations, practical impact that is very unlikely to occur? • Is it legitimate to do research that can only tell us what happened in one case at a particular time? • Is it legitimate to publish findings that one knows will be misinterpreted: eg PISA stats? • Is it legitimate to gain funds for research when doing art work or political activism?

25. Bibliography Banks, S. (2015) ‘From research integrity to researcher integrity: issues of conduct, competence and commitment’, paper prepared for Academy of Social Sciences seminar on Virtue Ethics in the Practice and Review of Social Science Research, Friday 1st May 2015. Published in Emmerich, N. (ed.) Virtue Ethics and Social Science Research: Integrity, Governance and Practice, Bingley, Emerald Books, 2018. Baud, M., Legêne, S., and Pels, P. (2013) Circumventing Reality, Amsterdam, VU University. Available at: https://www.vu.nl/en/Images/20131112_Rapport_Commissie_Baud_Engelse_versie_definitief_tcm270-365093.pdf Chalmers, I. (2005) ‘Academia's failure to support systematic reviews’, The Lancet, 265, 9458, p469 de Mille, R. Castaneda’s Journey, London, Abacus. de Mille, R. (ed.) (1990) The Don Juan Papers, Belmont CA, Wadsworth. Gage, N. (1991) ‘The obviousness of social and educational research results’, Educational Researcher, 20, 1, pp10-16 (Reprinted in Hammersley, M. (ed.) 2007 Educational Research and Evidence-Based Practice, London, Sage.) Hammersley, M. (2018) ‘Is phrónēsis necessarily virtuous?’, in Emmerich, N. (ed.) Virtue Ethics in the Conduct and Governance of Social Science Research, Bingley, Emerald. Hammersley, M. (forthcoming) ‘Epistemic integrity in social research’ in Iphofen, R. (ed.) Handbook of Research Ethics and Scientific Integrity, Basingstoke, Palgrave-Macmillan. Available at: https://martynhammersley.wordpress.com Hammersley, M. and Traianou, A. (2012) Ethics in Qualitative Research, London, Sage.

26. Bibliography Contd. Israel, M. (2014) Research Ethics and Integrity for Social Scientists: Beyond Regulatory Compliance, London, Sage. Leith, S. (2017) ‘Nothing like the truth’, Times Literary Supplement. Available at: https://www.the-tls.co.uk/articles/public/post-truth-sam-leith/ Levelt, W., Drenth, P., and Noort, E. (2012) Flawed science: The fraudulent research practices of social psychologist DiederikStapel. Tilburg: Commissioned by the Tilburg University, University ofAmsterdam and the University of Groningen. Available at: https://www.tilburguniversity.edu/upload/3ff904d7-547b-40ae-85fe-bea38e05a34a_Final%20report%20Flawed%20Science.pdf Macfarlane, B. (2009) ResearchingwithIntegrity:theethicsofacademicinquiry, London, Routledge. Tucker, W. (1997) ‘Re-considering Burt: Beyond a reasonable doubt’, Journal of the History of the Behavioral Sciences, 33, 2, pp.145–162. House of Commons Science and Technology Committee (2018) Research integrity Sixth Report of Session 2017–19, available at: https://publications.parliament.uk/pa/cm201719/cmselect/cmsctech/350/350.pdf Williams, B. (1981) Moral Luck, Cambridge, Cambridge University Press.. For official views of research integrity see: http://ukrio.org/ and http://www.universitiesuk.ac.uk/policy-and-analysis/reports/Documents/2012/the-concordat-to-support-research-integrity.pdf; https://www.iua.ie/publication/view/national-policy-statement-on-ensuring-research-integrity-in-ireland/