Causal Analysis in Qualitative Inquiry

1. Martyn Hammersley The Open University NCRM Research Methods Festival, St Catherine’s College, Oxford, July 2012 Causal Analysis in Qualitative Inquiry

2. Some Contrasting Attitudes Towards Causal Analysis Among Qualitative Researchers • The interesting case of Znaniecki and analytic induction. • The idea of unearthing causal relations through, or seeing them in, case studies. • The ambiguous example of grounded theorising. • Widespread denial of the relevance or possibility of causal analysis (Hammersley 2008).

3. In Practice, Most Qualitative Researchers Use Causal Analysis • Both within-case and cross-case data are employed, and often combined. • However, generally speaking, these are not used in a very systematic or explicit manner. • Indeed, the predominant mode seems to be a sort of pattern-matching, in which a fit between an explanatory idea and the internal features of some case, and/or a pattern across cases, is treated as conclusive evidence.

4. What is Required • Systematic development of models of causal processes on the basis of interpretation of evidence about particular cases, using both within- and cross-case analysis. • Systematic testing of these models, again using both within- and cross-case analysis. • While these two tasks will often be integrated, they are different and to some degree have divergent requirements.

5. A Model for Within-Case Analysis Lacey’s (1970) study of subculture formation among secondary school students offers one of the most sophisticated examples of within-case causal analysis. He documents: • Differences in attitude towards school values among fourth year secondary students • The development of such differences over time, and their reflection in peer groups • The complex relations between this subculture formation, processes of academic differentiation, and social class background.

6. Explaining Aggregate Differences in Outcomes • This has been a common goal, notably in the sociology of education. • Use of within-case analysis alone runs the risk of committing the compositional fallacy. • This fallacy is a counterpart to the ecological fallacy, which may occur through reliance upon cross-case analysis on its own.

7. Parallel Fallacies (Gomm 2012) • The ecological fallacy or, perhaps better, ‘the division fallacy’ = a false inference from a pattern found at the aggregate level to the assumption that it is also to be found within cases belonging to that aggregate. • The compositional fallacy = a false inference from some observed pattern found within one or a few cases to the conclusion that this will also be found at aggregate level, or that it explains some aggregate outcome.

8. Source of the Compositional Fallacy • Fundamentally, the compositional fallacy consists in a false assumption that there is substantial homogeneity across the cases belonging to an aggregate: that they all contribute in the same way to an aggregate outcome. In fact, heterogeneity amongst cases frequently means that they differ from one another in the sort of contribution they make. • Moreover, this means that we cannot always explain an aggregate pattern in terms of a pattern within cases, it is likely to result, at least in part, from differences among the cases.

9. An Example • As Roger Gomm (2012:98-9) points out, in the study of ethnic inequalities in educational achievement, researchers have often assumed that patterns found at the local level explain aggregate inequalities (see, for example, Gillborn and Gipps 1996). • The specific claim in much of this research is that aggregate inequalities are to be explained by teachers’ differential treatment of pupils from different ethnic groups. This may well be a factor, but it is not the only one.

11. Interpretation of the Table • These schools achieve different overall mean scores at GCSE, ranging from 18.9 to 65.0. • Forty eight percent of ethnic minority students are in the lowest achieving school. • We have assumed, for the sake of argument, that, in each school, minority ethnic students score 4 points less on average than ethnic majority students. • At LEA level the difference is actually 15.7 points: this leaves another 11.7 points of difference to explain. […] (Gomm 2012:106-7)

12. Qualitative Cross-Case Analysis • Cross-case analysis can take a variety of forms. • One of these is what might be called the ‘qualitative survey’. • An example: explaining social-class variation in applications to ‘high-status’ universities (Reay et al. 2005). These researchers relied on questionnaire data from 502 students, and interviews with 120 students, from 6 schools and colleges in the London area.

13. Weaknesses of Many Qualitative Surveys • The sampling strategies employed rarely offer much prospect of representative conclusions about the target population, and this population is not always clearly defined. • Actual frequencies are not given, instead there are vague claims about tendencies. • There is often little systematic comparison designed to check causal hypotheses. (See Cooper et al 2012:ch3)

14. Conclusion • Qualitative researchers cannot ignore the task of causal analysis. • Currently, for the most part, it is not carried out very effectively, but there are examples of good practice. • There is a need for more systematic approaches to both within-case and cross-case analysis. • And it is generally necessary to combine the two.

15. References Abbott, A. (2001) Time Matters, Chicago, University of Chicago Press. Cooper, B., Glaesser, J., Gomm, R., and Hammersley, M. (2012) Challenging the Qualitative-Quantitative Divide: Explorations in case-focused causal analysis, London, Continuum. Gillborn, D. and Gipps, C. (1996) Recent Research on the Achievements of Ethnic Minority Pupils, London, Office for Standards in Education, HMSO. Gomm, R. et al (eds.) (2000) Case Study Method, London, Sage. Gomm, R. (2012) ‘Qualitative causal analysis and the fallacies of composition and division: the example of ethnic inequalities in educational achievement’, in Cooper et al. Hammersley, M. (2008) ‘Causality as Conundrum: The Case of Qualitative Inquiry’ Methodological Innovations Online [Online], 2(3). Available at: http://www.pbs.plym.ac.uk/mi/pdf/Volume%202%20Issue%203/1.%20Hammersley%20-%201-5.pdf Jansen, Harrie (2010). ‘The Logic of Qualitative Survey Research and its Position in the Field of Social Research Methods’ [63 paragraphs]. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 11(2), Art. 11. http://nbn-resolving.de/urn:nbn:de:0114-fqs1002110 Lacey, C. (1970 Hightown Grammar, Manchester, Manchester University Press. Reay, D., David, M., and Ball, S. (2005) Degrees of Choice: Social class, race and gender in higher education, Stoke on Trent, Trentham Books.