1 / 86

Research Design

Research Design. David de Vaus Methods Festival, 2006 St Catherine’s College. The facts about illicit drugs speak for themselves. LET THE FACTS SPEAK FOR THEMSELVES March 28, 1986 Our reform began in the countryside … Some people don't like this policy.

eden
Download Presentation

Research Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research Design David de Vaus Methods Festival, 2006 St Catherine’s College

  2. The facts about illicit drugs speak for themselves

  3. LET THE FACTS SPEAK FOR THEMSELVES March 28, 1986 Our reform began in the countryside … Some people don't like this policy. Our approach is to allow people to hold differing views and to let the facts speak for themselves. David Lange, Former NZ Prime Minister

  4. “Let the facts speak for themselves” The contribution of agricultural crop biotechnology to American farming Produced on behalf of American Agri-Women American Soybean Association National Chicken Council National Corn Growers Association National Cotton Council National Milk Producers Federation National Potato Council National Turkey Federation United Soybean Board

  5. The facts speak for themselves, Mr Clarke Since 9/11, the British government has done a great deal to undermine the international rule of law. http://commentisfree.guardian.co.uk/philippe_sands/2006/04/the_facts_speak_for_themselves.html Let the Facts Speak for Themselves The UK Soil Association has released a report today purporting to show that the use of genetic engineering and biotechnology in US agriculture has been an unqualified disaster.

  6. Let the facts speak for themselves Catholic Health Australia urged politicians to look past "the spurious claims" made about embryonic stem cell research and human cloning. Embryonic stem cell research destroys human life and creates a dangerous precedent where one stage of human life is given priority over another."

  7. Facts are always … • Selective or partial • Ordered • Weighted • Interpreted • On their own they are Silent

  8. What do these facts mean? • 15% of those taking this trial drug died • Women have higher rates of religiousness than men • 50% of households in central Paris contain only one person • The country’s unemployment rate has increased from 4.9% to 6% over the last 12 months • Only a minority of families (38% ) are ‘traditional’ families (couple with dependent children). • This percentage has declined from 49% in 1976 to 38% in 2001 • Two thirds of divorces in 2005 were initiated by women

  9. Research design ultimately boils down to questions of interpretation or meaning of data

  10. Interpretation and meaning Context Comparison

  11. Research Design Reduce ambiguity when interpreting data

  12. Donald Campbell Plausible rival hypotheses Competing theories Corroboration via ‘ramification extinction’

  13. Is your interpretation more compelling than the alternatives?

  14. Plausible explanations • Observation: In month before CEO share options granted – fall in share prices • In month after options granted to CEOs – rise in share prices

  15. Plausible explanations • Good luck • Companies wait till price drop before issuing options (greater reward) • Low price creates most incentive for CEO to improve prices • CEOs manipulate market to reduce price before share offer due • Companies hold back good news until options allocated • Companies backdate issuing of options to maximise gain to CEO

  16. Plausibility ≠ ‘Proof’

  17. Example • Increase in divorce rates following introduction of no fault divorce legislation • No fault divorce  increase in divorce • Plausible rival explanations • NFD (leads to devaluing of marriage) • NFD is a response to increasing marriage breakdown • A sixties/seventies phenomenon • Any increase in divorce – simply a catch up in formalising ended marriages • Any increase – simply long term trend – would’ve happened anyway • Unique to particular country

  18. 1976 No fault Divorce Act

  19. Other Comparisons • Countries that did not introduce no fault divorce • Ireland • Countries that had no fault divorce throughout the period • Russia

  20. Design elements • Time: short vs long term • Trends • Before and after • Compared countries • Those with similar legislative changes • Those with no change • Absence of no fault • Always had no fault

  21. points • Comparisons help interpretation • Context (time and countries) help eliminate some plausible explanations • Narrow down alternatives • Facts don’t speak for themselves. • Context & comparisons are the translators

  22. Anticipate alternative explanations before collecting data • Design the collection to obtain context and comparisons • This will reduce ambiguity of data • Building these comparisons in = design

  23. Types of alternative explanations • Substantive • Alternative theory • Causal relationship? • Spuriousness • Direction of cause • Nature of cause (direct or indirect) • Multiple or single causation • Interaction effects • Selection effects

  24. Types of alternative explanations • Methodological • Internal Validity (design) • External validity • Measurement error • Validity • Reliability

  25. Types of comparisons • Between groups • Male cf female • Between units • Organisations • Countries • Regions • Between time points • Before/after • Individual unit over time (case study) • Compare groups over time

  26. point Comparisons must be valid comparisons Comparisons are designed to evaluate causal explanations

  27. Causal thinking • Explaining variance • Why does this group, category, person etc behave differently from other groups, categories, people? • Why did this person behave etc, this event occur etc

  28. Explaining variance • Account for variation in one phenomenon in terms of variation in other phenomena • A variable on which there is no variance cannot be the cause of variance in another variable • If education of a group is uniform then education cannot be the cause of income variations • If a variable exhibits variance and an ‘outcome’ variable exhibits no variance the variable with variance is not having any causal effect • If both men and women have similar income then gender is not affecting income

  29. Explaining variance • Why are women more religious than men? • Explain variation in religiousness • Why are so many people living alone (cf living in other arrangements)? • Why do women earn less than men? (Does variation in gender explain variation in income) • Why are people who cohabit before they marry have higher divorce rates (than those who don’t live together? • Why are men retiring earlier (now than in the past)?

  30. Isolating causes • Compare groups (etc) where • they are identical in all respects but one • Do they differ on specified outcome • Time sequencing • Must be plausible

  31. Experimental design T2 T1 Experim group Measure on Y (E1) Measure on Y (E2) ‘Treatment’ Diff=0 Diff=? Control group Measure on Y (C2) Measure on Y (C1) Treatment effect = (E1-C1) – (E2-C2) = 0 - ?

  32. Experimental design T2 T1 Experim group Measure on Y (E1) Measure on Y (E2) ‘Treatment’ Diff=? Control group Measure on Y (C2) Measure on Y (C1) Diff=? Treatment effect = (E2-E1) – (C2-C2) = ? - ?

  33. points Controlled comparison Only one difference  Only one explanation of different outcomes

  34. Longitudinal designs

  35. Purpose • Describing sequence of events • Helps identify (and eliminate) factors as possible causes • BUT: Sequence ≠ cause

  36. Exercise: Living alone • Lone person households are the fastest growing household type in Australia

  37. Lone person households, Australia, 1921-2021

  38. Plausible explanations

  39. Design to evaluate alternatives • Trends over time • Common to all countries? [select key country comparisons] • All age groups • Affluent and non affluent • Males and females • Marital status changes and of those who live alone • Compare those who live alone with those who do not live alone • Individual life courses [where do LA spells come] • Health • Other social relationships • Short term vs long term spells • Choice vs circumstance

  40. Longitudinal designs • Prospective • Panel • Cohort (e.g. birth, class of 48) • Broad sample (Household surveys BHPS) • Time series/ repeated surveys • Rates figures (Birth, divorce, marriage, crime) • British Attitudes Survey; ISSP; European Attitudes Survey • Census (although panel version is possible) • Retrospective • Collected at one point of time • Individual reconstructions • Life course events • Workforce history • Housing history • Parents characteristics • Social mobility studies

  41. points Time dimension Diachronic Longitudinal studies Capture change Establish sequence of events and  any causal order

  42. T2 T1

  43. T8 T7 T9 T6 T2 T10 T5 T1 T3 T4 Short or long term effect?

  44. T8 T7 T9 T6 T2 T10 T5 T1 T3 T4 Any post trend? Any pre trend?

More Related