Presentation Transcript

1. Harvard Summer School, 2013Graduate Research Methods and Scholarly Writing in the Social Sciences: Government and HistoryHarvard Summer School: SSCI S-100b Section 2 (32761) Joe BondClass 2June 26, 2013
2. Agenda Announcements Discuss Tonight’s Reading Definitions and Transparency/Documentation in Research Research Methods Content Analysis 2nd In-Class Writing Assignment Literature Reviews (Maybe)
3. Announcements Class format/office hours, etc. Guidelines for the second paper are posted on the course website I will review and comment on any drafts received by noon on Friday Determining if a journal is peer reviewed, book review samples, lead times in publication
4. Tonight’s Reading Reading and Understanding Research
5. Authoritative Numbers: Bullying in School Journal of the American Medical Association (JAMA) study The Study (random sample of approximately 16,000 students) Nearly 1 in 3 U.S. children in 6th through 10th grades have been bullied, or bully other students. SHOCKING, YOU SAY! How do they define bullying? Definition (buried in an appendix): “We say a student is being bullied when another student, or a group of students, say or do nasty things and unpleasant things to him or her. It is also bullying when a student is teased repeatedly in a way he or she doesn’t like. But it is not bullying when two students of about the same strength quarrel or fight.” (emphasis in original)
6. Your Risk of Divorce What proportion of marriages end in divorce? No one official agency keeps track Filing a marriage license and obtaining a divorce are both legal steps. Divide the number of divorces during a particular year by the number of marriages during that year to get a rough measure of the likelihood of marriage ending in divorce. Since the 1960s, the number of divorces has been nearly half that of marriages (referred to as the divorce rate).
7. The Problem with Divorce Rates Example: crime rates Crime rate calculated per 100,000 people (murder rate is 5.5 per 100,000 in the U.S., according to the FBI). But who makes up the population at risk when we try to calculate divorce rates? Obviously, it doesn’t just include those married during the same year. In fact, we know that relatively few couples get divorced during the calendar year in which they marry.
8. Divorce, Continued The population at risk for divorce is all married couples. Say we record 2 marriages in a year on the isle of Bliss and one of those marriages ends in divorce, yielding a 50% divorce rate. But cohorts matter. Divorce data derived from a sample rather than a population as a whole paints a more accurate picture. There are important cohort differences that reveal how society has changed. Basically, people in each cohort were likely to have remained married longer than those in the cohort that followed.
9. Divorce Rates by Cohort 35% of 60-year-old men had their first marriage end in divorce 40% of 50-year-old men had their first marriage end in divorce Of women who first married during 1945-1949, 70% were married 30 years later Only 55% of those married for the first time during 1960-1964 remained married What proportion of couples first married during 1980-1985 will celebrate their 30th wedding anniversary? Too early to tell. Projection based on records so far: Only 73% of women who wed during those years were still married 10 years later, compared to 90% of those first married in 1945-1949 whose marriages lasted at least 10 years Based on these data, investigators projected that, while a larger proportion of earlier marriages remain intact, about half of recent marriages will end in divorce. Key Point: estimating divorce rates is a fairly complicated matter
10. Body Counts Iraq
11. Examples of Five Research Methods (far from exhaustive) Experimental Correlation Natural Observation Survey Case Study
12. Experiments Researcher manipulates a variable (anything that can vary) under highly controlled conditions to see if this produces change in a second variable The variable, or variables, that the researcher manipulates is the independent variable(s) while the other variable, the one measured for changes, is the dependent variable.
13. Strengths: Experiments Experiments help us in understanding relationships (about as close to cause and effect as one can get) BUT Must be sure that manipulation of an independent variable is the only variable having an effect on the dependent variable. S/he does this by holding all other variables constant or equivalent (control variables) See quasi-experimental design in Creswell
14. Weaknesses: Experiments Experiments can only be used when it is practical and ethical for the researcher to manipulate the antecedent conditions (e.g. Tuskegee experiments - 399 black sharecroppers with syphilis (1932 -1972), Stanley Milgram’s experiments, etc.) Experiments are usually done in the highly controlled setting of the laboratory. These conditions are artificial and may not reflect what really happens in the less controlled and infinitely more complex real world (counterfactual thought experiments an exception) Randomization is often problematic, particularly in medical studies
15. Can one's biochemistry be, at least in part, responsible for aggressive or violent behavior? A biologist at Yale, attempted to answer this question by manipulating the biochemistry of group prison inmates Previous studies suggested that lower levels of serotonin was associated with increased aggression [and depression] and, conversely, higher levels of serotonin caused decreases in aggressive responses Sheard(1995) injected a group of prisoners (the experimental group) with lithium carbonate, known to enhance serotonin brain activity Another group (the control) of similar prisoners were given a placebo The lithium injections significantly reduced aggressive behavior in prisoners over a four month trial
16. Correlation Approach Correlation is classified as a non-experimental method because variables are not directly manipulated It is more of a statistical technique. These studies are designed to determine the strength and direction of a relationship between two or more variables
17. Strengths: Correlation It can be used to determine if there is a relationship between two variables without having to directly manipulate those variables It can be used when it is impractical and/or unethical to manipulate the variables It can be used as a basis for prediction. For instance, if we know that two variables are highly correlated, say +.85, we can predict the value of the dependent variable by knowing the value of the independent variable (e.g. SAT scores and success in college)
18. Weaknesses: Correlation Correlation does not tell researchers whether or not the relationship is causal [e.g. drowning and ice cream consumption or children’s shoe size and math skills] Since correlation does not, cannot, prove causation, we never know for sure If we can’t disprove it, it supports [but does not prove] our hypothesis
19. Natural Observation Field studies; non-experimental approaches used in the field or in real-life settings The researcher very carefully observes and records some behavior or phenomenon, sometimes over a prolonged period, in its natural setting This usually involves observing humans or animals as they go about their activities in real life settings Nickel and Dimed: On (Not) Getting by in America (Barbara Ehrenreich; 2002)
20. Strengths: Natural Observation It allows the researcher to observe behavior in the setting in which it normally occurs rather than the artificial and limited setting of the laboratory It might validate some laboratory finding or theoretical concept
21. Weaknesses: Natural Observation This is a descriptive method, not an explanatory one. The behavior can only be described, not explained It can take a great amount of time It is sometimes difficult to observe behavior without disrupting it and the difficulty of coding results in a manner appropriate for statistical analysis
22. What is the impact of having a label (stigma) attached to one's self. How does this label influence others' perceptions of and responses to the stigmatized person? 1973 (before One Flew over the Cuckoos Nest), 3 women, 5 men, 12 hospitals in 5 states A doctor (Rosenhan) had himself and eight other volunteers admitted to mental hospitals throughout the United states by faking symptoms of schizophrenia Once admitted, the pseudo-patients immediately began behaving "normal” Despite the normal behavior, none of the patients were recognized as "sane" Their stays ranged from 7-52 days with an average stay of 19 days To get released, they had to agree to take antipsychotic meds Normal behaviors were interpreted to fit the label (e.g. the pseudo-patients took extensive notes while they were on the wards and in the open. This behavior was seen as an aspect of their pathological behavior
23. Survey Research Usually not an end-product Typically used as input for statistical analyses (correlation analysis) Administering Surveys (refer to Trochim): Telephone Mail Online surveys Personal in-home survey Personal mall intercept survey Response Rates Items (the fewer, the better, avoid negations, etc.) Constructing and administering a good survey is not simply a matter of coming up with a series of questions Pastoral Conflict Survey (bride price, elder blessing, etc.)
24. Case Studies (more on this later) In-depth exploration of an event, an activity, a process, of one or more individuals. Psychobiography: The Colonial House by George & George) Anna & Isabella: Case studies of Feral children (Kingsley Davis) The Extreme Social Isolation of a Child (1960), and its follow-up, A Final Note on a Case of Extreme Isolation (1993)
25. Content Analysis
26. Content Analysis Content Analysis is a systematic attempt to examine some form of verbal or visual communication such as newspapers, diaries, letters, speeches, movies, or television. Can be inductive or deductive. Objective is to classify content Can be either qualitative or quantitative (e.g. frequency counts). Manifest Content: what explicitly appears in a text. Latent Content: meanings implied by the written content that do not actually appear in the text.
27. Examples Content Analysis of Video Games Look at “E” (like rated G) games and look for violence, killing, and the use of weapons in the course of normal play. Analyze The Daily Show and Assess for: % of the stories addressing political topics % of the stories addressing a public policy theme % of the stories addressing international news in some way % of the stories having to do with the news media % of the stories that address celebrity/entertainment news % of the guests who could be labeled serious (e.g. politicians, government officials, authors, etc.) % of stories that involved traditional news media or video footage vs. % coming from network of cable news shows
28. Group Exercise Identify as many themes as you can associated with each ad Rate the intensity of each theme as it corresponds to that particular ad
29. Verbal Behavior Analysis Verbal Behavior Analysis (VBA) is a content analytic technique designed to tap "styles of speaking with patterns of thinking and behaving" (Weintraub, 1989: 7). Weintraub devised a system to analyze samples of speech (monologues elicited by a standardized procedure) to obtain the frequencies of occurrence of members of fourteen categories, not all of which are, strictly speaking, "syntactic" (e.g. long pauses and the rate of speech). Then groups representing "no pathology" and various psychopathological syndromes are compared with respect to the frequencies with which these categories appear in speech.
30. VBA, Continued Fifteen indicators: 1) I, 2) We, 3) Me, 4) Negatives, 5) Qualifiers, 6) Retractors, 7) Direct References, 8) Explainers, 9) Expressions of Feeling, 10) Evaluators, 11) Adverbial Intensifiers, 12) Non-personal References, 13) Creative Expressions, 14) Rhetorical Questions, and 15) Interruptions
31. VBA, Continued frequent use of evaluators are indicative of individuals possessing a punitive conscience frequent use of retractors convey impulsivity high adverbial intensifier scores indicate persons "who see the world in black and white terms;" frequent use of explainers signify tendencies toward rationalization high qualifiers scores indicate anxiety and avoidance to commitment frequent use of negatives signify negation and denial frequent use of rhetorical questions indicate aggressiveness frequent use of direct references indicates that the speaker has difficulty speaking and prefers to divert the attention of the audience low use expressions of feeling convey an impression of aloofness frequent use of creative expressions indicate creativity
32. Quick PT Analyzer
33. Content Analysis of Newsfeed
34. Integrated Data for Events Analysis (IDEA): A Third Generation Event Framework Most domain specific events data frameworks have been or can be mapped to IDEA, including WEIS, PANDA, MIDS, etc. IDEA is a multi-framework compatible data standard designed to facilitate the comparison of data developed by different conceptual frameworks The IDEA framework is currently comprised of 249 social, economic, environmental and political events

35. IDEA Class Hierarchy (N=249)

    Level 0: All phenomena Level 1: Animal incidents, Human actions, Human conditions, Other incidents Level 2: Animal attack, Accident, Accuse, Animal death, Agree, Animal illness, Cognitive state, Complain, Consult, Human death, Demand, Demonstrate, Deny,Economic activity, Endorse, Economic status, Expel, Force Use, Grant,Human illness, Natural disaster, Other animal incident, Other human action, Other human condition, Other incident, A&E Performance, Promise, Propose, Reject, Request, Reward, Comment, Sanction, Seize, Sports contest, Threaten, Warn, Yield Level 3: Abduction, Affective state, Agree or accept, Alerts, Promise to mediate, Apologize, Arrest and detention, Ask for material aid, Request protection, Assure,Beliefs and values, Criticize or denounce,Balance of payments, Break relations, Call for action, Unconventional weapons attack, Acknowledge responsibility, Extreme climactic condition, Collaborate, Crowd control, Commodity prices, Demand aid, Decline comment, Demand ceasefire, Default on payment, Defy norms, Demand information, Demand mediation, Demand meeting, Demand protection, peacekeeping, Demand policy support, Demand rights, Demand withdrawal, Discussion, Drought, Declare war, Corporate Earnings, Earthquake, Ease sanctions, Extend economic aid, Extend humanitarian aid, Extend military aid, Empathize, Equity prices, Exchange rates, Formally complain, Flood, Forgive, Executive adjustment, Halt discussions, Infectious human illness, Host a meeting, Hurricane, Informally complain, Improve relations, Interest rates, Extend invitation, Judicial actions, Armed force blockade, Armed force mobilization, Armed force display, Covert monitoring, Armed force threats, Non-infectious human illness, Other physical force threats, Radioactive leak, Optimistic comment, Physical assault, Pessimistic comment, Protest demonstrations, Praise, Promise material support, Promise policy support, Offer to mediate, Offer to Negotiate, Offer peace proposal, Armed actions, Refuse to allow, Ratify a decision,Real estate prices, Reduce or stop aid, Reduce routine activity, Release or return, Currency reserves, Riot Reject proposal, Investigate Seize possession, Provide shelter, Solicit support, Hazardous material spill, Rally support, Strikes and boycotts, Tornado, Transactions, Sanctions threat, Tsunami, Non-specific threats, Give ultimatum, Travel to meet, Volcano, Elect representative, Wildfire, Yield to order, Yield position Level 4:Armed force air display, Missile attack, Ask for economic aid, Ask for humanitarian aid, Ask for armed assistance, Assassination, Agree to mediation, Agree to negotiate, Agree to peacekeeping, Agree to settlement, Impose restrictions, Beatings, Border fortification, Break law, Armed force border violation, Chem-bio attack,Private transactions, Private default on payments, Impose censorship, Armed battle, Bodily punishment, Coups and mutinies, Criminal arrests, De-mining, Demobilize armed forces, Earnings above expectations, Earnings below expectations, Ease economic sanctions, Ease military blockade, Equity prices down, Equity prices up, Evacuate victims, Grant asylum, Government transactions, Government default on payments, Artillery attack, Reduce or stop humanitarian assistance, Halt negotiation, Halt mediation, Reduce or stop economic assistance, Political flight, Reduce or stop military assistance, Reduce or stop peacekeeping, Hostage taking and kidnapping, Investigate human rights abuses, Security alert, Downward trend in interest rates, Investigate war crimes, Upward trend in interest rates, Hijacking, Torture, Armed force alert, Mediate talks, Mine explosion, Armed force activation, Armed force occupation, Armed force naval display, Engage in negotiation, Nuclear alert or test, Disclose information, Protest altruism, Small arms attack, Protest procession, Political arrests, Protest obstruction, Protest defacement, Promise economic support, Promise humanitarian support, Promise military support, Reject ceasefire, Relax curfew, Request mediation, Request an investigation, Reject mediation, Nuclear attack, Reject peacekeeping, Reject proposal to meet, Relax censorship, Reject request for material aid, Return, release person(s), Return, release property, Relax administrative sanction, Reject settlement, Request withdrawal or ceasefire, Suicide bombing, Sexual assault, Threaten forceful attack, Threaten forceful blockade, Threaten to boycott or embargo, Threaten biological or chemical attack, Armed force troops display, Threaten to halt negotiations, Threaten to halt mediation, Threaten nuclear attack, Threaten forceful occupation, Threaten to reduce or break relations, Threaten to reduce or stop aid, Observe truce, Threaten war, Vehicle bombing, Veto

36. Selected Branches (IDEA Event Framework)

    Allphenomena HumanactionHumanconditionsAnimalincidentsOtherincidents Other human conditionHumanillnessEconomicstatusHuman deathCognitivestate Infectious human Non-infectious Affective stateBeliefs illnesshuman illness and values BalanceCommodityDebtEquityExchangeRealCurrency reserves of paymentspricesyieldspricesratesestate prices Red signifies a terminal event (i.e., the lowest node on a branch)
37. Basic Structure of Extracting Meaning from a Report Subject Verb Direct Object/Indirect Object Examples: US President George W. BushblastedIraqi President Saddam Hussein for defying the no-fly zone. The U.S.delivered$20 million of humanitarian aid to Afghanistan yesterday. Subject (source), Verb (event), Direct Object/Indirect Object (target)
38. Variables Coded from Each Clause 1) ID (auto-generated, unique ID, 2) Sentence ID, 3) Event ID, 3) Event Date, 4) Report Date, 5) Event Place (e.g. Baku), 6) Event Administration (e.g. Azerbaijan), 7) Source Value (e.g. President IlhamAliyev 's grip), 8) Source Name (e.g. AZJ), 9) Source Administration (AZJ), 10) Source Level (e.g. INDI), 11) Source Sector (e.g. NEXE), 12) Event Negated, 13) Event Status (e.g. past, ongoing, foreshadowing), 14) Event Type (e.g. conflict/cooperation), 15) Is Flagged (i.e. pre-defined search terms), 16) Event Form (i.e. IDEA code, 17) Event Value (literal value), 18) Target Value (literal value), 19) Target Name, 20) Target Admin, 21) Target Level, 22) Target Sector, 23) Information Value, 24) Information Name, 25) Information Admin, 26) Information Level, 27) Information Sector, 28) Locus, 29) Affect, 30) Mechanism, 31) Injury, 32) Damage From these 32 variables we create 50 + additional variables in a post-parse process

39. Example Description of an IDEA Event Form

    IDEA Event Code: 2122 Name: Criminal arrests and detentions Description: Arrests and detentions explicitly characterized as criminal Usage Notes: Example*: French police on Tuesday arrested a man trying to sneak through Paris airport customs with a boa snake hidden in his underpants, an airport spokeswoman said. *Source = blue, Event = red and Target = green

40. Wordnet’s 15 Senses for the Verb “kill”)

    1. kill -- (cause to die; put to death, usually intentionally or knowingly; "This man killed several people when he tried to rob a bank"; "The farmer killed a pig for the holidays")2. kill, defeat, vote down, vote out -- (thwart the passage of; "kill a motion")3. kill -- (cause the death of, without intention; "She was killed in the collision of three cars")4. stamp out, kill -- (end or extinguish by forceful means; "Stamp out poverty!")5. kill -- (be fatal; "cigarettes kill"; "drunken driving kills")6. kill -- (be the source of great pain for; "These new shoes are killing me!")7. kill -- (overwhelm with hilarity, pleasure, or admiration; "The comedian was so funny, he was killing me!")8. kill -- (hit with so much force as to make a return impossible, in racket games; "She killed the ball")9. kill -- (hit with great force, in sports; "He killed the ball")10. kill -- (deprive of life; "AIDS has killed thousands in Africa")11. toss off, bolt down, belt down, pour down, down, drink down, kill -- (drink down entirely; "He downed three martinis before dinner"; "She killed a bottle of brandy that night")12. kill, obliterate, wipe out -- (mark for deletion, rub off, or erase, as of writings; "kill these lines in the President's speech")13. kill -- (tire out completely; "The daily stress of her work is killing her")14. kill -- (cause to cease operating; "kill the engine")15. kill -- (destroy a vitally essential quality of or in; "Eating artichokes kills the taste of all other foods") SeeWordNet
41. Sense 2 "Hypernyms (this is one way to...)" of verb "kill" Sense 2 kill, defeat, vote down, vote out -- (thwart the passage of; "kill a motion veto, blackball  -- (vote against; refuse to endorse; refuse to assent) oppose, controvert, contradict - (be resistant to; "The board opposed his motion.“ refute, rebut - (overthrow by argument, evidence, or proof; “the speaker refuted his opponent's arguments") renounce, repudiate (cast off or disown; "She renounced her husband“) reject -- (refuse to accept or acknowledge; "I reject the idea of starting a war"; "The journal rejected the student's paper") judge -- (form an opinion of or pass judgment on)
42. Sense 2: Container (synset) of verb "kill" judge reject renounce; repudiate refute; rebut oppose; controvert; contradict veto; blackball kill; defeat; vote down; vote out
43. Semantic Framework: Noun Classes Top Level All agents Level 1 True agents (political actors) Pseudo agents (other actors, like the environment) Level 2 Civil society agents, Government agents Intangible things, Tangible things Level 3 Armed civilian groups, Artists, Athletes, Body parts, Communication, Events, Human actions, Businesses, Candidates, Civic group agents, Human artifacts, Human cognition, Human attitudes, Criminals, Detainees, Diplomats, Educators, Natural environment, Status, Time-related Ethnic agents, Farmers, Health care agents, phenomena Judiciary, Legislators, Mass media, Migrants, Military, National executive, Nominal agents, Occupations, Officials, Political opposition, Political parties, Philanthropic agents, Religious agents, Royalty, Sub-national officials, Students, Unions Level 4 Arabs, Bosnian-Croats, Bosnian-Moslems, Animals, Ancient beliefs, Disease, Bosnian-Serbs, Christians, Cult, Christian-Orthodox, Food, Health conditions, Historical figures, Hindu, Insurgents, Jew, Kurds, Moslems, Ideology, Human languages, Legislation, Peace-keeping forces, Paramilitary groups, Location, Markets, Human-made Police (note that the entries at this level are for conditions, Monetary units, Non-gun weapons, illustration purposes only; a comprehensive list of Protest actions, Plants and flora, Polls & surveys, entries is included in actual data development) Travel to meet, Violent actions, Weapons, Weather conditions Level 5 (greater differentiation among agents is possible Accident, Apology, Assassination, Balance of through user specification) payments, Biological agent/weapon, Bombing action, CBR weapons use, Censorship, Chemical agent/weapon, Commodity prices, Debt yields, Drought, Earthquakes, Equity prices, Exchange rates, Explosive device, Floods, Firearms, Harassment, Hurricanes and typhoons, Infectious disease, Litigation, Military actions, Military hardware, Monetary reserves, Nuclear devices, Protest altruism, Real estate prices, Military raids, Riot, Rapes, Shooting, Strike and boycott action, Tornados, Tsunami, Volcanic events, Wild fires Level 6 Biological weapons use, Car bomb, Car bombing, De-mining vehicle, Grenade/RPG, Grenade/RPG use, Mine explosions, Mines, Missile, Missile attack
44. Synset Example: Selected Entries of Nouns and Noun Phrases Mapped to the Noun Class “Migrant” (WordNet) abandoned, abandoned person, bag lady, beggar, beggarman, beggarwoman, bird of passage, bum, castaway, deportee, derelict, displaced person, internally displaced person, dosser, down-and-out, DP, IDP, drifter, evacuee, exile, foundling, gamin, have-not, hobo, homeless, homeless person, immigrant, mendicant, migrant, nomad, orphan, outcast, outcaste, panhandler, pariah, poor person, profligate, ragamuffin, rake, refugee, rip, roamer, roue, rover, shipwreck survivors, quatter, squatter, stateless person, street arab, street person, sundowner, tatterdemalion, throwaway, tramp, transient, urchin, vagabond, vagrant, waif, wanderer
45. From Syntax to Events The Readergenerates and maps “events,” or who does what to whom when where & how syntax (from the read module) semantics (from an external dictionary) and user-specified information (from a protocol) into data matrices that can be used in statistical and other analyses.
46. Mapped to (user defined) Events The mapping procedure is guided by a user-defined set of dictionaries or protocol. The (IDEA) protocol maps specific words and phrases to their various meanings. Relevant behavioral referents are considered events, around which the event data matrix is built. Each event is linked to its actors who are identified as individuals, groups, organizations or states.
47. The Reader & IDEA Operate together to support monitoring and interactive assessment of evolving conflict situations,
48. Automated Coding Advantages of automated coding no longer in dispute (exceptions: idiosyncratic text, low N) As good as humans (see King and Lowe article “An Automated Information Extraction Tool for International Conflict Data with Performance as Good as Human Coders: A Rare Events Evaluation Design” in International Organization, Vol. 57, No. 3, pp. 617-642, July 2003) 100% Consistent (could be consistently bad but it is consistent) 100% Transparent Flexible & Extensible
49. MACD Line = 12-day exponential moving average – 26-day exponential moving average Signal Line = 9-day exponential moving average of MACD Line MACD Histogram = MACD Line – Signal Line Since the data is aggregated by weeks rather than days, we use 12-wwk, 26-week, and 9-week exponential moving averages respectively. MACD Histogram – Country Stability for Egypt, 2011
50. Bollinger bands can be expressed as follows: Middle Band = 20-period simple moving average Upper Band = 20- period simple moving average + (20- period standard deviation of value * 2) Lower Band = 20-period moving average - (20-period standard deviation of value * 2) Bollinger Bands – Country Stability for Egypt, 2011
51. Group Exercise
52. 2nd In-Class Writing Exercise