1 / 67

Probability Sampling

Probability Sampling. uses random selection N = number of cases in sampling frame n = number of cases in the sample N C n = number of combinations of n from N f = n/N = sampling fraction. Variations. Simple random sampling based on random number generation Stratified random sampling

Download Presentation

Probability Sampling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Probability Sampling • uses random selection • N = number of cases in sampling frame • n = number of cases in the sample • NCn = number of combinations of n from N • f = n/N = sampling fraction

  2. Variations • Simple random sampling • based on random number generation • Stratified random sampling • divide pop into homogenous subgroups, then simple random sample w/in • Systematic random sampling • select every kth individual (k = N/n) • Cluster (area) random sampling • randomly select clusters, sample all units w/in cluster • Multistage sampling • combination of methods

  3. Nonprobability sampling • accidental, haphazard, convenience sampling ... • may or may not represent the population well

  4. Measurement • ... topics in measurement that we don’t have time to cover ...

  5. Research Design • Elements: • Samples/Groups • Measures • Treatments/Programs • Methods of Assignment • Time

  6. Internal validity • the approximate truth about inferences regarding cause-effect (causal) relationships • can observed changes be attributed to the program or intervention and NOT to other possible causes (alternative explanations)?

  7. Establishing a Cause-Effect Relationship • Temporal precedence • Covariation of cause and effect • if x then y; if not x then not y • if more x then more y; if less x then less y • No plausible alternative explanations

  8. Single Group Example • Single group designs: • Administer treatment -> measure outcome X -> O • assumes baseline of “0” • Measure baseline -> treat -> measure outcome 0 X -> O • measures change over baseline

  9. Single Group Threats • History threat • a historical event occurs to cause the outcome • Maturation threat • maturation of individual causes the outcome • Testing threat • act of taking the pretest affects the outcome • Instrumentation threat • difference in test from pretest to posttest affects the outcome • Mortality threat • do “drop-outs” occur differentially or randomly across the sample? • Regression threat • statistical phenomenon, nonrandom sample from population and two imperfectly correlated measures

  10. Addressing these threats • control group + treatment group • both control and treatment groups would experience same history and maturation threats, have same testing and instrumentation issues, similar rates of mortality and regression to the mean

  11. Multiple-group design • at least two groups • typically: • before-after measurement • treatment group + control group • treatment A group + treatment B group

  12. Multiple-Group Threats • internal validity issue: • degree to which groups are comparable before the study • “selection bias” or “selection threat”

  13. Multiple-Group Threats • Selection-History Threat • an event occurs between pretest and posttest that groups experience differently • Selection-Maturation Threat • results from differential rates of normal growth between pretest and posttest for the groups • Selection-Testing Threat • effect of taking pretest differentially affects posttest outcome of groups • Selection-Instrumentation Threat • test changes differently for the two groups • Selection-Mortality Threat • differential nonrandom dropout between pretest and posttest • Selection-Regression Threat • different rates of regression to the mean in the two groups (if one is more extreme on the pretest than the other)

  14. Social Interaction Threats • Problem: • social pressures in research context can lead to posttest differences that are not directly caused by the treatment • Solution: • isolate the groups • Problem: in many research contexts, hard to randomly assign and then isolate

  15. Types of Social Interaction Threats • Diffusion or Imitation of Treatment • control group learns about/imitates experience of treatment group, decreasing difference in measured effect • Compensatory Rivalry • control group tries to compete w/treatment group, works harder, decreasing difference in measured effect • Resentful Demoralization • control group discouraged or angry, exaggerates measured effect • Compensatory Equalization of Treatment • control group compensated in other ways, decreasing measured effect

  16. Intro to Design/ Design Notation • Observations or Measures • Treatments or Programs • Groups • Assignment to Group • Time

  17. Observations/Measure • Notation: ‘O’ • Examples: • Body weight • Time to complete • Number of correct response • Multiple measures: O1, O2, …

  18. Treatments or Programs • Notation: ‘X’ • Use of medication • Use of visualization • Use of audio feedback • Etc. • Sometimes see X+, X-

  19. Groups • Each group is assigned a line in the design notation

  20. Assignment to Group • R = random • N = non-equivalent groups • C = assignment by cutoff

  21. Time • Moves from left to right in diagram

  22. Types of experiments • True experiment – random assignment to groups • Quasi experiment – no random assignment, but has a control group or multiple measures • Non-experiment – no random assignment, no control, no multiple measures

  23. Design Notation Example Pretest-posttest treatment versus comparison group randomized experimental design

  24. Design Notation Example Pretest-posttest Non-Equivalent Groups Quasi-experiment

  25. Design Notation Example Posttest Only Non-experiment

  26. Goals of design .. • Goal:to be able to show causality • First step: internal validity: • If x, then y AND • If not X, then not Y

  27. Two-group Designs • Two-group, posttest only, randomized experiment Compare by testing for differences between means of groups, using t-test or one-way Analysis of Variance(ANOVA) Note: 2 groups, post-only measure, two distributions each with mean and variance, statistical (non-chance) difference between groups

  28. To analyze … • What do we mean by a difference?

  29. Possible Outcomes:

  30. Three ways to estimate effect • Independent t-test • One-way Analysis of Variance (ANOVA) • Regression Analysis (most general) • equivalent

  31. The t-test • appropriate for posttest-only two-group randomized experimental design • See also: paired student t-test for other situations.

  32. Measuring Differences …

  33. Computing the t-value

  34. Computing standard deviation • standard deviation is the square root of the sum of the squared deviations from the mean divided by the number of scores minus one • variance is the square of the standard deviation

  35. ANOVA • One-way analysis of variance

  36. ANOVA • Analysis of variance – tests hypotheses about differences between two or more means • Could do pairwise comparison using t-tests, but can lead to true hypothesis being rejected (Type I error) (higher probability than with ANOVA)

  37. Between-subjects design • Example: • Effect of intensity of background noise on reading comprehension • Group 1: 30 minutes reading, no background noise • Group 2: 30 minutes reading, moderate level of noise • Group 3: 30 minutes reading, loud background noise

  38. Experimental Design • One factor (noise), three levels(a=3) • Null hypothesis: 1 =2 =3

  39. Notation • If all sample sizes same, use n, and total N = a * n • Else N = n1 + n2 +n3

  40. Assumptions • Normal distributions • Homogeneity of variance • Variance is equal in each of the populations • Random, independent sampling • Still works well when assumptions not quite true(“robust” to violations)

  41. ANOVA • Compares two estimates of variance • MSE – Mean Square Error, variances within samples • MSB – Mean Square Between, variance of the sample means • If null hypothesis • is true, then MSE approx = MSB, since both are estimates of same quantity • Is false, the MSB sufficiently > MSE

  42. MSE

  43. MSB • Use sample means to calculate sampling distribution of the mean, = 1

  44. MSB • Sampling distribution of the mean * n • In example, MSB = (n)(sampling dist) = (4) (1) = 4

  45. Is it significant? • Depends on ratio of MSB to MSE • F = MSB/MSE • Probability value computed based on F value, F value has sampling distribution based on degrees of freedom numerator (a-1) and degrees of freedom denominator (N-a) • Lookup up F-value in table, find p value • For one degree of freedom, F == t^2

  46. Factorial Between-Subjects ANOVA, Two factors • Three significance tests • Main factor 1 • Main factor 2 • interaction

  47. Example Experiment • Two factors (dosage, task) • 3 levels of dosage (0, 100, 200 mg) • 2 levels of task (simple, complex) • 2x3 factorial design, 8 subjects/group

  48. Summary table SOURCE df Sum of Squares Mean Square F p Task 1 47125.3333 47125.3333 384.174 0.000 Dosage 2 42.6667 21.3333 0.174 0.841 TD 2 1418.6667 709.3333 5.783 0.006 ERROR 42 5152.0000 122.6667 TOTAL 47 53738.6667 • Sources of variation: • Task • Dosage • Interaction • Error

  49. Results • Sum of squares (as before) • Mean Squares = (sum of squares) / degrees of freedom • F ratios = mean square effect / mean square error • P value : Given F value and degrees of freedom, look up p value

  50. Results - example • Mean time to complete task was higher for complex task than for simple • Effect of dosage not significant • Interaction exists between dosage and task: increase in dosage decreases performance on complex while increasing performance on simple

More Related