Non-official statistics (down the rabbit hole)

1. Education vs. Training Train = prepare against surprise. Educate = prepare for surprise. (Finite and Infinite Games. James P. Carse, 1986) goo.gl/krkEi Non-official statistics(down the rabbit hole) David Sulz, Public Services Librarian University of Alberta 2013 Winter Institute on Statistical Literacy for Librarians

2. Statistical Literacy revisited • Information literacy is … • "recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information." • Determine extent of info needed • Access needed info • Evaluate info and sources critically • Incorporate selected info • Use info to accomplish a specific purpose • Understand economic, legal, and social issues (access and use ethically and legally) • ACRL: Information Literacy Competency Standards for Higher Education. • www.ala.org/acrl/standards/informationliteracycompetency

3. Statistical Literacy is… • Ability to read and interpret numbers in statements, surveys, tables, and graphs. • media, public policy, advertising • Critical thinking about numbers and statistics used as evidence in arguments. • association/correlation vs. causation • errors, weasel words, and vague claims • frequently (count) vs. likely (probability) • real statistics vs. speculative statistics • definitions of a group (Statistical Literacy: A Short Introduction http://www.statlit.org/pdf/2010Schield-StatLit-Intro4p.pdf)

4. Intuitive statisticians? • “Are people good intuitive statisticians?” • Good intuitive grammarians? Yes. • Statisticians? No. • See Daniel Kahneman’sThinking Fast and Slow (2011) for many fascinating examples.

5. Intuitive Probability • Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations. • Which is more probable? • Linda is a bank teller • Linda is a bank teller and an active feminist? • (Daniel Kahneman. Thinking Fast and Slow. 2011)

6. Review Official Statistics • Governmental sources • formal / legislated mandate to gather and process statistical information = “special, official status” • Sources: administrative records, national surveys

7. Non-offical statistics • a.k.a. non-governmental statistics • = statistics that are not • “official statistics” • “No, seriously, what are they?” • Q1) Who are they produced by? • Q2) What do they cover? ;-)

8. What do they cover? • Different areas • markets, sectors, products, populations • More detail • New markets and interests • Different types of data

9. “THE” starting point Who might care?

10. Who produces them?

11. Who produces them? • trade/industry associations • professional bodies • interest groups • academic institutions / academic researchers • research institutes / think-tanks • economic research & forecasting institutes • chambers of commerce / econ dev’t offices • financial services companies • banks, insurance, brokerage firms • commercial publishers • market research organisations • consultants

12. Caveats - Quality • Quality • Reliability and accuracy varies • Accountability and scrutiny (beyond gov’t / public) • Official/non-official line blurs • official stats distributed by private sector • non-official stats incorporated into official series

13. Caveats - Purpose • Purpose: Logic or rhetoric? (thought / analysis vs. instruct /persuade) • How? • selected presentation of results • built right into the question  Push Poll • “designed to plant opinions rather than harvest them”

14. Push Poll example • “If you knew the borscht at the café next door was made with cat-urine stock, would you be more or less likely to dine there?”

15. Caveats - Scale • Scale and Magnitude • UK, regularly produced : >900 sources, 440 orgs. • (Sources of non-official UK statistics. 2005) • BTW: google search “non-official statistics canada” • statistics on non-official languages in Canada from census! “Either the well was very deep, or she fell very slowly, for she had plenty of time as she went down to look about her and to wonder what was going to happen next.” n.b. The “well” IS very deep but you may not have plenty of time!

16. Caveats - Accessibility • Accessibility • Synopsis in press release, article, website, report • Details only to clients, members, participants Options: • Free • Institutional affiliation, Personal affiliation • Purchase • DIY (collect, analyse)

17. Free vs Fee • 1) By def’n, non-official not paid by us (taxpayer). • Collected or paid by someone else for their own purposes • 2) Ask self, “why would this be offered free?” • Expensive to collect, process, and disseminate • Promotion? Already paid by another means? • 3) You can always do it yourself! • Aggregate from various sources (e.g. annual reports) • Design own survey • Commission a survey

18. Free • “the sides of the well … were filled with cupboards and book-shelves ...  • She took down a jar from one of the shelves as she passed; it was labelled `ORANGE MARMALADE', • but to her great disappointment it was empty.”

19. Fee • “There were doors all round the hall, but they were all locked. Suddenly … a tiny golden key … but, alas! either the locks were too large, or the key was too small.” • “On the second time round, … a passage into the loveliest garden you ever saw. How she longed to get out of that dark hall, and wander about among those beds of bright flowers and those cool fountains, but she could not even get her head though the doorway.” • (“and even if my head would go through…it would be of very little use without my shoulders.”)

20. Exploration • How much do aircraft, avionics, and space technology companies spend on R&D? • Brainstorm: in groups of 2-4 • goo.gl/krkEi

21. Industry Associations Q: What year is this for? Q: What is the source? Q: What does “aerospace” cover? Q: How does it compare?

22. AIAC Statistical Highlights Comparison

23. Associations • Industry statistics & economic indicators: • Sales figures • Production rates • Export/import facts • Market share rankings • Demographics • Consumer spending habits • Surveys

24. Exploration Any issues with this chart?

25. Consulting / Professional firms Q: How was the information for this graph collected? Q: What year? Q: What population was sampled? Q: Any issues with methodology? goo.gl/krkEi

26. Explore e-book stats further • TableBase (RDS business suite) - fee • “journal index” for tabular information • trade/business journals, industry newsletters, newspapers, statistical annuals, brokerage reports, trade assoc. reports, gov’t reports. • Library.ualberta.ca  databases  tablebase Q: Is the previous table in TableBase (i.e. from “turning the page” report)? Q: Share observations on searching for e-books goo.gl/krkEi

27. Market research agencies $1000s, to $10,000s and limited use rights

28. Passport GMID • (a.k.a.) Global Market Information Database • Life sciences researcher wants market information for a new, minor ear wax “issue” treatment (not swab).  • Key market players? • Dominant brands? • Sales forecasts ? • Potential international markets to target? • (instructor-led brainstorm) goo.gl/krkEi Q: What units of observation?

29. GMID issues • Permission screen • 2.2. No form of commercial use of market reports, articles, written materials and data is permitted. You may not publish, re-distribute or make available to third parties any such content which you extract from this service, whether by itself or as part of any work or other material. • Q? What about an academic researcher wants to include in conference presentation, conference proceedings, and published paper? • A: UofA license: "To reproduce insubstantial and limited amounts of the Intelligence in the Authorised User's own course work, reports, essays, projects and similar materials which he or she creates for academic purposes only. For the avoidance of doubt this includes use in scholarly publications, and discussion or presentation of such research or publications at academic conferences".

30. Exploration • What are the viewer demographics for the TV show “Biggest Loser”? • Brainstorm: Who would care? goo.gl/krkEi

31. PMB • - library.ualberta.ca  databases  PMB - see “how to read a PMB report” • What are the viewer demographics for the TV show “Biggest Loser”? Q: Is “demographics” an appropriate request? Q: Observations on the questionnaire and collection format used to generate the information. Opt. Q: What are Edmonton’s top 3 coffee shops

32. From the bottom up • Ipsosi-say Panel (www.i-say.com) • exposure to variety of surveys: wording, intents, etc • Sign yourself up? • Canadian Financial Monitor • View actual survey goo.gl/ssZ2L • Internet search  Ipsos, Environics • Wealthscapes: www.environicsanalytics.ca/data_financial_behaviour.aspx?item=wealthscapes

33. Insurance Companies • Sun Life Canadian Health Index • Canada score = 68.5 • Perceived health index = 70.9 • Attitudinal health index = 67 • Behavioural health index = 67.7 NB: for more information on the survey contact: Corporate.Marketing.and.Communications@sunlife.com” Q: Better information from official sources?

34. Financial institutions Nearly half of Canadians approaching retirement consider their legacy, Investors Group poll finds “The legacy Canadians plan to leave to charitable and non-profit organizations will average about 28 per cent of their financial assets according to the Investors Group poll, regardless of age.”

35. Exploration • “Breast cancer is the leading cause of death among women in Canada” • Brainstorm: non-official sources goo.gl/krkEi

36. Charities 14.4% of 29.6% = 4% maximum Diseases of the heart, Lung cancer Cerebrovascular disease Lower respiratory Accidents

37. Cancer Stats

38. Survey / polling indexes • iPoll Databank (Roper Center, Public Opinion Archives • - library.ualberta.ca  databases  Japan earthquake • Odesi.ca • digital repository for social science data, including polling data. • search metadata then contact data producer directly. • www.aapor.org/Poll_andamp_Survey_FAQ/5511.htm

39. Exploration • What’s the economic climate like in Edmonton these days? • Brainstorm: goo.gl/krkEi

40. Chambers of Commerce /Economic Development Organizations Q: Which Edmonton Monthly Key Economic Indicators stats are from official sources and which from non-official? goo.gl/krkEi

41. Research Institutes/Think tanks Q: Can you replicate this chart? goo.gl/krkEi

42. Classification standards • Statistics Canada • Key resource -> Definitions, data sources and methods • NAICS: North American Industry Classification System. • “Other classifications”: eg non-official languages, religions, NOC (occupations) • Note: SIC (Standard Industrial Classification) • Replaced by NAICS but historical • Analyze U.S. economy

43. NAICS (nakes) U.S., Canada and Mexico (‘97 - NAFTA) to compare business activity production-oriented: industries grouped by similar input structures, labour skills and production processes. Caveat: used for classifying companies BUT NOT vertically or horizontally integrated activities of large and complex, multi-establishment companies.

44. Challenge Question • Q: Is this response legit? • Q: Who is right? • Hints: • What is Rx&D? • Is this 2011 KPMG study available? • What stats were used to make this 20% assertion? • What source did Simpson use for his conclusion? goo.gl/krkEi

45. Search strategies • Basic questions: • Who would collect this? • Why? • Keywords: • statistics • percent • market share, market size, industry overview • financial data, expenditures, costs • demographics • trends: increased, decreased, grew/grow, rise(n), fall(en)

46. Other tools - Free • Winspear Canadian Industry Database • Libguides (UA or other institutions) • Pew Research Centre - attitudes toward politics, the press and public policy issues • Alacra Industry Spotlights - reviews and commentary on the best web sources of industry specific content

47. Other Tools (Fee) Print • Who Buys What? (Euromonitor) • Canadian Demographics (FPmarkets) • Databases • Superdemographics • Economist Intelligence Unit • Association directories • Associations Canada, Associations Unlimited • Bureau van dijk

48. Gauging quality • “What matters is not necessarily the institutional position of the source, but the quality of data.” • (EnricoGiovannini, Chief Statistician - OECD) • Metadata available • Producer’s reputation • Gap between collection and publication • Primary or Secondary (sources identified) • Do results make sense? • Confirmed in other sources

49. To spend or not spend? • Audience / potential usage • Positive reviews • Cost (consortial pricing?) • Access: Simultaneous users • Electronic or print • Technical issues: • Compatibility -> browsers, proxy servers • Usage stats

50. Further practice • How safe is it to work on an oil rig in Canada? • Brainstorm: goo.gl/krkEi