1 / 90

Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge

Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge. Nicklaus A. Giacobe , Hyun-Woo “Anthony” Kim and Avner Faraz [nxg13, hxk263] @ ist.psu.edu , fia5005@psu.edu . The DARPA Network Challenge.

diep
Download Presentation

Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mining Social Media in Extreme Events: Lessons Learned from the DARPA Network Challenge Nicklaus A. Giacobe, Hyun-Woo “Anthony” Kim and AvnerFaraz [nxg13, hxk263] @ist.psu.edu, fia5005@psu.edu

  2. The DARPA Network Challenge • Conceived to better understand and harness the power of the Internet • How does one get a video to go viral on YouTube? • Leveraging social networks to solve “intractable” or “impossible” tasks • Short time frame • Announced on October 29, 2009 • Challenge occurred on December 5, 2009 (5 weeks of possible prep time)

  3. The DARPA Network Challenge • DARPA launched 10 red weather balloons • Tethered to fixed locations in public places across the continental United States • Find them and report GPS coordinates of all 10 before anyone else does and win the $40,000 prize • The balloons would be “up” from 0900 EST until 1600 local time (7-10 hours) and then taken down

  4. Final Results • 4600 “Teams” registered with DARPA • 58 Teams were “in the hunt” and submitted 2 or more correct locations • Top teams used mass marketing and mass motivation techniques (offered to share the prize $ with observers) • Each team has a “lesson” applicable to homeland security

  5. Top Finalists – We’re #10! From: https://networkchallenge.darpa.mil/FinalStandings.pdf

  6. Overview • Who are the ? • Caucus of academic institutions studying information science • The iSchools are interested in the relationship between information, peopleand technology. • There are 28 academic Institutions (Colleges or Departments) in various universities across 8 countries • Some iSchools have roots in Library Science, Computational Sciences, MIS, Business Management, Cognitive Science, Human-Computer Interaction (HCI) and other fields. Each iSchool has its own focus and competencies.

  7. Team Organization • Command Structure • Attempted to follow ICS from Fire Service • Most Team Members unfamiliar with this organizational structure, therefore very limited success • Operational Section • 2 Branches – Direct Observation and Cyberspace Search • Facilities • 211 IST – EEL – Command Post • 208 IST – Classroom – Cyberspace Search

  8. Methods • Direct Observation • Recruit observers from the iSchools Caucus • Report sightings through • Website • Phone / SMS • Email • Cyberspace Search • Multiple Intel Teams searching open communications online • Twitter, Competitor Websites, No Hacking • Confirmation and Decision Making

  9. Methods • Technologies Used • Twitter Capture – Anthony Kim • Custom Crawler – Madian Khabsa • Maltego2 – AvnerFaraz

  10. Methods

  11. Methods • Dempster-Shafer Evidence Theory Combination • Combine evidence from multiple sources under uncertainty • Apply confidence weights to sensor data • Intended, but applied cognitively • Analysts were to provide report data with confidence values (0=low, 10=high) • Some algorithmic process would have been needed to combine large numbers of reports • … but we had *extremely low* # of reports

  12. Pre-Challenge Org.

  13. Team Organization • Other Universities • Various Schools to send recruiting messages • UNC – distributed own phone number and email address for their recruiting messages • Univ. of Illinois – single-person cyberspace search division

  14. Team Organization • Finance and Logistics • Google Voice (814-4BALL01) • Website Design (balloon.ist.psu.edu) • Email Address (balloon@ist.psu.edu) • Google Wave (Intel and Command Comms) • Private Twitter (late attempt at outbound comms) • Coffee, Donuts, Pizza, Soda and Homemade Cookies ($100!) • Incentives and Rewards (10 GPS Systems Offered by iSchools)

  15. Team Members • University of Illinois • John Unsworth • Maeve Reilly • Karyn Applegate • University of North Carolina • Aaron Brubaker • Kjersti Kyle • Other iSchools • Marketing/Communications Staff

  16. Team Members • Penn State University • Command Post Team • Nick Giacobe • Wade Shumaker • Louis-Marie NgamassiTchouakeu • John Yen • Jon Becker (p/t) • Michelle Young (p/t) • Logistics / Website • Shannon Johnson • Lei Lei Zhu

  17. Team Members • Penn State University • Operations (Cyberspace Search Branch) • Crawler Task Force • Madian Khabsa and Jian Huang • Twitter Capture Task Force • Hyun Woo “Anthony” Kim and Airy Guru • Intelligence Analysts • Chris Robuck • Greg Traylor • Anthony Maslowski • Gregory O’Neill • Joe Magobeth • AvnerFaraz • Matt Maisel • Earl Yun

  18. Results • Recruiting Message Reports • University of Illinois – Email to iSchool alumni; to faculty, staff and students. • University of Pittsburgh – Email to all alumni; to faculty, staff and students; Webpage article on main page & alumni news page; LinkedIn announcement to iSchool at Pitt group members; and Facebook announcement to iSchool at Pitt group members. Alumni email distributed to 4,674 alumni. 114 opened the DARPA link. Email blasts to 936 students 41 faculty. • Penn State – 338 fans on Facebook and 377 followers on Twitter. Re-tweets, questions about the Challenge on Twitter, but no activity on Facebook. 2,125 alumni received the e-mail. • UCLA – Sent messages to faculty, staff, students and alumni. • Drexel – Alumni listserv, Facebook, Tweeter electronic newsletter for undergrads and grads, online learning system grad web site release.

  19. Results • Direct Reporting Data • 1 Report From Observer • 1 Pre-recruited Observer tasked for confirmation • 8 Observers recruited for confirmation • Website Hit Data • 567 hits (Terrible!) • 16 Case Studies of Individual Balloon Reports…

  20. Overview of Cases

  21. Results: Case Study 1 • Location: Erie, PA • Original Report: 10balloons.com website • Method: Observer mobilized • Notes: • Early report – 9:30AM • Evaluated whether 10balloons.com website was going to be useful intel or not • Observer was not pre-registered, was personal friend of command post personnel

  22. Results: Case Study 2 • Location: Albany, NY • Original Report: Twitter Feed • Method: Observer mobilized • Notes: • Observer provided convincing photo evidence – no balloon at this location • Subsequent photo analysis confirmed this was a manufactured image

  23. Results: Case Study 2 • Evidence for • Reputability (3/10) • Established Twitter Acct • Content (6/10) • Photo of a balloon • Balloon Number 6? • Weather – match • Location – exact coords not provided, but discoverable • Evidence Against • Reputability (9/10) • Pre-recruited observer • Known person (PSU Alumni Assn chapter president) • Content (10/10) • Excellent quality photo • Same angle, confirming coincidental landmarks/features

  24. Results: Case Study 3 • Location: Royal Oak, MI • Original Report: Twitter • Method: Observer Mobilized • Notes: • Observer went to the location and talked to the store owner who admitted that the balloon was a “publicity stunt” – knew about DARPA challenge and put up own balloon. • Observer was not pre-recruited

  25. Results: Case Study 3 • Evidence for • Reputability (3/10) • Established Twitter Acct • Content (4/10) • Photo of a balloon • No “DARPA” pennant , not balloon 6 • Weather – match for location • Location – Had GPS coordinates for location from photo • Evidence Against • Reputability (9/10) • Observer recruited after the fact • Content (8/10) • No photos sent • Good report, verbal description of events

  26. Results: Case Study 4 • Location: Providence, RI • Original Report: Twitter • Method: Observer Mobilized • Notes: • Observer was a friend of one of the analysts in the Intel Division – reported no balloon at that location • Photo analysis provided repeat of the exact fabricated image

  27. Results: Case Study 4 • Evidence for • Reputability (3/10) • Established Twitter Acct • Content (4/10) • Photo of a balloon • Evidence Against • Reputability (9/10) • Observer recruited after the fact • Content (8/10) • No photos sent • Good report, verbal description of location • Photo Evaluation (10/10) • Reproduced shop job • Edge evaluation

  28. Results: Case Study 5 • Location: Seattle, WA • Original Report: • Method: Observer Mobilized • Notes: • Report was for balloon over the University of Washington Library • Observer was Assoc. Dean of LIS College at UW

  29. Results: Case Study 6 • Location: Champaign, IL • Original Report: 10Balloons.com • Method: Observer Mobilized • Notes: • No picture • Known observer/Team member in close proximity to this location

  30. Results: Case Study 7 • Location: Des Moines, IA • Original Report: Twitter • Method: Self-Recant • Notes: • Tweet gave location/street address of reporters home • No photo evidence • Reporter then re-tweeted complaining about people running through her yard and then recanted

More Related