1 / 103

Implementation and Research Issues in Query Processing for Wireless Sensor Networks

Implementation and Research Issues in Query Processing for Wireless Sensor Networks. Wei Hong Intel Research, Berkeley whong@intel-research.net. Sam Madden MIT madden@csail.mit.edu. MDM Tutorial, January 19th 2004. Motivation. Sensor networks (aka sensor webs, emnets) are here

mcglade
Download Presentation

Implementation and Research Issues in Query Processing for Wireless Sensor Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Implementation and Research Issues in Query Processing for Wireless Sensor Networks Wei Hong Intel Research, Berkeley whong@intel-research.net Sam Madden MIT madden@csail.mit.edu MDM Tutorial, January 19th 2004

  2. Motivation • Sensor networks (aka sensor webs, emnets) are here • Several widely deployed HW/SW platforms • Low power radio, small processor, RAM/Flash • Variety of (novel) applications: scientific, industrial, commercial • Great platform for mobile + ubicomp experimentation • Real, hard research problems to be solved • Networking, systems, languages, databases • We will summarize: • The state of the art • Our experiences building TinyDB • Current and future research directions Berkeley Mote

  3. Earthquake monitoring in shake-test sites. Vehicle detection: sensors along a road, collect data about passing vehicles. • Traditional monitoring apparatus. Sensor Network Apps Habitat Monitoring: Storm petrels on Great Duck Island, microclimates on James Reserve. Just the tip of the iceberg -- more tomorrow!

  4. Declarative Queries • Programming Apps is Hard • Limited power budget • Lossy, low bandwidth communication • Require long-lived, zero admin deployments • Distributed Algorithms • Limited tools, debugging interfaces • Queries abstract away much of the complexity • Burden on the database developers • Users get: • Safe, optimizable programs • Freedom to think about apps instead of details

  5. TinyDB: Prototype declarativequery processor • Platform: Berkeley Motes + TinyOS • Continuous variant of SQL : TinySQL • Power and data-acquisition based in-network optimization framework • Extensible interface for aggregates, new types of sensors

  6. Agenda • Part 1 : Sensor Networks (50 Minutes) • TinyOS • NesC • Short Break • Part 2: TinyDB (1 Hour) • Data Model and Query Language • Software Architecture • Long Break + Hands On • Part 3: Sensor Network Database Research Directions (1 Hour, 10 Minutes)

  7. Part 1 • Sensornet Background • Motes + Mote Hardware • TinyOS • Programming Model + NesC • TinyOS Architecture • Major Software Subsystems • Networking Services

  8. A Brief History of Sensornets • People have used sensors for a long time • Recent CS History: • (1998) Pottie + Kaiser: Radio based networks of sensors • (1998) Pister et al: Smart Dust • Initial focus on optical communication • By 1999, radio based networks, COTS Dust, “Motes” • (1999) Estrin + Govindan • Ad-hoc networks of sensors • (2000) Culler/Hill et al: TinyOS + Motes • (2002) Hill / Dust: SPEC, mm^3 scale computing • UCLA / USC / Berkeley Continue to Lead Research • Many other players now • TinyOS/Motes as most common platform • Emerging commercial space: • Crossbow, Ember, Dust, Sensicast, Moteiv, Intel

  9. Why Now? • Commoditization of radio hardware • Cellular and cordless phones, wireless communication • (some radio pictures, etc.) • Low cost -> many/tiny -> new applications! • Real application for ad-hoc network research from the late 90’s • Coming together of EE + CS communities

  10. Motes 4Mhz, 8 bit Atmel RISC uProc 40 kbit Radio 4 K RAM, 128 K Program Flash, 512 K Data Flash AA battery pack Based on TinyOS Mica Mote Mica2Dot

  11. History of Motes • Initial research goal wasn’t hardware • Has since become more of a priority with emerging hardware needs, e.g.: • Power consumption • (Ultrasonic) ranging + localization • MIT Cricket, NEST Project • Connectivity with diverse sensors • UCLA sensor board • Even so, now on the 5th generation of devices • Costs down to ~$50/node (Moteiv, Dust) • Greatly improved radio quality • Multitude of interfaces: USB, Ethernet, CF, etc. • Variety of form factors, packages

  12. Motes vs. Traditional Computing • Lossy, Adhoc Radio Communication • Sensing Hardware • Severe Power Constraints

  13. From Ganesan, et al. “Complex Behavior at Scale.” UCLA/CSD-TR 02-0013 Radio Communication • Low Bandwidth Shared Radio Channel • ~40kBits on motes • Much less in practice • Encoding, Contention for Media Access (MAC) • Very lossy: 30% base loss rate • Argues against TCP-like end-to-end retransmission • And for link-layer retries • Generally, not well behaved

  14. Types of Sensors • Sensors attach via daughtercard • Weather • Temperature • Light x 2 (high intensity PAR, low intensity, full spectrum) • Air Pressure • Humidity • Vibration • 2 or 3 axis accelerometers • Tracking • Microphone (for ranging and acoustic signatures) • Magnetometer • GPS

  15. Power Consumption and Lifetime • Power typically supplied by a small battery • 1000-2000 mAH • 1 mAH = 1 milliamp current for 1 hour • Typically at optimum voltage, current drain rates • Power = Watts (W) = Amps (A) * Volts (V) • Energy = Joules (J) = W * time • Lifetime, power consumption varies by application • Processor: 5mA active, 1 mA idle, 5 uA sleeping • Radio: 5 mA listen, 10 mA xmit/receive, ~20mS / packet • Sensors: 1 uA -> 100’s mA, 1 uS -> 1 S / sample

  16. Energy Usage in A Typical Data Collection Scenario • Each mote collects 1 sample of (light,humidity) data every 10 seconds, forwards it • Each mote can “hear” 10 other motes • Process: • Wake up, collect samples (~ 1 second) • Listen to radio for messages to forward (~1 second) • Forward data

  17. Sensors: Slow, Power Hungry, Noisy

  18. Programming Sensornets: TinyOS • Component Based Programming Model • Suite of software components • Timers, clocks, clock synchronization • Single and multi-hop networking • Power management • Non-volatile storage management

  19. Programming Philosophy • Component Based • “Wiring” to components together via interfaces, configurations • Split-Phased • Nothing blocks, ever. • Instead, completion events are signaled. • Highly Concurrent • Single thread of “tasks”, posted and scheduled FIFO • Events “fired” asynchronously in response to interrupts.

  20. NesC • C-like programming language with component model support • Compiles into GCC-compatible C • 3 types of files: • Interfaces • Set of function prototypes; no implementations or variables • Modules • Provide (implement) zero or more interfaces • Require zero or more interfaces • May define module variables, scoped to functions in module • Configurations • Wire (connect) modules according to requires/provides relationship

  21. Component Example: Leds module LedsC { provides interface Leds; } implementation { uint8_t ledsOn; enum { RED_BIT = 1, GREEN_BIT = 2, YELLOW_BIT = 4 }; …. async command result_t Leds.redOn() { dbg(DBG_LED, "LEDS: Red on.\n"); atomic { TOSH_CLR_RED_LED_PIN(); ledsOn |= RED_BIT; } return SUCCESS; } …. }

  22. Configuration Example configuration CntToLedsAndRfm { } implementation { components Main, Counter, IntToLeds, IntToRfm, TimerC; Main.StdControl -> Counter.StdControl; Main.StdControl -> IntToLeds.StdControl; Main.StdControl -> IntToRfm.StdControl; Main.StdControl -> TimerC.StdControl; Counter.Timer -> TimerC.Timer[unique("Timer")]; IntToLeds <- Counter.IntOutput; Counter.IntOutput -> IntToRfm; }

  23. Split Phase Example module IntToRfmM { … } implementation { … command result_t IntOutput.output (uint16_t value) { IntMsg *message = (IntMsg *)data.data; if (!pending) { pending = TRUE; message->val = value; atomic { message->src = TOS_LOCAL_ADDRESS; } if (call Send.send(TOS_BCAST_ADDR, sizeof(IntMsg), &data)) return SUCCESS; pending = FALSE; } return FAIL; } event result_t Send.sendDone (TOS_MsgPtr msg, result_t success) { if (pending && msg == &data) { pending = FALSE; signal IntOutput.outputComplete (success); } return SUCCESS; } } }

  24. Major Components • Timers: Clock, TimerC, LogicalTime • Networking: Send, GenericComm, AMStandard, lib/Route • Power Management: HPLPowerManagement • Storage Management: EEPROM, MatchBox

  25. Timers • Clock: Basic abstraction over hardware timers; periodic events, single frequency. • LogicalTime: Fire an event some number of H:M:S:ms in the future. • TimerC: Multiplex multiple periodic timers on top of LogicalTime.

  26. Radio Stack IntMsg *message = (IntMsg *)data.data; … message->val = value; atomic { message->src = TOS_LOCAL_ADDRESS; } call Send.send(TOS_BCAST_ADDR, sizeof(IntMsg), &data)) • Interfaces: • Send • Broadcast, or to a specific ID • split phase • Receive • asynchronous signal • Implementations: • AMStandard • Application specific messages • Id-based dispatch • GenericComm • AMStandard + Serial IO • Lib/Route • Mulithop event TOS_MsgPtr ReceiveIntMsg. receive(TOS_MsgPtr m) { IntMsg *message = (IntMsg *)m->data; call IntOutput.output(message->val); return m; } Wiring to equate IntMsg to ReceiveIntMsg

  27. A B B R:{…} R:{…} B B C B B B B R:{…} D R:{…} B Node D NeighQual B .75 C .66 E .45 F .82 Node C NeighQual A .5 B .44 D .53 F .35 R:{…} B B B F E B Multihop Networking • Standard implementation “tree based routing” Problems: Parent Selection Asymmetric Links Adaptation vs. Stability

  28. Geographic Routing • Any-to-any routing via geographic coordinates • See “GPSR”, MOBICOM 2000, Karp + Kung. • Requires coordinate system* • Requires endpont coordinates • Hard to route around local minima (“holes”) B A *Could be virtual, as in Rao et al “Geographic Routing Without Coordinate Information.” MOBICOM 2003

  29. Power Management • HPLPowerManagement • TinyOS sleeps processor when possible • Observes the radio, sensor, and timer state • Application managed, for the most part • App. must turn off subsystems when not in use • Helper utility: ServiceScheduler • Peridically calls the “start” and “stop” methods of an app • More on power management in TinyDB later • Approach works because: • single application • no interactivity requirements

  30. Non-Volatile Storage • EEPROM • 512K off chip, 32K on chip • Writes at disk speeds, reads at RAM speeds • Interface : random access, read/write 256 byte pages • Maximum throughput ~10Kbytes / second • MatchBox Filing System • Provides a Unix-like file I/O interface • Single, flat directory • Only one file being read/written at a time

  31. TinyOS: Getting Started • The TinyOS home page: • http://webs.cs.berkeley.edu/tinyos • Start with the tutorials! • The CVS repository • http://sf.net/projects/tinyos • The NesC Project Page • http://sf.net/projects/nescc • Crossbow motes (hardware): • http://www.xbow.com • Intel Imote • www.intel.com/research/exploratory/motes.htm.

  32. Part 2 The Design and Implementation of TinyDB

  33. Part 2 Outline • TinyDB Overview • Data Model and Query Language • TinyDB Java API and Scripting • Demo with TinyDB GUI • TinyDB Internals • Extending TinyDB • TinyDB Status and Roadmap

  34. Sensor Network TinyDB Revisited SELECT MAX(mag) FROM sensors WHERE mag > thresh SAMPLE PERIOD 64ms • High level abstraction: • Data centric programming • Interact with sensor network as a whole • Extensible framework • Under the hood: • Intelligent query processing: query optimization, power efficient execution • Fault Mitigation: automatically introduce redundancy, avoid problem areas App Query, Trigger Data TinyDB

  35. Feature Overview • Declarative SQL-like query interface • Metadata catalog management • Multiple concurrent queries • Network monitoring (via queries) • In-network, distributed query processing • Extensible framework for attributes, commands and aggregates • In-network, persistent storage

  36. Architecture TinyDB GUI JDBC TinyDB Client API DBMS PC side 0 Mote side 0 TinyDB query processor 2 1 3 8 4 5 6 Sensor network 7

  37. Data Model • Entire sensor network as one single, infinitely-long logical table: sensors • Columns consist of all the attributes defined in the network • Typical attributes: • Sensor readings • Meta-data: node id, location, etc. • Internal states: routing tree parent, timestamp, queue length, etc. • Nodes return NULL for unknown attributes • On server, all attributes are defined in catalog.xml • Discussion: other alternative data models?

  38. Query Language (TinySQL) SELECT <aggregates>, <attributes> [FROM {sensors | <buffer>}] [WHERE <predicates>] [GROUP BY <exprs>] [SAMPLE PERIOD <const> | ONCE] [INTO <buffer>] [TRIGGER ACTION <command>]

  39. Comparison with SQL • Single table in FROM clause • Only conjunctive comparison predicates in WHERE and HAVING • No subqueries • No column alias in SELECT clause • Arithmetic expressions limited to column op constant • Only fundamental difference: SAMPLE PERIOD clause

  40. TinySQL Examples SELECT nodeid, nestNo, light FROM sensors WHERE light > 400 EPOCH DURATION 1s “Find the sensors in bright nests.” Sensors 1

  41. 2 SELECT AVG(sound) FROM sensors EPOCH DURATION 10s • SELECT region, CNT(occupied) AVG(sound) • FROM sensors • GROUP BY region • HAVINGAVG(sound) > 200 • EPOCH DURATION 10s 3 Regions w/ AVG(sound) > 200 TinySQL Examples (cont.) “Count the number occupied nests in each loud region of the island.”

  42. Event-based Queries • ON event SELECT … • Run query only when interesting events happens • Event examples • Button pushed • Message arrival • Bird enters nest • Analogous to triggers but events are user-defined

  43. Query over Stored Data • Named buffers in Flash memory • Store query results in buffers • Query over named buffers • Analogous to materialized views • Example: • CREATE BUFFER name SIZE x (field1 type1, field2 type2, …) • SELECT a1, a2 FROM sensors SAMPLE PERIOD d INTO name • SELECT field1, field2, … FROM name SAMPLE PERIOD d

  44. Using the Java API • SensorQueryer • translateQuery() converts TinySQL string into TinyDBQuery object • Static query optimization • TinyDBNetwork • sendQuery() injects query into network • abortQuery() stops a running query • addResultListener() adds a ResultListener that is invoked for every QueryResult received • removeResultListener() • QueryResult • A complete result tuple, or • A partial aggregate result, call mergeQueryResult() to combine partial results • Key difference from JDBC: push vs. pull

  45. Writing Scripts with TinyDB • TinyDB’s text interface • java net.tinyos.tinydb.TinyDBMain –run “select …” • Query results printed out to the console • All motes get reset each time new query is posed • Handy for writing scripts with shell, perl, etc.

  46. Using the GUI Tools • Demo time

  47. SELECT AVG(temp) WHERE light > 400 T:1, AVG: 225 T:2, AVG: 250 Queries Results Aggavg(temp) Name: temp Time to sample: 50 uS Cost to sample: 90 uJ Calibration Table: 3 Units: Deg. F Error: ± 5 Deg F Get f: getTempFunc()… got(‘temp’) get (‘temp’) Tables Samples getTempFunc(…) Inside TinyDB Multihop Network Query Processor ~10,000 Lines Embedded C Code ~5,000 Lines (PC-Side) Java ~3200 Bytes RAM (w/ 768 byte heap) ~58 kB compiled code (3x larger than 2nd largest TinyOS Program) Filterlight > 400 Schema TinyOS TinyDB

  48. Q:SELECT … A Q Q R:{…} R:{…} Q B C Q Q Q Q R:{…} D R:{…} Q R:{…} Q Q Q F E Q Tree-based Routing • Tree-based routing • Used in: • Query delivery • Data collection • In-network aggregation • Relationship to indexing?

  49. Power Management Approach Coarse-grained app-controlled communication scheduling Epoch (10s -100s of seconds) Mote ID 1 … zzz … … zzz … 2 3 4 5 time 2-4s Waking Period

  50. Time Synchronization • All messages include a 5 byte time stamp indicating system time in ms • Synchronize (e.g. set system time to timestamp) with • Any message from parent • Any new query message (even if not from parent) • Punt on multiple queries • Timestamps written just after preamble is xmitted • All nodes agree that the waking period begins when (system time % epoch dur = 0) • And lasts for WAKING_PERIOD ms • Adjustment of clock happens by changing duration of sleep cycle, not wake cycle.

More Related