Time Synchronized Meshing K. Pister Prof. EECS, UC Berkeley Founder & CTO, Dust Networks

1. Time Synchronized MeshingK. PisterProf. EECS, UC BerkeleyFounder & CTO, Dust Networks This document is provided strictly for the purpose of gathering information leading to the development of an ISA standard, recommended practice or technical report. Copies may be reproduced and distributed, in whole or in part, but only for the following purposes: Review of and comment on the ISA-SP100 draft proposal Submission to ISA-SP100 Committee Informing and educating others about the ISA-SP100 draft standard development process.

2. Goals • Non-fanaticism • TDMA & CSMA • Centralized & decentralized management • Efficient use of powered infrastructure when available • Conceptually and practically simple • 802.15.4 MAC w/ extensions • Provide framework to approach limits of the radio • 16x250kbps, ~1ms packets 2

3. Statement of Religious Alignment • Time synchronization is required • Application • Low power • Multi-channel • Multi-channel is required • Reliability • Bandwidth, scale 3

4. Semi-active Channel-hopping 802.15.4 Slot and superframe timing • Slot length • When SO = 0  60 symbols  0.96ms • Active superframe duration • 16 slots  15.36ms when SO=0 • Superframe duration • 15.36ms * 2BO ; BO = 0..14 • Up to 4 minutes (> 250,000 slots) 16|17|18|19|20|21|22|23|24|25|26|27|28|… 4

5. Frame Unallocated Slot Allocated Slot Timeslots and Frames • Each mote-to-mote communication happens within a scheduled timeslot • All timeslots are contained within a frame • Frames repeat in time • Multiple frames can operate simultaneously within a network 5

6. RX start, CCA, RX->TX Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC Tx->Rx Tg ACK RX ACK Slot Structure Transmit Time Slot Device Current Transmit operations (not to scale) 6

7. Radio RX startup Empty RX Idle listen (no packet exchanged) Mote Current Energy cost (2004): 70 uC 7

8. D B Time Slot and Channel Mapping One Slot Time • The two links from B to A are dedicated • D and C share a link for transmitting to A • The shared link does not collide with the dedicated links Chan. 2.405 GHz A BA 2.470 GHz CA DA C 2.445 GHz BA 2.425 GHz Slot links for devices 2.475 GHz 2.440 GHz … 2.480 GHz 8

9. Frequency Hopping Each link rotates through k available channels over k cycles. Blacklisting can be defined globally and locally. Time BA BA CA DA BA Channel BA BA CA DA CA DA BA Cycle N+1 Cycle N CycleN+2 9

10. destination > one one • Contention free one source • Collisions possible > one • Unicast • ACKed • Broadcast • Duo-ACK? Link Types Describes the assignments for a single cell = slot X channel_offset 10

11. Performance Limits • Data collection • 100 pkt/s per gateway channel • 16*100 pkt/s with no spatial reuse of frequency • Throughput • ~80kbps secure, reliable end-to-end payload bits per second per gateway • 15 * 80k = 1.2Mbps combined payload throughput w/ no spatial reuse of frequency • Latency • 10ms / PDR per hop • Statistical, but well modeled • Scale • > 1,000 nodes per gateway channel 11

12. Industrial Automation Use Cases Monitoring Diagnostics Configuration Handheld Peer to Peer Phase II Simulation of a 250 node network (courtesy Bob Karschnia) 12

13. C B A Multiple graphs  Multiple frames Time Channel Cycle M+1 Cycle M of red frame 13

14. C B A Frames overlayed Time Channel • Cell collisions can be avoided by time or channel partitioning • Intentional scheduling collisions are resolved by packet priority and graph priority 14

15. B H C G E A F Subnetworks: single-hop, low latency • Black superframe • All motes • 1,000 slots (10 seconds) • Data, Health reports up • Control info down • Red superframe • Mote F is light switch • Mote A is light • 1 slot, ~10ms latency • Blue superframe • Mote H is temp sensor • Mote B is HVAC control point • 100 slots, ~1second latency • Motes A and B are likely powered • All frames on all the time • All other motes run at <100uA 15

16. B H C G Subnetworks 2: reliable multi-hop control • Black superframe • All motes • 10s period • Data, Health reports up • Control info down • Red superframe • ~2s latency • Mote H is industrial process sensor • Mote A is industrial process controller • Both frames on all the time • All motes run at <100uA E A F 16

17. B H C A E G F Without black graph Subnetworks 3: query/response & log upload • Black superframe • All motes • Data, Health reports up • Control info down • Red superframe • Query/response from A to G • 50 slots (0.5 second) • Mean round-trip latency < 1s • Blue superframe • Mote H sends a log file • 2 slots, 1 payload delivered to A per cycle • ~80kbps • Red & Blue frames are only on occasionally • All motes run at <100uA under “normal” conditions • Zero collisions, zero lost packets 17

18. B H S Q C F R P A D Subnetworks 4, et cetera Red frame: 1 packet delivered from G to D every other slot W X Blue frame: 1 packet delivered from H to A every slot E G Y Gold frame: 1 packet delivered from W to Z every other slot Z Green frame: 1 packet delivered from S to P every slot 18

19. Many Knobs to Turn • Trade performance and power • Sample & reporting rate • Latency • High bandwidth connections • Tradeoffs can vary with • Time • Location • Events • Use power intelligently if you’ve got it • Highest performance with powered infrastructure 19

20. Network B H C A Gateway E G F Communication Abstraction • Packets flow along independent digraphs • Digraphs/frames have independent periods • Energy of atomic operations is known, (and can be predicted for future hardware) • Packet TX, packet RX, idle listen, sample, … • Capacity, latency, noise sensitivity, power consumption models match measured data • Build connectivity & applications via gateway or sensor interface • Create & delete graphs • Activate & deactivate graphs • Add & delete links 20

21. Network Management • Secure, Rapid Joining • TJOIN = CT/PD • C = number of joining channels • T = mean time between advertising packet • P = PDR • D = duty cycle • Seconds per mote for reasonable parameter values • Continuous optimization • Global knowledge of CIJ(t) useful, not required • Optimization, not failure recovery - always have alternate paths • Dynamic requirements • Bandwidth on demand • Shared links • Pre-provisioned graphs turned on & off • Wireless worker 21

22. 50 motes, 7 hops 3 floors, 150,000sf >100,000 packets/day

23. Oil Refinery – Double Coker Unit • Scope limited to Coker facility and support units spanning over 1200ft • No repeaters were needed to ensure connectivity • Gateway connected via Ethernet port in control room to process control network • Electrical/Mechanical contractor installed per wired practices GW 400m Unamplified cc2420 85 dB SW-limited link margin 23

24. 1 Protocol, Alternate Approaches • All motes battery operated • Intelligent Management: SmartMesh • Minimal network management: Slotted aloha • Some motes powered • Hybrid: Sleepy Slotted Aloha • Routers powered, leaf nodes minimum power • Point-to-point • Star networks • Compare to CSMA approach for • Latency • Scalability • Power consumption 24

25. Moving forward • Radio RX current going down • QIDLE < 10uC  • listening every slot < 1mA • 100ms latency/hop  100uA current • Embedded microprocessor capabilities scaling at least 10x • 32 bit cores, > 1MB flash, >128kB RAM, 100MHz • Lower current! • Our standard should embrace these changes 25

26. 26

27. 1,100 m • 1400 Motes -20 Managers - 32 Acres 600 m Scalability: Outdoor Test Network Approaching 8 mote-centuries 27

28. Additional link types • “Primary”, “secondary” parent 28

29. Slotted Aloha performance • Peak payload bandwidth • 100 slots/sec * 80B/packet = 8kB/s = 64kbps • Peak payload goodput with collisions • 64kbps * e-1 = 23.5kbps • Average power consumption, non-congested • IRX * (2ms/10ms) = 0.2 IRX • Average latency, non-congested • 10ms/hop • Relative to Aloha w/ 80B ACKed payloads • Payload goodput = ~150kbps * e-2 = • Average Current = IRX • Non-congested Latency 5ms/hop • Relative to Aloha w/ 10B ACKed payloads 29

30. Duty Cycling • Slotted aloha • Any fractional slot duty cycle a possible with varying frame length • Use x links in a y slot frame to get a = x/y duty cycle • Current decreases proportional to a • Latency increases as 1/a • 10% slot duty cycle  100ms latency per hop. • Radio duty cycle is still lower, i.e. 2% (=10% slot duty cycle * 20% radio duty cycle in slot) • Aloha • “chunky-ness” of the duty cycle will set latency • Typical approaches (e.g. Millennial) use long sleep intervals, e.g. 6 seconds on, 54 seconds off to get 10% duty cycle • Latency is tens of seconds, radio duty cycle is same as overall duty cycle, =10% 30

31. Powered routers 31

32. Point to point links • Use scheduled communication, e.g. one Tx and one RX link in a two slot frame • Available guaranteed bandwidth • 50 slots/sec in each direction • =50 payloads/sec = 4kB/s = 32kbps full duplex • Idle current • 50 listens/sec * 2ms/listen = 100ms/s = 10% radio duty cycle • Average latency = 1 slot = 10ms Or… • Use bandwidth on demand (slotted aloha), e.g. one aloha slot in a one slot frame • Available one-way bandwidth is 100 packets/sec = 64kbps • Average One-way latency is 5ms 32

33. Star connected networks • 1 hub, N end-points • Scheduled communication • 1 downstream broadcast, N upstream links in a 1+N slot frame • Downstream bandwidth = 100 packets/sec / (1+N) • Average Latency = 10ms * (1+N)/2 • Bandwidth on demand • 1 downstream broadcast, 1 aloha in a 2 slot frame • Downstream bandwidth = 50 packets/sec • Peak upstream bandwidth, 1 mote = 50 packets/sec • Average Query/response latency = 3 slots = 30ms • End-point radio duty cycle = 50% slot * 20% rx/slot = 10% • Reducing endpoint duty cycle • E.g. 5 downstream slots/sec • Average query/response latency = 120ms • End-point radio duty cycle = 5% * 20% = 1% 33

34. Variable slot length • Slot length will be a variable number of 1/1024ths of a second, hereafter referred to as milliseconds for convenience. • Expected values for slot length are 8-20ms. • Single slot length networks • All slots, frames, and motes in a network will use the same slot size, chosen by the first mote in the network (e.g. the gateway) • There will be no provision for changing the slot length of an existing network without restarting the network. Hence, a “slow” mote (e.g. without hardware crypto) would not be able to join a fast network. • Problem: Rob won’t go for it for “bad customer experience” • Multi- slot length networks • Different frames could have different slot lengths • Different paths could have different slot lengths. Manager blocks out the appropriate amount of time for each link. • Dual slot length • 8-10ms slot length • M2135 motes can only handle even-numbered slots 34

35. CCA: RX startup, listen, RX->TX Timing – perfect synchronization A Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC RX startup or TX->RX RX ACK B RX startup RX packet Verify CRC Verify MAC MIC Calculate ACK MIC+CRC Transmit ACK RX/TX turnaround A transmits to B TX, RX ACK timing 35

36. CCA: RX startup, listen, RX->TX Timing – imperfect synchronization (latest possible transmitter) A Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC RX startup or Tx->Rx Tg ACK RX ACK Tcrypto B RX startup Tg Tg RX packet Verify CRC Verify MAC MIC Calculate ACK MIC+CRC Transmit ACK RX->TX Expected first bit of preamble • TCCA = 0.512ms to be standards compliant • Worst case is a receive slot followed by a transmit slot to a different partner, as radio will be finishing up the ACK TX just as it needs to look for a clear channel, so • TCCA = TTX->RX + Tchannel assessment + TRX->TX = 0.192ms + 0.128ms + 0.192ms • With gold24, we believe we can do a faster turnaround, so we’d get 0.228 instead of 0.512 • Tpacket = 4.256ms for a maximum length packet • Preamble+SS+packet = 4+1+128B = 133B = 1064 bits  4.256ms @ 250kbps • Tcrypto needs to be chosen. For gold24 it will be about 0.25 or 0.5 ms. For the cc2420 it appears to be a bit slower – maybe 0.5 to 1 ms. • TgACK needs to be chosen. It is the tolerance to variation in Tcrypto and/or mote B’s turnaround time from RX to TX • TACK is a function of the ACK length. It is likely to be just under 1ms. • Tslot = TCCA+2*Tg+Tpacket+Tcrypto+TgACK+TACK = 0.512+2+4.256+1+0.1+1 = 9ms 36

37. TX, RX ACK (late) Late TX, early neighbor TX next slot X X Preamble+SS, 160us TX, RX ACK (early) Expected first bit of preamble CCA = 178us Expected first bit of preamble First bit of late transmitter shows up at +X relative to network-wide clock. That late transmitter performed a CCA starting 178us earlier. The early transmitter in the next slot wakes up early enough to perform a CCA and get the first bit of its preamble out at –X relative to network-wide clock. The last bit of the late transmitter is done before the first sniff of the early CCA has taken place. 37

38. Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC Transmit Packet: Preamble, SS, Headers, Payload,MIC, CRC Tcrypto Tcrypto Tcrypto Tcrypto Tcrypto Tcrypto TACK TACK TACK TACK TACK TACK TCCA TCCA TCCA TCCA TCCA TCCA Tg Tg Tg Tg Early Perfect Late Tcomm = Tpacket+Tcrypto+TACK Tslot = 2Tg+Tcomm+TCCA Tcrypto includes TgACK and all CRC, crypto, and radio turnaround times. It’s the time from the last bit of the packet to the first bit of the preamble of the ACK. 38

39. Star-mesh or Star-LAN Q: Star-connectivity is known to be death for reliability, so why do it? A: Don’t trust the motes, don’t think that they have the power to be routers. 39

40. Star-mesh or Star-LAN What if WiFi gets jammed (easier to do than freq-hopping 802.15.4)? What if you lose ethernet? (power failure, cable, …) 40

41. Mesh, with backbone Use powered infrastructure when you have it. Lower latency Lower power But, if it goes away… 41

42. Mesh, with backbone Assume that the motes are smart, and that their radios are good. Use protocols that leverage those capabilities: Time-synchronized, TDMA, Channel Hopping MAC Mesh routing 42

43. TSMP Dedicated Services • Periodic traffic • Time Division Multiplexing assigned to slots in frames • Dedicated access and Quality of Service • Deterministic latency • Bandwidth assignment • Configurable latency • Transport and resource priority • Connectivity • One-to-one • One-to-many 43

44. TSMP Shared Services • Used for burst traffic • Provides pool of available slots as needed • Low latency alarms • High-speed on-demand file transfer • Slotted Aloha assigned to slots in frames • Time slots can be configured to be shared • MAC level ACK detects collisions • Exponential back off algorithm • Transport and resource priority • Connectivity • Many-to-one • Many-to-many 44

45. TSMP Network Management • Unified resource allocation • Dynamic: • Adapts to changing RF environment – global response to local changes • Robust against network device failures • Responds to application resource requests and provides QOS • Optimized: allocation of resources across the network • Flexible: Network management and device interoperability do not require the standardization of how resources are allocated • Innovations can be added after the standard is released • Specialized network managers can target vertical markets • Secure: Critical functions are removed from physically unsecured locations 45