Controlling and Configuring Large UAV Teams

1. Controlling and Configuring Large UAV Teams Paul Scerri, Yang Xu, Jumpol Polvichai, Katia Sycara and Mike Lewis Carnegie Mellon University and University of Pittsburgh

2. Context • Aim to build large heterogeneous teams for complex tasks • Robots, agents, people • 10,000s of actors • Multiagent version of Belief Desire Intention approach to autonomous behavior • Builds on key abstraction of a Team Oriented Plan • Defines the activities that must take place and interactions between those activities • Supported by extensive theoretical (logical) work • Key algorithms are NP-complete or worse • Heuristics required for scalability

3. Specific Target Application: Wide Area Search Munitions (WASMs) • Part munition, part unmanned aerial vehicle • Single use • Variety of sensors • Limited fuel supply, approximately 30 minutes • Communicate with each other, manned aircraft • Concept of Operations (under development) • Small number of manned aircraft • Potentially other ground forces • 100s of WASMs, performing a variety of missions • Attack • Search • Battle damage assessment • Decoys • Communication relays • Flight test planned September, 2005 • 1 real and 3 simulated WASMs

4. Token-Based Coordination • Token: self contained packet capable of being sent around team • Information content • Control content • Local models • Team members use receipt of tokens to create local models of other team members • What sorts of things are they/are they not working on? • What sorts of things might they need to know? • Local models are used to improve the routing of future tokens • Token “flows” implement coordination • No brittle, non-scalable “message protocols”

5. Token Based Algorithms • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings

6. Token Based Algorithms • Information about environment passed around in tokens, when agent receives tokens matching plan pre-conditions plan is initiated • Very high probability all applicable plans are initiated • Liao et al, 2004 • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings

7. Token Based Algorithms • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings • Information about initiated plans shared in tokens • Very high probability some agent gets to find out about any duplicate plans • Liao et al, 2004

8. Token Based Algorithms • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings • Responsibility to perform a role encapsulated in token • Only team member with token can perform role • Team member must have capability > threshold to perform role • Threshold calculated from estimates of likely role allocation outcome • Okamoto, 2004

9. Token Based Algorithms • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings • Locally sensed information shared via token to get information to team members performing role effected by information • Does not require sensing agent to know who needs information • Xu et al, 2004

10. Token Based Algorithms • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings • Access to resource represented by token • Only team member with token can use resource • Team member must have need > threshold to keep resource • Threshold changes dynamically as it moves around the team, seeing resource need

11. Token Based Algorithms • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings • Uncertain sensor readings encapsulated in tokens and sent around the team • Very high probability that at least one team member gets related sensor readings and fuses for higher confidence

12. Token Based Algorithms • Plan instantiation • Removal of duplicate plans • Role allocation • Information sharing • Resource allocation • Sensor fusion • Recovering from faulty sensor readings • Assumptions used to make decisions are attached to tokens resulting from that decision • Very high probability agents with contradictory information see the assumptions and initiate checking process

13. Meta Token Routing Model Local Layered Team Member Architecture • Bottom: Local Reasoning Team member’s local actions are restricted by the tokens they have • E.g., without resource token, cannot use resource • Middle: Coordination Reasoning Movement of tokens around the team implements the coordination • Flows of tokens • Local models inform token routing • Top: Meta-reasoning Ensures token flows work effectively

14. Synergies Between Token Algorithms • Overall performance depends on how well tokens are routed • Team members use local models to improve routing • Observation: Execution of algorithms shares information that other algorithms can exploit • E.g., if role for strike near Pittsburgh was allocated to WASM X, then air space resources around Pittsburgh likely of interest to WASM X • Implementation: Use all tokens to improve local models • E.g., role tokens change local models, resource tokens move according to local models • Result: When all algorithms are working together the system actually performs better

15. Implementing Synergies - Example • When receiving an role token from neighbor i • Probability of sending information token to i changed proportionally to similarity between “plan initiated” token and information token • Probability of sending resource token to i changed inverse proportionally to similarity between role and resource token

16. Meta-Reasoning for Token Flows • Specific tokens are meta-reasoned about • Identify tokens that are not behaving as expected • Bring to human attention • Examples • Tokens that “live” too long • Tokens that travel too much • See Scerri et al, AIAA 2004 • Overall flows can be controlled to optimize for specific criteria • By controlling the flows, we control how the coordination works

17. Neural Networks for Modeling and Controlling Token Flows • Use a simple input/output feed-forward neural networks to represent a team performance model • Three-layer FFNN is capable of representing any arbitrary functions • Extend to Dynamic Networks concept to cope with dynamic behaviors • This kind of network enlarges the capacity to deal with non-deterministic problem • Learn the model with genetic algorithms methodology • Excellent for dealing with very huge and noisy training data set • Based on around 1,000,000 simulation runs

18. Configuring Algorithms with NN • Offline • Change environment and algorithm parameters, observe expected performance • Online • Use NN “in reverse” to find parameter settings for specific optimization criteria • As environment changes • As requirements change • Allows tradeoff between performance of all algorithms

19. Configuration Interface

20. Results: Token Based Coordination Messages Total Reward Fully Integrated Random

21. Results: Configuring Algorithms

22. Results: Online Control

23. Machinetta: Bringing it all Together • Encapsulate token-based coordination approach into reusable software module • Called a proxy • Proxies provide domain independent coordination algorithms • Do not provide domain specific communication and interface code • Available in the public domain • Machinetta used to demonstrate coordination of up to 500 distributed, heterogeneous team members in several distinct domains • Demonstrates that token based coordination is feasible • Biggest teams developed to date?

24. Conclusions and Future Work • Token-based coordination as a feasible alternative paradigm for large teams • Additional layer over token flows gives high levels of control • Future Work • Can we make more precise mathematical models? • Markov chains?