Spatio-Temporal Case-Based Reasoning for Behavioral Selection

1. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Maxim Likhachev and Ronald Arkin Mobile Robot Laboratory Georgia Tech

2. Part of Multi-Level Learning in Hybrid Deliberative/Reactive Mobile Robot Architectural Software Systems project at Georgia Tech Sponsored by the DARPA MARS program Spatio-Temporal Case-Based Reasoning for Behavioral Selection Broad Picture of the Work Maxim Likhachev and Ronald Arkin

3. Constant parameterization of robotic behavior results in inefficient robot performance Manual selection of “right” parameters is difficult and tedious work Spatio-Temporal Case-Based Reasoning for Behavioral Selection Motivation Maxim Likhachev and Ronald Arkin

4. Use of Case-Based Reasoning methodology for an automatic selection of optimal parameters in run-time Spatio-Temporal Case-Based Reasoning for Behavioral Selection Motivation (cont’d) Maxim Likhachev and Ronald Arkin

5. Simulations Real robot ATRV-JR in outdoor environment Nomad 150 in indoor environment Spatio-Temporal Case-Based Reasoning for Behavioral Selection Evaluated on: Maxim Likhachev and Ronald Arkin

6. ACBARR, SINS and KINS systems use of case-based reasoning and reinforcement learning for the optimization of behavioral parameters contribute to some ideas behind the present algorithm Automatic optimization of parameters genetic programming reinforcement learning Spatio-Temporal Case-Based Reasoning for Behavioral Selection Related Work Maxim Likhachev and Ronald Arkin

7. CBR Module controls: Weights for each behavior BiasMove Vector Noise Persistence Obstacle Sphere Spatio-Temporal Case-Based Reasoning for Behavioral Selection Behavioral Control and CBR Module Maxim Likhachev and Ronald Arkin

8. Vector of spatial characteristics of environment D - distance to the goal <σ, r> - degree of obstruction and distance to the most obstructing cluster of obstacles for each of K angular regions around the robot Spatio-Temporal Case-Based Reasoning for Behavioral Selection Input Features for Case Selection Maxim Likhachev and Ronald Arkin

9. Vector of temporal characteristics of environment Rs - short term robot movement Rl - long term robot movement Spatio-Temporal Case-Based Reasoning for Behavioral Selection Input Features for Case Selection Maxim Likhachev and Ronald Arkin

10. F: represents traversability of each region approximates obstacle density function around the robot independent of goal distance smoothed over time: Spatio-Temporal Case-Based Reasoning for Behavioral Selection Computation of Traversability Vector F Maxim Likhachev and Ronald Arkin

11. Spatio-Temporal Case-Based Reasoning for Behavioral Selection f1=0.58 f0=0.92 f2=1.0 f3=0.68 Vtemporal ShortTerm: Rs=1.0 LongTerm: Rl=0.7 Vtemporal ShortTerm: Rs=0.01 LongTerm: Rl=1.0 f1=0.22 f0=0.02 f2=0.63 f3=0.02 Input Features: Example Maxim Likhachev and Ronald Arkin

12. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Set of Spatially Matching cases Spatial Features Vector Matching (1st stage of Case Selection) Temporal Features Vector Matching (2nd stage of Case Selection) Feature Identification Spatial Features & Temporal Features vectors Current environment Set of Spatially and Temporally Matching cases All the cases in the library Case Library Random Selection Process (3rd stage of Case Selection) Best Matching case Case Output Parameters (Behavioral Assemblage Parameters) Case Adaptation Best Matching or currently used case Case Application Case ready for application Case switching Decision tree High Level Structure of CBR Module Maxim Likhachev and Ronald Arkin

13. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Case Example I CLEARGOAL Spatial Vector: D (goal distance) = 5 density distance Region 0: σ0 = 0.00; r0 = 0.00 Region 1: σ1 = 0.00; r1 = 0.00 Region 2: σ2 = 0.00; r2 = 0.00 Region 3: σ3 = 0.00; r3 = 0.00 Temporal Vector: (0 - min, 1 - max) ShortTerm_Motion Rs = 1.000 LongTerm_Motion Rl = 0.700 Case Output Parameters: MoveToGoal_Gain = 2.00 Noise_Gain = 0.00 Noise_Persistence = 10 Obstacle_Gain = 2.00 Obstacle_Sphere = 0.50 Bias_Vector_X = 0.00 Bias_Vector_Y = 0.00 Bias_Vector_Gain = 0.00 CaseTime = 3.0 Maxim Likhachev and Ronald Arkin

14. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Case Example II FRONTOBSTRUCTED_SHORTTERM Spatial Vector: D (goal distance) = 5 density distance Region 0: σ0 = 1.00; r0 = 1.00 Region 1: σ1 = 0.80; r1 = 1.00 Region 2: σ2 = 0.00; r2 = 1.00 Region 3: σ3 = 0.80; r3 = 1.00 Temporal Vector: (0 - min, 1 - max) ShortTerm_Motion Rs = 0.000 LongTerm_Motion Rl = 0.600 Case Output Parameters: MoveToGoal_Gain = 0.10 Noise_Gain = 0.02 Noise_Persistence = 10 Obstacle_Gain = 0.80 Obstacle_Sphere = 1.50 Bias_Vector_X = -1.00 Bias_Vector_Y = 0.70 Bias_Vector_Gain = 0.70 CaseTime = 2.0 Maxim Likhachev and Ronald Arkin

15. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Results Simulations: Average travel distance Mission success rate ATRV-JR: 12% average performance improvement in time steps ( based on 10 runs for each system in outdoor environment) Maxim Likhachev and Ronald Arkin

16. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Simulations & real robot experiments: Performance improvement as a function of obstacle density Simulations Nomad 150 Based on 10 runs for each system in indoor environment Maxim Likhachev and Ronald Arkin

17. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Real Robot Run with CBR Maxim Likhachev and Ronald Arkin

18. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Real Robot Run without CBR Maxim Likhachev and Ronald Arkin

19. Spatio-Temporal Case-Based Reasoning for Behavioral Selection Trajectories of the robot Robot with CBR module Robot without CBR module 11% less travel distance Maxim Likhachev and Ronald Arkin

20. Automatic selection of optimal behavioral parameters results in robot performance improvement (based on simulations and real robot experiments) Careful manual selection of behavioral parameters is no longer required from a user Future Work Automatic learning of cases: identifying when to create a new case applying reinforcement learning techniques in finding optimal parameters for existing cases Integration with other adaptation & learning methods (e.g., Learning Momentum) Spatio-Temporal Case-Based Reasoning for Behavioral Selection Conclusions Maxim Likhachev and Ronald Arkin