Neural Robot Control

1. Neural Robot Control Cornelius Weber Hybrid Intelligent Systems University of Sunderland Talk at Nottingham Trent University, 8th December 2004 on the occasion of returning the MI competition trophy Collaborators: Mark Elshaw, Alex Zochios, Chris Rowan and Stefan Wermter

2. Contents • Visual cortex & reinforcement network for docking • Cortex self-imitation network for docking • Imitation networks for multiple actions: 1-stage/2-stage hierarchical network • Outlook

3. Contents • Visual cortex & reinforcement network for docking • Cortex self-imitation network for docking • Imitation networks for multiple actions: 1-stage/2-stage hierarchical network • Outlook

4. Example Task: Docking

5. Docking ArchitectureInformation Flow

6. Docking ArchitectureTraining (1/3) unsupervised training generative model sparse distributed coding

7. V1 Receptive Fields(training result)

8. Comparison ofResponse Characteristics linear sparse competitive winner

9. Attractor Network:Competition via Relaxation weight profile activation profile activation update y(t+1) = f (Wlat y(t))

10. Docking ArchitectureTraining (2/3) supervised training, attractor network for pattern completion

11. Docking ArchitectureVisual System

12. Docking ArchitectureTraining (3/3) reinforcement training actor-critic model

13. Contents • Visual cortex & reinforcement network for docking • Cortex self-imitation network for docking • Imitation networks for multiple actions: 1-stage/2-stage hierarchical network • Outlook

14. Mirror NeuronDocking ArchitectureInformation Flow

15. Mirror NeuronDocking ArchitectureTraining unsupervised training generative model distributed coding

16. Mirror NeuronDocking ArchitectureTraining supervised training, attractor network for prediction

17. Mirror Neuron Self-ImitationDocking ArchitectureInformation Flow

18. Basal Ganglia vs. Motor Cortex Basal ganglia units are active during early task acquisition but not at a later stage (rat T maze decision task). Jog et al. (1999) Science, 286, 1158-61 early: late: Basal Ganglia ≙ state space? Motor cortex might take over BG function via self-imitation.

19. Docking via Mirror Neurons

20. Contents • Visual cortex & reinforcement network for docking • Cortex self-imitation network for docking • Imitation networks for multiple actions: 1-stage/2-stage hierarchical network • Outlook

21. Simulated Robot Environment

22. Imitation Model Choice

23. Areas of Motor- and Language Representations motor units forward back left right individual unit’s receptive fields in hidden area language units ‘go’ ‘pick’ ‘lift’ all

24. Areas of Task-Specific Activations ‘go’ ‘pick’ ‘lift’ Production: Recognition: Activations agree with the Somatotopy-of-Action-Words Model. ‘go’ ‘pick’ ‘lift’

25. Language InstructedImitative Behaviour ‘go’ ‘pick’ ‘lift’

26. Imitation Model Choice

27. Neuron’s Receptive Fields in HM Area forward backward left right motor units 4 SOM-area units

28. Conclusion for Imitation Network A neural network as a generative model for sensory stimuli • generates interactive action sequences • allows for context dependent interactive action sequences

29. Contents • Visual cortex & reinforcement network for docking • Cortex self-imitation network for docking • Imitation networks for multiple actions: 1-stage/2-stage hierarchical network • Outlook

30. Outlook (1/2): Object-Background Separation for Enhanced Object Learning

31. Outlook (2/2): Docking Range Extension by Neural Coordinate Transformations