Resource-Bounded Machines are Motivated to be Effective, Efficient, and Curious

1. Resource-Bounded Machines are Motivated to be Effective, Efficient, and Curious Bas R. Steunebrink, Jan Koutník, Kristinn R. Thórisson, Eric Nivel, Jürgen Schmidhuber The Swiss AI Lab IDSIA, USI & SUPSI, Reykjavik University

2. The Main Argument • Explicitly acknowledge resource constraints • Identify the constrained resources • Design AI system to be driven by better and better resource utilization • Order activities around resource utilization • Emergent result: effectiveness, efficiency & curiosity

3. Fundamental Resources

4. Resource Compression: Why? • Less time & energy spent  more reward (resources shared with other agents) • Less time & energy spent  more left for future tasks • Compression of input = learning  better prepared for unknown future

5. Driven by Resource Compression Progress Minimize resource consumption through: • Knowledge • Learn new more effective & efficient routines • Curious exploration to “fill knowledge gaps” • Architecture • Re-encode known routines for more effective & efficient execution • Example: self-compilation

6. Work—Play—Dream Framework • Utilize the kinds of activities afforded by the patterns of interaction with human teachers / supervisors / users • Work: fulfill main purpose, no exploration, store interesting / unexpected events • Play: curiosity-driven exploration, perform experiments, may still requires supervision • Dream: analyze unprocessed events, self-compilation, task invention for Play

7. WPD Framework, cont. • Work—Play—Dream are processes, not states • Can run in parallel, but sometimes not possible due to resource/situational constraints • Combination of Work & Play leads to creativity • Dreaming may be a necessary side-effect for any system constrained in computational power and memory • “Tired” = buffers reach capacity • “Dream” = process input history backlog

8. AERA: An Explicitly Resource-bounded Architecture • Knowledge is operationally constructive • Model-based & model-driven, hierarchically • Simulation through forward chaining • Planning through backward chaining • Compilation of useful & reliable chains • Originally for scalability • Now realized to satisfy the architectural way of achieving resource compression: re-encoding

9. How AERA does resource compression • Consider 3 resources: Time, RAM, HDD • Time more precious than memory • Self-compilation leads to better resource utilization • Thus AERA must be motivated to self-compile • Simple analysis of control values yields goals that give rise to empirical testing of unstable models • Crux: scheduling needed  Work, Play, Dream

10. Conclusion • AERA is being developed as a cognitive architecture towards AGI • Based on many firm principles, but not curiosity • But all ingredients for curiosity are present • Learning & re-encoding both possible! • Thanks to self-compilation ability • Resource usage compression = principled middle ground • No twisting of AERA or Theory of Curiosity • AERA still based on solid principles • Curiosity generalized to resources-bounded view