ISLE Transfer Learning Team Main Technology Components

1. ISLE Transfer Learning TeamMain Technology Components Long-Term Memories Procedural Episodic Semantic Generates beliefs using observed environment and long term conceptual knowledge Creates internal description of the perceived environment Contains descriptions of the perceived objects Contains relational and hierarchical knowledge about relevant concepts Perceptual Buffer Reinforcement Learning Chunking Semantic Learning Episodic Learning Long-Term Conceptual Memory Short-Term Conceptual Memory Conceptual Inference Perception Short-Term Memory Decision Procedure Contains inferred beliefs about the environment Selects relevant skills based on beliefs and goals Environment Skill Retrieval Perception Action Contains hierarchical knowledge about executable skills Finds novel solutions for achieving goals Contains goals and intentions Body Long-Term Skill Memory Short-Term Goal/Skill Memory Problem Solving Skill Learning Skill Execution Motor Buffer Acquires skills from successful problem solving traces Executes skills in the environment The Soar Architecture Markov Logic Networks Inductive Logic Programming Weight Learning Source Domain Target Domain Markov Logic Weighted Satisfiability Markov Chain Monte Carlo The ICARUS Architecture The Companions Architecture

2. Year 1: Transfer in Three Testbeds Urban Combat is a first-person shooter game that involves spatial reasoning and reactive control General Game Playing covers a broad class of N-person games that involve strategic reasoning. Crawl under Climb over source source Year 1 emphasizes lower levels (1 to 8) of transfer learning target target Urban Combat ETS Physics GGP Soar Our Year 1 efforts focused on Urban Combat (Soar and ICARUS), but with some work on GGP (ICARUS) and ETS Physics Companions) Companions ETS Physics involves finding answers to physics problems through a mixture of plausible inference and search ICARUS

3. ISLE Team: Year 2 Integration Plans β(A’)→A γ(S’)→S 1  2  Year 2 integration will revolve around replacing existing ICARUS modules with software from other team members Inductive Logic Programming Weight Learning ICARUS (ISLE) Perceptual Buffer Markov Logic Long-Term Conceptual Memory Short-Term Conceptual Memory Conceptual Inference Perception Weighted Satisfiability Markov Chain Monte Carlo Environment Skill Retrieval Alchemy (Washington) Insert Cycorp Mountain here Long-Term Skill Memory Short-Term Goal/Skill Memory Problem Solving Skill Learning Skill Execution Motor Buffer Reinforcement Learning (UT Austin) CYC (Cyrcorp) Hierarchical Task Networks (Maryland)

4. Year 2: Deep Transfer in Three Testbeds In Urban Combat, we will demonstrate transfer from urban military missions to fire rescue operations In General Game Playing, we will show transfer across quite different games that use related strategies (e.g., forking moves) Crawl under Climb over source source Year 2 will focus on higher levels (9 -10) of transfer learning target target Urban Combat ETS Physics GGP Soar In Year 2, We will evaluate each pair of architectures on at least one shared testbed and we will test them all on the General Game Playing testbed source Companions In ETS Physics, we will show transfer from linear systems to rotational, thermal, hydraulic, and electrical systems ICARUS target

5. ISLE Team: Year 2 Evaluation Plans Testbeds Urban Combat ETS Physics Comparison among architectures that use different mechanisms should reveal which approaches best support transfer learning. We will evaluate each pair of architectures on at least one shared testbed and will test them all on the General Game Playing testbed. Experiments will examine how well the frameworks support transfer in settings that emphasize reactive control, conceptual inference, and heuristic search. Year 2 evaluations will focus on high-level (9 and 10) transfer in each of the three testbeds. GGP Soar Architectures Companions ICARUS