1 / 39

Prologue - Making Models Easy to Use

Prologue - Making Models Easy to Use. Elkind, Card, Hochberg, & Huey. (1988) - We need work Pew & Mavor,for sim & training books.nap.edu/catalog/6173.html - Ready to use

leesmalls
Download Presentation

Prologue - Making Models Easy to Use

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prologue-Making Models Easy toUse • Elkind, Card, Hochberg, & Huey. (1988)- We need work • Pew & Mavor,for sim & trainingbooks.nap.edu/catalog/6173.html- Ready to use • Ritter, Shadbolt, Elliman, Young, Gobet, Baxter (1998/2003)iac.dtic.mil/hsiac/S-docs/SOAR-Jun03.pdf- Usability also counts- Lists of open tasks • Pew & Mavor, 2007, for HSI

  2. Making Soar More Articulate and Modeling More Affordable through Explanation, Interaction, & Environment Frank E. Ritter, Steven R. Haynes, and Mark A. Cohen Applied Cognitive Science Lab School of Information Sciences and Technology Penn State Since November 2001.  acs.ist.psu.edu/papers ritter@ist.psu.edu Working with the Herbal group, and past contributions fromSue Kase, Geoff Morgan, Andrew Reifers, Ian Schenck. This project is supported by the ONR N00014-02-1-0021

  3. Cognitive Model Affordability has Many Dimensions • What to do and How to do it  Ritter, F. E. (2004). Choosing and getting started with a cognitive architecture to test and use human-machine interfaces. MMI-Interaktiv-Journal, 7, 17-37. useworld.net/mmiij/musimms. • Better languages • Reusability • Better training materials (e.g., tutorials, environments like Tank-Soar) • Making models explainable-What we do • Tools based on user needs and theories of explanation and HCI • Software techniques to create the tool • Allow models to access interfaces • Allow modelers access to models (SPICE/FSF)

  4. Presentation Outline • 1 Herbal, IDE and displays • Studies that lead to it • Design • Implementation • Evaluation (3x, pre-BS vs. MS, 8x code expansion) • Applications to 2 USMC programs • 2. G2A, compiler • Implementation • Evaluation (5x to 30x speedup, 7.5 x expansion) • 3. SegMan, simulated eyes and hands • Evaluations (e.g.,  of time to tie to some interfaces) • Applications ARL interested, but needs to be more packaged • 4. dTank, ModSAF-sandbox • Design • Implementation • Evaluation (UG vs. MS, 100 x faster to tie to than ModSAF) • Applications to DMSO, USAFOSR, MoD projects • Summary • Lessons learned • Speedups • Future work

  5. Increased Access to Agent Knowledge (35) Lack of Information (7) Radio Log (1) Milestones (9) Visual Display of Agent Knowledge (7) Proposal for New Tool to Access Agent Knowledge (7) Goal Stack (17) VISTA Explanation Facility (6) Clarity (4) Positive (4) Content (5) Negative (2) Clarity (4) Superfluous (5) Content (6) What Users Want(based on 4 expert SAP users, Councill et al., 2003) Queries (111) Aspects not yet in Current Model (4) Meta and simulation queries and comments (5) Divergent Purposes (5) Usefulness of Existing Knowledge Access Tools (33) On-Topic Questions (36) Queries of agent state (36) Need For Why-Type Explanation (14) Summary: Better description of procedural knowledge neededMany smaller parts to explain

  6. Result of Analysis - Questions Asked(Haynes, Ritter, Councill, & Cohen, submitted, 2005)

  7. Analysis of Soar Explanation Elements • Design-based analysis of CGF explanation-seeking questions [12 experts x 1 hours]( Haynes et al., 2004, summary to AAAI Wkshp) • Existing Solutions with Lessons • Visual Soar, SoarDoc, ViSoar • G2A, TAQL, CAST, HLSR • All graphical IDEs, e.g., JACK, iGen, Cogent • Systems that use ontologies

  8. Herbal Design • Design rationale anchors model explanations ( Haynes, 2001;  Haynes et al., 2004) • Design knowledge capture during development • Augment existing planning language with design rationale • Chose RDF and Protege : tool availability, generality • Also studied direct translation ( St. Amant, Freed, & Ritter, 2004) • Model description needed for explanations ( Haynes, 2003) • Create model parts within IDE • Explanation from declarative representation + behavior (trace) + rationale (Haynes, Cohen, & Ritter, forthcoming)

  9. Herbal - Reusable Design • Responsibility-driven approach through a compiler • Organize knowledge and rules ( Haynes et al., 2004; St. Amant, Freed, & Ritter, 2004, www4.ncsu.edu/~stamant/G2A) • Compile into Soar rules • (could also compile into ACT-R, Jess, JACK ?) • Designed to leverage VISTA / CaDaDis • (declarative representation supports model tracing)acs.ist.psu.edu/vista for our local training examples

  10. Eye-hand dTank Sim. Users

  11. Ontology Editor (Protégé) Preprocessor XSLT Soar Rules Herbal IDE Design: Language Overview • Features • Captures Design Rationale • Support for Reusable Knowledge Bases • Based on standard tools (Protégé and XML) • Extendable to other architectures • Graphical State Layout • Used by 38+3+30 people Architecture H.Viewer H.Viewer

  12. Herbal- Implementation (IDE)

  13. Herbal- Implementation (IDE)

  14. Herbal IDE (2003): Example Compiler Output sp {select*attack*move2 (state <s> ^operator <o1> + ^operator <o2> + ^io.input-link.status.mana < 10) (<o1> ^name move) (<o2> ^name attack) --> (<s> ^operator <o1> > <o2>)} sp {propose*attack (state <s> ^superstate nil) (<s> ^io.input-link <il>) (<il> ^foodog <foodog> ^refreshed <ref>) (<foodog> ^visible yes ^x <x> ^y <y>) --> (<s> ^operator <o> + =) (<o> ^name attack) (<o> ^x <x> ^y <y> ) } sp {propose*move (state <s> ^superstate nil) (<s> ^io.input-link <il>) (<il> ^location <loc> -^move-blocked yes ^refreshed <ref>) (<loc> ^<dir>.content empty) --> (<s> ^operator <o> + =) (<o> ^name move) (<o> ^direction <dir> ) } sp {apply*ol*attack (state <s> ^operator <o> ^io.output-link <ol>) (<o> ^name attack ^x <xval> ^y <yval>) --> (<ol> ^attack <action>) (<action> ^x <xval> ^y <yval>) } sp {apply*ol*move (state <s> ^operator <o> ^io.output-link <ol>) (<o> ^name move ^direction <dir>) --> (<ol> ^move <action>) (<action> ^direction <dir>) } sp {select*attack*move1 (state <s> ^operator <o1> + ^operator <o2> + ^io.input-link.status.mana >= 10) (<o1> ^name attack) (<o2> ^name move) --> (<s> ^operator <o1> > <o2>)} STATE top-state OPERATOR attack PRECOND INPUT ^foodog <dog1> PRECOND <dog1> ^visible yes ^x <x1> ^y <y1> OUTPUT attack <x1> <y1> OPERATOR move PRECOND INPUT ^location <loc> -^move-blocked yes PRECOND <loc> ^<direction>.content empty OUTPUT move <direction> OutputACTION attack x1 y1 EFFECT ^x <x1> ^y <y1> OutputACTION move dir EFFECT ^direction <dir> PREFERENCE attack move CHOICE attack ^il.status.health >= 10 CHOICE move ^il.status.health < 10

  15. Herbal IDE 2004: Compiler Output (~ 4x more information) 5/9 (<OnTop> ^blocks:Bottom <> <Block2> ^blocks:Top <Block1> ^blocks:Bottom <CurrentBottomBlock>) --> (<s> ^operator <o> + =) (<o> ^name |BlocksWorldState*MoveBlockToBlock| ^Block1 <Block1> ^Block2 <Block2> ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop>) } #-------------------------------- # Production that applies operator: MoveBlockToBlock # Operator Definition: # Moves a clear block on top of another clear block # Operator actions: # Moves Block1 onto Block2 #-------------------------------- sp {apply*BlocksWorldState*MoveBlockToBlock (state <s> ^herbal:Name |BlocksWorldState| ^operator <o>) (<o> ^name |BlocksWorldState*MoveBlockToBlock| ^Block2 <Block2> ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop>) --> 1/9 #-------------------------------- # This Soar model was created using the Herbal Language version 0.4 # Model Name: # Blocks World Model # Author: # Mark Cohen # Created On: # 6/01/2004 # Updated On: # 7/14/2004 # Purpose: # The purpose of this model is to arrange three blocks on a table in a specified order. #-------------------------------- #-------------------------------- # Production that elaborates the top state with initial working memory. # Top State Definition: # The Blocks World Problem Space #-------------------------------- 6/9 (<Block2> ^blocks:Clear |true| -) (<Block2> ^blocks:Clear |false|) (<OnTop> ^blocks:Bottom <CurrentBottomBlock> -) (<OnTop> ^blocks:Bottom <Block2>) (<CurrentBottomBlock> ^blocks:Clear |false| -) (<CurrentBottomBlock> ^blocks:Clear |true|) } #-------------------------------- # Production that proposes operator: MoveBlockToTable # Operator Definition: # Moves a clear block onto a clear table # Proposal Conditions: # If there is a clear block ontop of another block, and the table is clear #-------------------------------- sp {propose*BlocksWorldState*MoveBlockToTable (state <s> ^herbal:Name |BlocksWorldState| ^herbal:WorkingMemory <Block> ^herbal:WorkingMemory <OnTop> ^herbal:WorkingMemory <Table>) (<Block> ^blocks:Clear |true| ^blocks:Type |Block|) (<Table> ^blocks:Clear |true| ^blocks:Type |Table|) 7/9 (<OnTop> ^blocks:Bottom <> <Table> ^blocks:Top <Block> ^blocks:Bottom <CurrentBottomBlock>) --> (<s> ^operator <o> + =) (<o> ^name |BlocksWorldState*MoveBlockToTable| ^Block <Block> ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop> ^Table <Table>) } #-------------------------------- # Production that applies operator: MoveBlockToTable # Operator Definition: # Moves a clear block onto a clear table # Operator actions: # Moves Block from a block onto the Table #-------------------------------- sp {apply*BlocksWorldState*MoveBlockToTable (state <s> ^herbal:Name |BlocksWorldState| ^operator <o>) (<o> ^name |BlocksWorldState*MoveBlockToTable| ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop> ^Table <Table>) --> 2/9 sp {elaborate*initial-state*BlocksWorldState (state <BlocksWorldState> ^type state ^superstate nil) --> (<BlocksWorldState> ^herbal:Definition |The Blocks World Problem Space| ^herbal:Name |BlocksWorldState| ^herbal:WorkingMemory <Table> ^herbal:WorkingMemory <AOnTopOf> ^herbal:WorkingMemory <BOnTopOf> ^herbal:WorkingMemory <COnTopOf> ^herbal:Elaborations <GoalStateReached> ^herbal:WorkingMemory <A> ^herbal:WorkingMemory <B> ^herbal:WorkingMemory <C> ) (<Table> ^blocks:Clear |true| ^blocks:Type |Table| ^herbal:Definition |A table| ^herbal:Name |Table| ) (<AOnTopOf> ^herbal:Definition |A relationship that specifies what block A is on top of| ^herbal:Name |AOnTopOf| ^blocks:Bottom <Table> ^blocks:Top <A> ) 9/9 (<Table> ^blocks:Clear |true| ^blocks:Type |Table|) (<BlockA> ^blocks:Clear |true| ^blocks:Type |Block| ^herbal:Name |A|) (<BlockB> ^blocks:Clear |false| ^blocks:Type |Block| ^herbal:Name |B|) (<BlockC> ^blocks:Clear |false| ^blocks:Type |Block| ^herbal:Name |C|) (<COnTable> ^blocks:Top <BlockC> ^blocks:Bottom <Table>) (<BOnC> ^blocks:Top <BlockB> ^blocks:Bottom <BlockC>) (<AOnB> ^blocks:Top <BlockA> ^blocks:Bottom <BlockB>) --> (write |Goal Reached!|) (halt)} 3/9 (<A> ^blocks:Clear |true| ^blocks:Type |Block| ^herbal:Definition |A block with the letter A printed on it| ^herbal:Name |A| ) (<BOnTopOf> ^herbal:Definition |A relationship that specifies what block B is on top of| ^herbal:Name |BOnTopOf| ^blocks:Bottom <Table> ^blocks:Top <B> ) (<B> ^blocks:Clear |true| ^blocks:Type |Block| ^herbal:Definition |A block with the letter B printed on it| ^herbal:Name |B| ) (<COnTopOf> ^herbal:Definition |A relationship that specifies what block C is on top of| ^herbal:Name |COnTopOf| ^blocks:Bottom <Table> ^blocks:Top <C> ) (<C> ^blocks:Clear |true| 8/9 (<OnTop> ^blocks:Bottom <CurrentBottomBlock> -) (<OnTop> ^blocks:Bottom <Table>) (<CurrentBottomBlock> ^blocks:Clear |false| -) (<CurrentBottomBlock> ^blocks:Clear |true|)} #-------------------------------- # Elaboration: GoalStateReached # Elaboration Definition: # Terminates the model if block A is on block B, block B is on block C, and block C is on the table. #-------------------------------- sp {elaborate*BlocksWorldState*GoalStateReached (state <s> ^herbal:Name |BlocksWorldState| ^herbal:WorkingMemory <AOnB> ^herbal:WorkingMemory <BOnC> ^herbal:WorkingMemory <BlockA> ^herbal:WorkingMemory <BlockB> ^herbal:WorkingMemory <BlockC> ^herbal:WorkingMemory <COnTable> ^herbal:WorkingMemory <Table>) 4/9 ^blocks:Type |Block| ^herbal:Definition |A block with the letter C printed on it| ^herbal:Name |C| ) } #-------------------------------- # Production that proposes operator: MoveBlockToBlock # Operator Definition: # Moves a clear block on top of another clear block # Proposal Conditions: # If there are two clear blocks and the first block is not on top of the second block #-------------------------------- sp {propose*BlocksWorldState*MoveBlockToBlock (state <s> ^herbal:Name |BlocksWorldState| ^herbal:WorkingMemory <Block1> ^herbal:WorkingMemory <Block2> { <Block2> <> <Block1> } ^herbal:WorkingMemory <OnTop>) (<Block1> ^blocks:Clear |true| ^blocks:Type |Block|) (<Block2> ^blocks:Clear |true| ^blocks:Type |Block|)

  16. Herbal IDE 2005: Compiler Output (~ 8x more information) 5/18 # Proposal Conditions: # If there are two clear blocks and the first block is not on top of the second block #----------------------------------------------------------------------------- sp {propose*MoveBlockToBlock (state <s> ^herbal:Name |BlocksWorldState| ^herbal:WorkingMemory <Block1> ^herbal:WorkingMemory <Block2> { <Block2> <> <Block1> } ^herbal:WorkingMemory <OnTop>) (<Block1> ^blocks:Clear |true| ^blocks:Type |Block|) (<Block2> ^blocks:Clear |true| ^blocks:Type |Block|) (<OnTop> ^blocks:Bottom <> <Block2> ^blocks:Top <Block1> ^blocks:Bottom <CurrentBottomBlock>) --> (<s> ^operator <o> + ) (<o> ^herbal:Name |MoveBlockToBlock| ^herbal:Definition |Moves a clear block on top of another clear block, if there is a tie, resolve by stacking blocks correctly| 1/18 #----------------------------------------------------------------------------- # This Soar model was created using the Herbal Language version 1.0 # Compiled On: Tue May 31 08:03:34 CDT 2005 # Model Name: # Blocks World Model # Author: # Mark A Cohen # Created On: # 06/01/2004 # Updated On: # 05/30/2005 # Purpose: # The purpose of this model is to arrange three blocks on a table in a specified order. #----------------------------------------------------------------------------- #----------------------------------------------------------------------------- # Custom Code Included Here #----------------------------------------------------------------------------- # using -a because it works with both 8.5 and 8.6 excise -a init-soar 6/18 ^herbal:Purpose |to move one block onto another block| ^herbal:How-it-works || ^Block1 <Block1> ^Block2 <Block2> ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop>) } #----------------------------------------------------------------------------- # Production that applies operator: MoveBlockToBlock # Operator Definition: # Moves a clear block on top of another clear block, if there is a tie, resolve by stacking blocks correctly # Operator actions: # Moves Block1 onto Block2 #----------------------------------------------------------------------------- sp {apply*MoveBlockToBlock (state <s> ^herbal:Name |BlocksWorldState| ^operator <o>) (<o> ^herbal:Name |MoveBlockToBlock| ^Block2 <Block2> ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop>) --> (<Block2> ^blocks:Clear |true| -) (<Block2> ^blocks:Clear |false|) 9/18 # Production that applies operator: MoveBlockToTable # Operator Definition: # Moves a clear block onto a clear table, if there is a tie, resolve by preferring to stack blocks correctly # Operator actions: # Moves Block from a block onto the Table #----------------------------------------------------------------------------- sp {apply*MoveBlockToTable (state <s> ^herbal:Name |BlocksWorldState| ^operator <o>) (<o> ^herbal:Name |MoveBlockToTable| ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop> ^Table <Table>) --> (<OnTop> ^blocks:Bottom <CurrentBottomBlock> -) (<OnTop> ^blocks:Bottom <Table>) (<CurrentBottomBlock> ^blocks:Clear |false| -) (<CurrentBottomBlock> ^blocks:Clear |true|) } ^blocks:Clear |false| -) (<CurrentBottomBlock> ^blocks:Clear |true|) 2/18 #----------------------------------------------------------------------------- # Production that elaborates the top state with initial working memory. # Top State Definition: # The Blocks World Problem Space #----------------------------------------------------------------------------- sp {elaborate*initial-state*BlocksWorldState (state <BlocksWorldState> ^type state ^superstate nil) --> (<BlocksWorldState> ^herbal:Definition |The Blocks World Problem Space| ^herbal:Name |BlocksWorldState| ^herbal:Purpose |to put block C on the table, block B on C, and block A on B| ^herbal:WorkingMemory <Table> ^herbal:WorkingMemory <AOnTopOf> ^herbal:WorkingMemory <BOnTopOf> ^herbal:WorkingMemory <COnTopOf> ^herbal:WorkingMemory <A> ^herbal:WorkingMemory <B> ^herbal:WorkingMemory <C> ) 7/18 (<OnTop> ^blocks:Bottom <CurrentBottomBlock> -) (<OnTop> ^blocks:Bottom <Block2>) (<CurrentBottomBlock> ^blocks:Clear |false| -) (<CurrentBottomBlock> ^blocks:Clear |true|) } #----------------------------------------------------------------------------- # Production that proposes operator: MoveBlockToTable # Operator Definition: # Moves a clear block onto a clear table, if there is a tie, resolve by preferring to stack blocks correctly # Proposal Conditions: # If there is a clear block ontop of another block, and the table is clear #----------------------------------------------------------------------------- sp {propose*MoveBlockToTable (state <s> ^herbal:Name |BlocksWorldState| ^herbal:WorkingMemory <Block> ^herbal:WorkingMemory <OnTop> ^herbal:WorkingMemory <Table>) (<Block> ^blocks:Clear |true| ^blocks:Type |Block|) 10/18 #----------------------------------------------------------------------------- # Elaboration: GoalStateReached # Elaboration Definition: # Terminates the model if block A is on block B, block B is on block C, and block C is on the table. #----------------------------------------------------------------------------- sp {elaborate*GoalStateReached (state <s> ^herbal:Name |BlocksWorldState| ^herbal:WorkingMemory <AOnB> ^herbal:WorkingMemory <BOnC> ^herbal:WorkingMemory <BlockA> ^herbal:WorkingMemory <BlockB> ^herbal:WorkingMemory <BlockC> ^herbal:WorkingMemory <COnTable> ^herbal:WorkingMemory <Table>) (<Table> ^blocks:Clear |true| ^blocks:Type |Table|) (<BlockA> ^blocks:Clear |true| ^blocks:Type |Block| ^herbal:Name |A|) 3/18 (<Table> ^blocks:Clear |true| ^blocks:Height 22 ^blocks:Type |Table| ^herbal:Definition |A table| ^herbal:Name |Table| ) (<AOnTopOf> ^herbal:Definition |A relationship that specifies what block A is on top of| ^herbal:Name |AOnTopOf| ^blocks:Bottom <Table> ^blocks:Top <A> ) (<A> ^blocks:Clear |true| ^blocks:Type |Block| ^herbal:Definition |A block with the letter A printed on it| ^herbal:Name |A| ) (<BOnTopOf> ^herbal:Definition |A relationship that specifies what block B is on top of| ^herbal:Name |BOnTopOf| ^blocks:Bottom <Table> ^blocks:Top <B> ) 8/18 (<Table> ^blocks:Clear |true| ^blocks:Type |Table|) (<OnTop> ^blocks:Bottom <> <Table> ^blocks:Top <Block> ^blocks:Bottom <CurrentBottomBlock>) --> (<s> ^operator <o> + ) (<o> ^herbal:Name |MoveBlockToTable| ^herbal:Definition |Moves a clear block onto a clear table, if there is a tie, resolve by preferring to stack blocks correctly| ^herbal:Purpose |to move a block onto the table| ^herbal:How-it-works || ^Block <Block> ^CurrentBottomBlock <CurrentBottomBlock> ^OnTop <OnTop> ^Table <Table>) } #----------------------------------------------------------------------------- 4/18 (<B> ^blocks:Clear |true| ^blocks:Type |Block| ^herbal:Definition |A block with the letter B printed on it| ^herbal:Name |B| ) (<COnTopOf> ^herbal:Definition |A relationship that specifies what block C is on top of| ^herbal:Name |COnTopOf| ^blocks:Bottom <Table> ^blocks:Top <C> ) (<C> ^blocks:Clear |true| ^blocks:Type |Block| ^herbal:Definition |A block with the letter C printed on it| ^herbal:Name |C| ) } #----------------------------------------------------------------------------- # Production that proposes operator: MoveBlockToBlock # Operator Definition: # Moves a clear block on top of another clear block, if there is a tie, resolve by stacking blocks correctly 11/18 (<BlockB> ^blocks:Clear |false| ^blocks:Type |Block| ^herbal:Name |B|) (<BlockC> ^blocks:Clear |false| ^blocks:Type |Block| ^herbal:Name |C|) (<COnTable> ^blocks:Top <BlockC> ^blocks:Bottom <Table>) (<BOnC> ^blocks:Top <BlockB> ^blocks:Bottom <BlockC>) (<AOnB> ^blocks:Top <BlockA> ^blocks:Bottom <BlockB>) --> (write |Goal Reached!|) (halt) } #----------------------------------------------------------------------------- # Production that elaborates this state with initial working memory. # State Definition: 12/18 # If there is a tie between operators, this state will figure out what operator is best. It prefers that we move B on C first, and then A on B. #----------------------------------------------------------------------------- sp {elaborate*initial-state*BlocksWorldState*ResolveTie (state <ResolveTie> ^type state ^attribute operator ^choices multiple ^impasse tie ^superstate <parent>) (<parent> ^herbal:Name |BlocksWorldState|) --> (<ResolveTie> ^herbal:Name |ResolveTie| ^herbal:Purpose |to determine what block is best to move| ^herbal:Definition |If there is a tie between operators, this state will figure out what operator is best. It prefers that we move B on C first, and then A on B.| ) } #----------------------------------------------------------------------------- # Production that proposes operator: PreferBOnC # Operator Definition: 13/18 # If there is an operator proposed to move B on top of C give that operator a best preference. Since this operator is best, it will be applied before moving A on B. # Proposal Conditions: # If there is an operator proposing to move B on top of C #----------------------------------------------------------------------------- sp {propose*PreferBOnC (state <s> ^herbal:Name |ResolveTie| ^item <myOp> ^superstate <ss>) (<myOp> ^herbal:Name |MoveBlockToBlock| ^|Block1| <B1> ^|Block2| <B2>) (<B1> ^herbal:Name |B|) (<B2> ^herbal:Name |C|) --> (<s> ^operator <o> + >) 14/ (<o> ^herbal:Name |PreferBOnC| ^herbal:Definition |If there is an operator proposed to move B on top of C give that operator a best preference. Since this operator is best, it will be applied before moving A on B.| ^herbal:Purpose |to create a preference that favors putting block B on top of Block C| ^herbal:How-it-works || ^myOp <myOp> ^ss <ss>) } #----------------------------------------------------------------------------- # Production that applies operator: PreferBOnC # Operator Definition: # If there is an operator proposed to move B on top of C give that operator a best preference. Since this operator is best, it will be applied before moving A on B. # Operator actions: # Mark operator "myOp" as best #----------------------------------------------------------------------------- 16/18 # If there is an operator proposing to move A on top of B #----------------------------------------------------------------------------- sp {propose*PreferAOnB (state <s> ^herbal:Name |ResolveTie| ^item <myOp> ^superstate <ss>) (<myOp> ^herbal:Name |MoveBlockToBlock| ^|Block1| <B1> ^|Block2| <B2>) (<B1> ^herbal:Name |A|) (<B2> ^herbal:Name |B|) --> (<s> ^operator <o> + =) (<o> ^herbal:Name |PreferAOnB| ^herbal:Definition |If there is an operator proposed to move A on 15/ sp {apply*PreferBOnC (state <s> ^herbal:Name |ResolveTie| ^operator <o>) (<o> ^herbal:Name |PreferBOnC| ^myOp <myOp> ^ss <ss>) --> (<ss> ^operator <myOp> >) } #----------------------------------------------------------------------------- # Production that proposes operator: PreferAOnB # Operator Definition: # If there is an operator proposed to move A on top of B, give that operator a best preference. Since this operator is indifferent, it will be selected only if there is not an option to move B on C. # Proposal Conditions: 17/18 top of B, give that operator a best preference. Since this operator is indifferent, it will be selected only if there is not an option to move B on C.| ^herbal:Purpose |to create a preference that favors putting block A on top of Block B| ^herbal:How-it-works || ^myOp <myOp> ^ss <ss>) } #----------------------------------------------------------------------------- # Production that applies operator: PreferAOnB # Operator Definition: # If there is an operator proposed to move A on top of B, give that operator a best preference. Since this operator is indifferent, it will be selected only if there is not an option to move B on C. # Operator actions: # Mark operator "myOp" as best 18/18 #----------------------------------------------------------------------------- sp {apply*PreferAOnB (state <s> ^herbal:Name |ResolveTie| ^operator <o>) (<o> ^herbal:Name |PreferAOnB| ^myOp <myOp> ^ss <ss>) --> (<ss> ^operator <myOp> >) }

  17. Herbal - Viewer

  18. Herbal- Viewer

  19. Herbal- IDE Change/Compile Loop w/ Impasses

  20. Herbal- IDE Change/Compile Loop

  21. Herbal- IDE Viewer Again, with Impasse

  22. Herbal- IDE Viewer Again, with Impasse

  23. Herbal Viewer - More Complex Model

  24. Support in Herbal for Explanation

  25. What - relation What - structure What - identity Definition - structure + IDE When - order - how often Contrast classes Support in Herbal Viewer for Explanation How does it work? Manual How do I use it? Planned How do I use it ? -model

  26. Herbal: VISTA Display Designed for Sequence Representation, CaDaDis, integrationacs.ist.psu.edu/CaDaDis, act-r, soar, jess, + jack, cast(Daughtry & Ritter, 2005; Morgan et al., 2005b;) In progress

  27. Herbal Evaluation • Ease programming • 3xProductivity (Yost, 1993, 3 min./production) • 1x Productivity (IST 402 class, no complaints of Soar) • 2x (Kukreja, first two programs @ 10 hours/prog.) • 3x productivity (Morgan et al., 2005) • X? x for 16 tutees at BRIMS + 15 walk-ups, in 3 hours • Promote reuse (document and import models) • Explanations supported to increase learning and use

  28. Experiences Using Herbal • IST 402 students noted Herbal’s ability to reuse conditions and actions across operators. They also commented on the usefulness the automatically generated comments, and the lack of impasses • Impasses added in v. 0.9. • Students noted usability issues with Protégé. • Protégé (version 3.0) addressed many of these issues • In 2005 4/9 final projects used Herbal (all groups used Herbal in their homework). • 2/4 Herbal teams had a model that ran, and 2/4 of just-Soar groups had a model that ran. • The most complex working model was an Herbal model with 16 pages of rules with 25 operators. • All students included a code headers to hook up the models to dTank • 30 professionals took Herbal tutorial at BRIMS ‘05, 8 enquiries

  29. Use and Reuse • Herbal supports reuse within the same model and across models • Operators and elaborations can share conditions and actions; states can share operators, elaborations, and impasses • Marginal cost:(N=1, Morgan et al. 2005)

  30. Endorsements • Chris Glur on Soar-list: “After the first few hours spent looking into SOAR [sic], I wondered why one was forced to grovel down at low level. And I've been waiting decade(s) for what Herbal seems to describe.” • Brian Haugh, IDA, tutee at BRIMS’05“…it deserves more funding.” • BRIMS tutorial (+ 8 enquires)

  31. Herbal Applications • Study of errors and error correction (OUP chapter) • Teaching Soar to UG and grad (couple schools) • USMC anti-terrorism/force protection (ATFP) cognitive support system • Will include Herbal agent-based intelligent user interface (Haynes et al., 2005) [funded, USMC-I&L] • SimuLAV(light armored vehicle) • Will include Herbal-based multi-agent sim., monitor vehicle health and operationalreadiness, and respond [funded, USMC-PM-LAV] • Opportunity to put explanation into Personal maintenance device (PMD) for LAVs, and use with interactive manuals, deployed (perhaps) in Aug05 POC. Haynes and Col. L. Larson/DPM Bob Lusadi

  32. 2. G2A (GOMS to ACT-R) • Implementation • Similar in some ways to IPME to ACT-R (Allander/Lebiere), and HLST • Evaluation • 2 weeks of MS thesis with ACT-R vs. 2 hours with G2A (40 x) • 5-10-30x, reanalysis of ACT-R model, another phone (St. Amant, Freed, Ritter, 2005) • 7.5x code expansion

  33. 3. Models Interact with Interface Directly( Ritter, Baxter, Jones, & Young, 2000) • Sim-eyes and -hands interact directly with interfaces (w/St. Amant)(Ritter et al., accepted; St. Amant et al., 2004, St. Amant et al., in pressa,b) • Avoids instrumenting interfaces • Needs support in Herbal compiler  x speedup on non-API interfaces ~20x speedup on phones? Save 30% user time

  34. 4. dTank Microworldacs.ist.psu.edu/dTank • Java-based game for testing explanation of dynamic, adversarial models - ModSAF-Sandbox • Multiple players and teams onmultiple machines • Improved 2004: complexity, vision theory, speed, interface, architecture use: 4 (cast, soar, jess, java)+3(act-r, jack, CoJACK**) • Listed on Jess web page of tools • Used by an Army MURI (Sun et al., 2004), at Lock Haven U. (www .lhup.edu/~mcohen/ dTank/dTankJess.htm,  Cohen, 2005), Federal U. of Uberlandia (Brazil) • 100x faster, 1,000/10 min to hook agent to

  35. 4. dTank Evaluation: Users to Use, Test, Expand Herbal & dTank • IST 402: Models of human behaviour (12+3, and 38+3 students) • Class projects • MacSoar 8.5 compilation (see Soar web site) • Architectural comparisons (Sun et al., 2004) • Example reuse curves (Morgan et al., 2005a) • Errors and Soar (Ritter et al., in prep.) • Students used dTank without problems

  36. 4. dTank Applications • For testing Herbal, of course! • Use in • DMSO project (being considered) • USAFOSR project (being considered) • UK MoD project on modeling Human variability and Computer Generated Forces (planned by PM) • ACT-R and COJACK distributions (likely) • Architectural comparisons (Sun et al., 2004),(Ritter et al., in prep.),(Morgan et al., 2005a)Models in Soar, Herbal-Soar, Java, Jess, CASTModels stubs in ACT-R, Jack • Use in rule-based programming at Lock Haven, Uberlandia, Portsmouth, Stanford

  37. Summary: Why Will This Work? • Principled design based on a theory of knowledge (PSCM, roughly and extended) • Based on theory of explanations supported by empirical results • Purpose built to capture design rationale • Software engineering principles • Modularity • Software reuse • Design patterns • Designed for usability by CS/HCI/Psy team • Extendable by users through Common tools • User base used for feedback (12+2+36+30, + more) • Speed up of 3x vs. v. good Soar programmers, 30x for ACT-R using G2A • Compiler expansion of 8x • Explanations as a payoff for using the tool • Use of SegMan will support interaction,  speedup • Sandbox to play in, 100x speedup

  38. Summary: Future Tasks • Improved usability based on classroom and BRIMS users • Drives development, gives feedback, trains programmers • Studies with Soar Tech on explanation • Populate library of models • Further visualizations/explanations • Applications to USMC projects and field studies of explanations • Package SegMan for reuse and refactor • dTank expansion to support wider user base, more complex tasks

  39. References(acs.ist.psu.edu/papers/) • Cohen, M. A., Ritter, F. E., & Haynes, S. R. (2005). Herbal: A high-level language and development environment for developing cognitive models in Soar. . In BRIMSr’05. 177-182. 05-BRIMS-044. Orlando, FL: U. of Central Florida. • Councill, I. G., Haynes, S. R., & Ritter, F. E. (2003). Explaining Soar: Analysis of existing tools and user information requirements. In Proceedings of the Fifth International Conference on Cognitive Modeling. 63-68. Bamberg, Germany: Universitats-Verlag Bamberg. • Daughtry, J. M., & Ritter, F. E. (2005). Supporting the active AI and cognitive modeling developer: CaDaDis version 2.1. AI and the Simulation of Behaviour Quarterly, 119, 4. • Haynes, S. R. (2001). Explanations in information systems: A design rationale approach. Unpublished Ph.D. dissertation, Departments of Information Systems and Social Psychology, London School of Economics. • Haynes, S. R., Councill, I. G., & Ritter, F. E. (2004). Responsibility-driven explanation engineering for cognitive models. In AAAI Workshop on Intelligent Agent Architectures: Combining the Strengths of Software Engineering and Cognitive Systems. • Haynes, S. R., Ritter, F. E., Councill, I. G., & Cohen, M. A. (submitted January 2005). Explaining intelligent agents. • Morgan, G. P., Ritter, F. E., Stevenson, W. E., Schenck, I. N., & Cohen, M. A. (2005). dTank: An environment for architectural comparisons of competitive agents. In BRIMS’05. 133-140. 05-BRIMS-043. Orlando, FL: U. of Central Florida. • Morgan, G. P., Haynes, S., Ritter, F. E., & Cohen, M. (2005). Increasing efficiency of the development of user models. In Proceedings of the 2005 Systems and Information Engineering Design Symposium. IEEE . • Ritter, F. E., Baxter, G. D., Jones, G., & Young, R. M. (2000). Supporting cognitive models as users. ACM Transactions on Computer-Human Interaction, 7(2), 141-173. • Ritter, F. E., Cohen, M. A., Morgan, G. P., & Stevenson, W. (2005). Herbal: A high-level language and development environment for developing cognitive models in Soar. 05-BRIMS-041. A tutorial presented and in The BRIMS ‘05. xxix-xxxi. 05-BRIMS-041. • Ritter, F. E., Van Rooy, D., St. Amant, R., & Simpson, K. (accepted pending revisions, January 2005). Using a simulated user to explore human-robot interfaces. IEEE Transactions on System, Man and Cybernetics, Part A: Systems and Humans. • St. Amant, R., Horton, T. E., & Ritter, F. E. (2004). Model-based evaluation of cell phone menu interaction. In Proceedings of the CHI‘04 Conference on Human Factors in Computer Systems. 343-350. New York, NY: ACM. • St. Amant, R., Freed, A. R., & Ritter, F. E. (2005). Specifying ACT-R models of user interaction with a GOMS language. Cognitive Systems Research, 6(1), 71-88. • St. Amant, R., Riedel, M. O., Ritter, F. E., & Reifers, A. (in press). Image processing in cognitive models with SegMan. In Proceedings of HCI International '05. (Invited.) • St. Amant, R., Horton, T. E., & Ritter, F. E. (accepted 2005). Model-based evaluation of expert cell phone menu interaction. ToCHI. • Shah, K., Rajyaguru, S., St. Amant, R., & Ritter, F. E. (2003). Connecting a cognitive model to dynamic gaming environments: Architectural and image processing issues. In Proceedings of the Fifth International Conference on Cognitive Modeling. 189-194. Bamberg, Germany: Universitats-Verlag Bamberg. • Sun, S., Councill, I., Fan, X., Ritter, F. E., & Yen, J. (2004). Comparing teamwork modeling in an empirical approach. In Proceedings of the Sixth International Conference on Cognitive Modeling. 388-389. Mahwah, NJ: Lawrence Erlbaum. • Yost, G. R. (1993). Acquiring knowledge in Soar. IEEE Expert, 8(3), 26-34.

More Related