Model-Driven Techniques for User Interface Generation

1. Model-Driven Techniques for User Interface Generation Jacob Adams Topic Paper Department of Computer Science Southern Illinois University Edwardsville

2. Model-Driven Development Wikipedia- “software development methodology which focuses on creating models, or abstractions, more close to some particular domain concepts rather than computing (or algorithmic) concepts”

3. Model-Driven Development • Benefits • Model is typically more declarative • Model is typically described in terms related to the problem domain • Required less technical knowledge to create

4. Reasons for Model-Driven User Interfaces Same benefits as regular model-driven development Creating user interfaces (UI’s), is still a largely manual process Creating UI’s for several different platforms can lead to redundant work

5. My Interest in Model-Driven UI Generation Basler Electric 2005-2008 Many products had hundreds of screens Needed similar user interfaces for both desktop and embedded applications Developed multiplatform, database-driven screen generator

6. Research • Evaluate techniques for model-driven UI generation from a software engineering standpoint. • Important Criteria • Extensibility • Maintainability • Efficiency • Simplicity

7. Previous Work

8. The User Interface Generator • Alan Heirich -1987 • Use declarative commands to describe system: • Token command – describe type of data to be entered • Descriptive phrases • Actions that can be executed • Acceptance tests-performs validation logic

9. The User Interface Generator Can also define dependencies between fields UI elements are limited to a small set of widgets (textbox, selection, button, menu, dialog box) Can also create command line version of the application

10. Wolff, Forbig, Dittmar, Reichart Allows an application to be developed in an evolutionary fashion Provides quick prototypes

11. Wolff, Forbig, Dittmar, Reichart Originally requires a task model

12. Wolff, Forbig, Dittmar, Reichart Task model is turned into a dialog graph

13. Wolff, Forbig, Dittmar, Reichart Dialog graph is used to create abstract UI (AUI) AUI is defined in XUL AUI is originally just placeholder widgets Placeholders are replaced by real widgets

14. Wolff, Forbig, Dittmar, Reichart Making updates Changes must be propagated back through task model, dialog graph and AUI

15. SUPPLE • Gajos and Weld • Requires3 Models • Interface specification – constraints (e.g. type, widget used) places on user interface elements • Device model – information and constraints for the particular device that the UI will be generated for • User model – information, such as usage patterns about the intended user of the application

16. SUPPLE After the data is collected, a pruning algorithm is performed to find the rendering of the UI the reduces the expected effort required to use the user interface UI generation and rendering is performed at runtime and is performed dynamically.

17. SUPPLE User Interface Generated for Mouse/Pointer-Based Input

18. SUPPLE User Interface Generated for Touch-Based Input

19. SUPPLE Different User Interfaces Generated Based on Different Usage Patterns

20. ARNAULD Gajos, Weld, and Wobbrock Used to generate effort estimations used by SUPPLE Asks user to choose between renderings

21. SUPPLE++ • Gajos, Weld, and Wobbrock • Another method of determining effort estimate for SUPPLE • Measures users motor abilities • In experiment with 11 participant with motor impairments • Users were 26% faster • Made 73% fewer errors

22. SUPPLE++ http://www.youtube.com/watch?v=B63whNtp4qc&feature=player_embedded

23. Feuerstack, Blumendorf, Schwartze, Albayrak Requires context model, task tree, domain model, abstract user interface, and dialog models Provides a tool to generate layout statement from these models. Layout statements can define containment, order, orientation and size

24. Feuerstack, Blumendorf, Schwartze, Albayrak

25. Feuerstack, Blumendorf, Schwartze, Albayrak Containment Order Orientation Size Statement are given a scope (screen, set of elements, entire application) Models and statements are evaluated at runtime

26. Kalvaldjian • Fully automated process of generating UI from model • Requires discourse model • Screens and state machine are created from discourse model • Discourse model contains set of actions • Common types – request, informing, question, answer, etc.

27. Kalvaldjian ATLAS transformation language converts actions and other information into an AUI Model2Code transformation language converts AUI into concrete UI (CUI) Widgets are selected based on type of information to be input or displayed

28. Stocq and Vanderdonckt • Requires domain model (relational database) • Wizard style tool • Select list or datasheet style UI • Select data source • Select data source for lookup widgets • Choose which widgets update data • Choose relative position of widgets • Make optional modifications to sizes

29. Stocq and Vanderdonckt Widgets are generated based on their type Widgets are sized based on their longest possible value Tool can create both desktop and mobile UI’s

30. Mastermind • Stirewalt and Rugaber • Requires • Presentation model – information presented to user • Application model – information and functions available to UI • Dialog model – interactions between user and application, relationships to other models • Automatically creates UI from these models

31. Mastermind • An agent is created to handle each model • Creates event to act as connection points • Contain information about actions and callbacks • Specified in generic, model independent way • UI is generated if models and connections are valid. • Created two proof of concept applications • Simple print and save widget • Air traffic controller application

32. Mahfoudhi, Abed, and Abid • Also generated UI completely from the models • Requires • Static structural task model (SSTM) – hierarchy of tasks and what they are supposed to do. • Dynamic structural task model (DSTM) – sequencing and synchronization of tasks in SSTM

33. Mahfoudhi, Abed, and Abid • Operational model converts task models into component objects • Contain state machine built from actions and hierarchy defined in SSTM and transitions defined in DSTM • Component objects are aggregated into larger objects. • Translated into information need for concrete UI: user models, local interface models, abstract interface models, and interface implementation models

34. Analysis of Previous Work

35. Models Required to Generate UI Some techniques required only one model, others required several The detail involved in the models also varied significantly There are tradeoffs between having a simpler or more complex model

36. Model Detail Tradeoffs • Simple models • Easier to develop models • Detailed Models • Easier to develop the rest of the application • Information is more declarative • Possible to require more work that traditional development techniques

37. Automatic Generation of UI • No outside steps required • Requires detailed model or application may become rigid and/or have poor quality • May be difficult or impossible to model UI entirely • Manual changes • Allows more flexibility and customization • Can make updates difficult

38. Making Updates and Changes Maintainability is a primary aspect of software engineering System needs to be flexible to change over time Automatic UI generation allows for quicker changes Making updates in multiple places defeats purpose of the using a model

39. Dependency on Model • Applications with automatic generation are often tightly coupled to their models • Can cause problems if model cannot handle future changed • Can also make getting rid of model difficult • Tradeoff: tightly coupled vs. harder to update

40. Use of Abstract User Interface Can help reduce dependencies on model Helps decouple generation process from concrete UI specifics Allows easier changes of UI widgets Provides easier creation of UI for different environments, including different form factors

41. Support for Different Form Factors Some approaches (Stocq and Vanderdonckt, User Interface Generator) produce UI’s for a few platforms Others, such as SUPPLE, support creation on an unlimited number of platforms Several even allow different input mechanisms (keyboard, mouse, touch) Increasingly important with rise of mobile computing

42. Support for Different Form Factors User may want different UI’s for different platforms Allowing customizations to UI can help solve this problem Using sample usage patterns can also address this problem

43. Usage Patterns Contains information such as frequency and order of actions performed by a user. Can make using application much more efficient. Requires more work. Can be generated dynamically and at runtime.

44. Runtime UI Creation Allows UI to change over time, which recompilation Can change with user as usage requirements change Can not easily be performed if manual steps are required Can be confusing to have dramatic changes to the UI Gajos, Weld, and Wobbrock showed that using a hybrid approach can be a good comprimise

45. Possible Future Enhancements Less manual involvement of the developer Provide richer interfaces Make more improvements from software engineering perspective

46. Conclusion Considerable research has already been done. However, UI generation is still a relatively new and expanding field There is still significant improvements that can be made.