Design Evaluation

1. ECLIPS Annual MeetingNovember 6, 2011Deepening Understanding and Application of Systems Concepts and Development of ECLIPS

2. Traditional Evaluation Framework Design Evaluation Collect Data Shape Practice • Make Meaning from Data

3. System-Oriented Evaluation Framework Design Evaluation Shape Practice CollectData Make Meaning from Data

4. Example of Phases of Evaluation Design Evaluation Shape Practice Collect Data MakeMeaning from Data

5. ZIPPER Z = Zooming in and out of evaluand and its environment. I = Interconnecting the parts P= Plunging into paradigms, structures, processes, conditions P= Perceiving patterns E= Envisioning energy R = Recognizing results

6. What is a System? A system is an interconnected set of elements that is coherently organized in a way that achieves something.

7. Dynamics of a Social System N T E O X T C Unorganized dynamics (random unpatterned seemingly chaotic) far from agreement Self-organizing dynamics (emerging patterns     coherent but not predictable) Agreement • Organized  dynamics • (predictable •    orderly • controlled) close to agreement far from certainty close to certainty Certainty

8. Systems Levers(Illustrating Visibility and Depth) Events/ Behaviors/ Results Patterns Structures/Processes Paradigms Conditions

10. Levers for Changing Complex Systems: Boundaries (B), Relationships (R), and Perspectives (P) • Boundaries: Demarcations that create a region/entity • Relationships (interconnections): Connections/exchanges among bounded system parts • Perspectives: Mental models, world views, purposes From Parsons, B. and Jessup, P. (2009). “Questions that Matter: A Tool for Working in Complex Situations”. Ft. Collins, CO: InSites.

11. Levers for Changing Complex Systems: BoundariesDemarcations that create a region/entity • Examples: physical entities, organizational identity, social systems, rules of conduct • May be permeable; allow exchange with environment • May be impermeable in that they distinguish the system or other entity from its environment From Parsons, B. and Jessup, P. (2009). “Questions that Matter: A Tool for Working in Complex Situations”. Ft. Collins, CO: InSites.

12. Levers for Changing Complex Systems: Relationships (R) (interconnections):Connections/exchanges among bounded system parts • Are key aspects of forming patterns • May be as/more important than the system entities • Form system structures (hierarchy, networks) • Cause and effect relationships: a type of relationship From Parsons, B. and Jessup, P. (2009). “Questions that Matter: A Tool for Working in Complex Situations”. Ft. Collins, CO: InSites.

13. Levers for Changing Complex Systems: Perspectives (P)Mental models, world views, purposes • Different stakeholders may have different perspectives • Different perspectives may exist within stakeholder groups From Parsons, B. and Jessup, P. (2009). “Questions that Matter: A Tool for Working in Complex Situations”. Ft. Collins, CO: InSites.

14. Levers for Changing Complex Systems: Boundaries (B), Relationships (R), and Perspectives (P) Example Situation: Balanced BRP Boundaries (B) Example: Classroom where students and teachers have clear expectations yet flexibility to make own decisions on some matters (B), respectful relationships among all members (R), and shared views about what is important to learn (P). Relationships (R) Perspectives (P) Adapted from Eoyang, G. (2002). CDE Model for Self-Organizing Systems: Case Examples. Circle Pines, MN: Human Systems Dynamics Institute.

15. Levers for Changing Complex Systems: Boundaries (B), Relationships (R), and Perspectives (P) Example Situation: Constrained Boundaries B Example: Community college holds to past successes in how they work together internally (R) in defining their courses and instructional methods (P). They do not keep up with innovations in technology and changes in the local industries they could serve (B). P R Adapted from Eoyang, G. (2002). CDE Model for Self-Organizing Systems: Case Examples. Circle Pines, MN: Human Systems Dynamics Institute.

16. Levers for Changing Complex Systems: Boundaries (B), Relationships (R), and Perspectives (P) Example Situation: Constrained Relationships B Example: School district with explicit silos and minimal connections among departments (B). Departments work independently and compete for resources annually (R). Few shared standards of customer service and strong conflict among managers, supervisors, and staff members (P). P R Adapted from Eoyang, G. (2002). CDE Model for Self-Organizing Systems: Case Examples. Circle Pines, MN: Human Systems Dynamics Institute.

17. Levers for Changing Complex Systems: Boundaries (B), Relationships (R), and Perspectives (P) Example Situation: Constrained Perspectives B Example: Community college where conflicts about priorities are not acknowledged (P). Faculty, administrators, and students understand their roles (B) and work harmoniously (R), but certain individuals and groups feel disenfranchised in the college as a whole. P R Adapted from Eoyang, G. (2002). CDE Model for Self-Organizing Systems: Case Examples. Circle Pines, MN: Human Systems Dynamics Institute.

18. Levers for Changing Complex Systems: Boundaries (B), Relationships (R), and Perspectives (P) Example Situation: Unconstrained Relationships B Example: In partnership among school, community, colleges, and industry, partners have clarity about their role in the partnership (B). They have a general agreement about why they are working together (P). Group meetings are filled with conflict as each partner says exactly what they think without concern about others' perspectives, needs, or feelings (R). P Adapted from Eoyang, G. (2002). CDE Model for Self-Organizing Systems: Case Examples. Circle Pines, MN: Human Systems Dynamics Institute. R