Informing Reform: Changing The Relationships Between Research, Policy and Practice in Context

1. Informing Reform: Changing The Relationships Between Research, Policy and Practice in Context Some Observations from the Work of the Consortium on Chicago School Research Anthony S. Bryk Founding Director

2. Reframing the relationship between . . . Research A distinct mission — informingreform—anchored in a set of core values about the role of applied inquiry in social problem solving Practice Policy

3. Value Perspectives • The relationship of social science to social problem solving in a democratic society is best viewed as an educational one. . . .In the broadest sense, we aim to educate the public about the improvement of public education.

4. Value Perspectives • We eschew the role of “technical advisor to the prince.” We embrace a more democratic vision of policy making as a competition of ideas, where our role is to inform the competition with the best possible evidence and ways of thinking about that evidence.

5. Value Perspectives • We embrace an ethical responsibility to seek out actively the diverse interests operative around school reform, and assure that a wide range of questions are continually addressed. The research agenda should not be beholden to any one group.

6. Value Perspectives • A steadfast commitment to technical quality undergirds the conceptualization, design, and actual conduct of the research. While ideas from various stakeholders infuse the process; in the end we engage in disciplined inquiry.

7. Value Perspectives • We are mindful that instrumental use of research findings is at best occasional. Distinctive contributions can also be made by reshaping the larger conceptualizations that focus subsequent policy debates—the importance of an empirically grounded, coherent theory of action.

8. Five Key Organizational Features • Extensive stakeholder consultation on research agenda, study design and interpretation of results • Distinctive model of funding and governance 3. Building analytic capacity for the long term 4. A commitment to intensive public informing 5. A distinctive researcher role within a network organizational structure

9. 1. A Leadership Structure Designed For Consultation Directors from multiple institutions • Responsible for day-to-day research Steering Committee brings diverse perspectives • Establishes agenda of supported research • Provides intellectual guidance • Insures multiple perspectives Constituent Advisory Board • Build a civic-elite constituency for the work • Advance use of information Ad hoc committees for particular studies

10. 2. A Distinctive Model of Funding and Governance • Formally, Directors have extensive control over the work, but… • Operate in an extraordinarily public and deliberately pluralistic environment • role of Steering Committee in strategic resource allocation • role of both Steering Committee and Constituent Advisory Board in shaping and reviewing work in progress • all data made public • General support from three major foundations (plus project specific support) • Accountability • on regular basis to Steering Committee and Constituent Advisory Board • when requesting grant renewal

11. 3. Building Analytic Capacity for the Long Term • Developing a base of researchers focused on reform in a place • Enhancing the capacity of the system to understand its own operations • Expanding the analytic resources for this work—a comprehensive longitudinal archive on students, schools and communities.

12. Multiple Methods and Measures • Surveys • Longitudinal case studies of schools • Short-term case studies • Analysis of administrative records • Classroom observations • Collection of student work • Network analysis • Longitudinal analysis of test scores

13. A Comprehensive Data Archive

14. A Distinctive Research Process • An interplay of theory, past research, and stakeholder concerns guide the questions we ask. • The crafting of a research report typically begins as descriptive. As analyses proceed, a small set of focal ideas emerge. • In directing attention to these ideas, we help to deepen public understanding, focus debate about policy concerns, and create opportunities to build common ground in a pluralist environment.

15. 4. Intensive Public Informing Media Output Audiences Stakeholder Input Researchers, other urban school systems Research Panels Conferences Reports Web site Chicago public-at-large Media Intermediary Groups (CALSC, Principals’ Assn..) Local school communities School reports Cable TV Business leaders, Advocacy & Community groups Briefings Reports and summaries Constituent Advisory Board Key Decision-makers Steering Committee Briefings Report summaries

16. 5. A Distinctive Researcher Role within a Network Organizational Structure

17. Sorting Through the Murky Terrain of Multiple Realities • Individuals view schools through particular lens, according to interests, position and background • Each brings insights that suggest questions and evidence to be gathered • Extraordinary diversity of questions are possible

18. Researchers’ Distinctive Role in Pluralistic Environment • Bring coherence and scientific discipline • Keep research questions centered on fundamental issues that materially affect learning

19. Transforming the Researcher’s Role to Work within a Local Context • Reframes social and organizational working relationships • Researchers have horizontal rather than vertical orientation • Involvement of local researchers expands analytic capacity focused on schools • Emphasis on mutual learning among researchers, policy makers and practitioners • Stimulates broader processes of community education • Advances conversation about school problems and how to proceed

20. Challenges For Researchers • Involvement of stakeholders complicates and lengthens the research process • Exacerbates tension between timeliness and research quality • Demands broader repertoire of skills among researchers • Pressure on senior researchers to communicate frequently with multiple audiences • Less flexibility for researchers in pursuing “their own research program”

21. Challenges Working with a School System Necessary conditions: • Mutual trust • Collaboration with the research and evaluation unit in the school system • Flexibility in adjusting to changes in leadership • Keeping top management informed: no surprises • Bring to light issues that need attention, avoiding ad hominem criticism • An inherent tension in “criticizing” the system as we seek to increase public understanding of what is required to expand opportunities