Reporting and Accountability Understanding Success

1. Reporting and Accountability Understanding Success Anne Couillard Settlement and Integration Policy Branch IRCC October – November 2017

2. Purpose • IRCC perspective on Settlement Program success and desired outcomes • Provide an overview of IRCC data collection, and the pros, cons, and gaps • Provide some highlights from data analysis • Present how this information is used and reported • Discuss next steps and ways forward 1

3. Outline • Background • IRCC’s Settlement Program • Focus on Outcomes • What is Settlement Program success? • How do we know we succeeded? • What is IRCC collecting? • Examples of findings • Annual Project Performance Report (APPR) • Clients and services (iCARE) • How is IRCC using this information? • Pros, cons, and gaps in data collection • Considerations & next steps 2

4. BackgroundIRCC Settlement Program Legislative Authorities Canada’s Settlement Program fulfills obligations of the Immigration and Refugee Protection Act to: “Promote the successful integration of permanent residents into Canada, while recognizing that integration involves mutual obligations for new immigrants and Canadian society.” • The Settlement Program aims to assists newcomers to overcome barriers to participation in social, cultural, civic and economic life in Canada and help to foster an integrated society • Services are funded through contribution agreements and deliveredby third-party providers • Funding is allocated by province and territory and centrally managed national and pre-arrival funds. This does not include the transfer to Quebec • It delivers services to help newcomers acquire necessary language skills, ensuring their access to information, connecting them to community resources, building their knowledge of life in Canada, and enabling labour market participation • To reach these objectives, IRCC also ensures Sector capacity and professional development, standardization of best practices, and building knowledge of the Program’s clients Integration is a shared role with involvement and engagement of multiple players, whose contributions and levers help to achieve longer-term outcomes and IRCC Settlement Program objectives 3

5. Background Focus on Outcomes There are a number of factors creating momentum for looking at outcomes, and for review, revision and refinement of data collection and reporting • Ministerial Mandate Letter: ‘…a rigorous approach to data in order to accurately measure outcomes.’ • Government of Canada and Departmental emphasis on results and delivery, focus on outcomes • Settlement Program evaluation • Levels Plan - program modernization • Ministerial consultations with stakeholders

6. Background Program Design Inputs and Delivery • Settlement Program • Terms and conditions • Funding • Logic Model with expected outcomes Outputs Immediate Outcomes • Settlement Services • Based on priorities for services and programming and directives to manage contribution agreements • Needs Assessment • Information and Orientation • Language Assessment • Language Training • Community Connections • Indirect Services Intermediate Outcomes Ultimate Outcomes 5

7. It starts with expected outcomes! What is Settlement Program success? Settlement Program Expected Outcomes Outcomes for clients are a key focus but two-way street important Specific level of outcomes for specific clients groups 6

8. How do we know we succeeded?Examples of performance measurement indicators Measured by: Outcomes Performance Indicators 7

9. What is IRCC collecting?Measuring performance at Projects and Program Level • Outputs: • Number of clients and their characteristics • Number of services and their characteristics • Geographic distribution • Language progress • Delivery: • Locations • Partnerships & resources • Challenges & opportunities • Successes • Needs assessment approaches • Performance assessment approaches • Ultimate Outcomes: • Economic, social and • cultural integration through iCARE data linkages to IMDB • Immediate/intermediate outcomes: • Knowledge, behaviour, integration in four areas: • Information about Canada • Official Language knowledge • Labour market • Community connections • including non-clients for comparison • Inputs/Resources: • Budgets • Targets • Operational monitoring Pilots, experimentation 8

10. Examples of Findings: Client Needs identified in APPR • Employment • Connecting with professional networks in the Canadian workplace • A desire to find employment within field of expertise • Balance between working in a survival job vs need for language training to obtain a better employment, e.g. within field of expertise • Language • Increase demand for literacy and lower level CLBs due to increase in refugees admissions • Need to obtain CLB 4 for citizenship purposes • Health and wellness • Mental health support for children, youth and adults due to trauma • “Health and Wellness” awareness • Support Service • Need for child care to facilitate participation in services, particularly language training • Translation and interpretation (outside of IRCC services) • Transportation required to participate in IRCC and non-IRCC services 9

11. Examples of Findings: Delivery Challenges from APPR Because of higher numbers of clients, we are trying to do more group orientations in order to reduce the waiting list for some of our programs. 10

12. Examples of Findings: Clients and Services in iCARE In 2016-17, over 410,000 clients accessed at least one settlement service in Canada, a 2.7% increase in clients served over 2015-16. 11

13. Measurement Activities at IRCC Audit Inputs and Delivery Monitoring Outputs • Settlement Program Design • Terms and Conditions • Objectives • Performance Measurement Strategy • Services Periodic Evaluation Ongoing Performance Measurement Immediate Outcomes Settlement Services Intermediate Outcomes Employment-related Services (ES) Employment-related Services (ES) Employment-related Services (ES) Employment-related Services (ES) Employment-related Services (ES) Employment-related Services (ES) Planning Ultimate Outcomes Research 12

14. How is IRCC using this information? • Planning • Identification of newcomer needs (from Needs Assessment and Referrals Services) • Design of services e.g.; Community Connections: Volunteer Coordination and Enhancements • Analysis of service patterns (e.g.: users profile, language progression, geographic distribution) • Project and Financial Monitoring • Achievement of project service targets by organizations • Service use at various groupings (local, provincial, national) • Financial monitoring • Performance Measurement • Performance reporting to Canadians, reports to Parliament • Reporting to SPOs, posting of data (Open Data Portal) • Matching with financial data to assess use of funds • Annual performance reports on Settlement Program • Support for briefings, Cabinet documents, Standing Committees; UN, OECD input • Evaluation • Assessment of outcomes, especially for clients but also indirect services • Research • Emerging needs • Factors for social, economic and cultural integration • Client vs non-clients analysis 13

15. Data Collection Pros, Cons, Gaps 14

16. Considerations • Are we collecting the right information? • Is it collected at the right time, and at the right frequency? • How/Can we alleviate the reporting burden? By combining data? Reducing overlaps? • How do we introduce new data elements (addressing gaps) without increasing the overall burden? • How do we make reporting more meaningful? • Are we providing you information to support decision-making? Project planning? 15

17. Next Steps • Share Settlement Program Evaluation Report and recommendations • Implement Evaluation recommendations including review of reporting requirements • Review and confirm performance measurement strategy (outcomes, indicators, data sources, reporting strategy) • Establish Sector Outcomes Working Group • Implement Newcomer Outcomes Survey • Engage with sector on the common settlement vision

18. Thank you! Comments & Questions? Anne CouillardDeputy Director, Performance Measurement Unit Settlement and Integration Policy Branchanne.couillard@cic.gc.ca