190 likes | 328 Views
Evaluating the Aboriginal Programs and Initiatives at the Department of Canadian Heritage Canadian Evaluation Society Conference 2003 Vancouver, British Columbia. Purpose of Presentation. Provide context to the evaluations within the Aboriginal Affairs Branch at PCH
E N D
Evaluating the Aboriginal Programs and Initiatives at the Department of Canadian Heritage Canadian Evaluation Society Conference 2003 Vancouver, British Columbia
Purpose of Presentation • Provide context to the evaluations within the Aboriginal Affairs Branch at PCH • Present two methodological approaches • Aboriginal Languages Initiative – project site visits • Aboriginal Women’s Program – Family Violence Initiative – learning circle • Discuss evaluation challenges and lessons learned • Discuss next steps
Aboriginal Affairs Branch - PCH • 13 programs and initiatives • Focussed mainly off-reserve • Pan-Aboriginal in nature • Developed over some thirty years • Multifaceted approach to complex Aboriginal issues • Facilitate participation in Canadian society • Strengthen Aboriginal cultural distinctiveness
Types of Program Support • Sustaining – established Aboriginal organizations • Representative, women’s, friendship centres, broadcasting • National, provincial/territorial, regional • Project • Partnering • Accords with territorial governments – mutual goals • Aboriginal organizations
Program Delivery – 3 approaches • Canadian Heritage administration • Aboriginal Affairs Branch • Regional offices • Aboriginal organizations • Six western cities – youth advisory councils
Comprehensive Review • Retrospective study • Environmental scan • Consultations – Aboriginal organizations/groups • Evaluation of 13 programs and initiatives
Evaluation Challenges • Complex approach to program delivery • Ensuring a collaborative approach (involvement of delivery partners) • Ensuring cultural sensitivity • Availability of baseline data and performance indicators
Evaluation of the Aboriginal Languages Initiative • Evaluation Methods • Document and File Review • Secondary Research and Statistical Review • Key Informant Interviews • Case studies of delivery organizations (AFN, ITK and MNC) • Project Site Visits – 10 • Working group and steering committee comprised of representatives from PCH (Aboriginal Affairs Branch and Corporate Review Branch) AFN, ITK, MNC)
ALI Project Site Visits • 2 Inuktitut sites, 2 Michif sites, 6 First Nations languages sites • Sites were selected to include all regions and geographic locations (near urban, rural, and isolated) and from each type of “project cluster”
Opportunities and Benefits • Opportunities • Conduct face-to fact interviews with those involved directly and indirectly with the project • Conduct focus groups/dialogue circles with project beneficiaries and other community people • Benefits • Observe projects and activities directly • Place project in context of the community within which it operates • Obtain information on program impact directly from beneficiaries - how they define success - • Obtain information in a culturally sensitive manner • Communities see themselves as part of evaluation process
Evaluation of the AWP Family Violence Initiative • Evaluation Methods Document& File Review LearningCircle Key Informant Interviews
What is a Learning Circle? • Gathering of people from different organizations • Opportunity for open sharing and exchange • Inclusion of range of projects and perspectives
Learning Circle Process Sharing knowledge Stories and journeys What have we learned? Common threads Issues and challenges Future direction
Learning Circle Opportunities and Benefits Benefits: Connection at the community level Meaningful results Lessons learned Inspiration & momentum Future program directions Opportunities to: Share experiences and insights Learn Explore ideas Reduce isolation
Disadvantage of methodological approaches • Costly and time consuming • Results from site visits or learning circles are not statistically representative of the results of the program as a whole – findings need to be balance with other lines of evidence • One time assessment rather then continuous measurement of results
Lessons Learned • Importance of including Aboriginal communities in evaluation work – legitimizes results • Qualitative data should be balanced with quantitative information • Evaluations can be designed that meet both government and Aboriginal community information needs • Importance of information sharing
Aboriginal Affairs Branch Next Steps… • Analysis of information • Development of position paper • Consultation • Development of a new consolidated policy
Evaluation reports available electronically Department of Canadian Heritage Website www.pch.gc.ca
Presenters • Audrey Greyeyes, Aboriginal Affairs Branch, PCH (819) 994-2121 audrey_greyeyes@pch.gc.ca • Dianne Lepa, Corporate Review Branch, PCH (819) 994-7485, dianne_lepa@pch.gc.ca • Wanda Jamieson, JamiesonHartGraves Consulting (613) 730-2595 wjamieson@jhgconsulting.com