1 / 15

Leukemia & Lymphoma Society (LLS) Information Resource Center (IRC)

Leukemia & Lymphoma Society (LLS) Information Resource Center (IRC). Planning for a Service Program Evaluation - Case Study Public Administration 522 Presented by Ervina Castillo-Newton, John Toadlena and Scott Atole October 16, 2012. Background on Service Program Case Study.

joshua
Download Presentation

Leukemia & Lymphoma Society (LLS) Information Resource Center (IRC)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Leukemia & Lymphoma Society (LLS) Information Resource Center (IRC) Planning for a Service Program Evaluation - Case Study Public Administration 522 Presented by Ervina Castillo-Newton, John Toadlena and Scott Atole October 16, 2012

  2. Background on Service Program Case Study Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation Leukemia & Lymphoma Society (LLS) Information Resource Center (IRC) “World’s largest voluntary health organization dedicated to funding blood cancer research, education, and patient services” • Private non-profit service organization • Funded by individual and corporate contributions • 68 chapters: U.S and Canada • Masters level Information specialists responding to 80,000 inquiries on leukemia, lymphoma, myeloma and other blood cancer patients, etc. • Provide information, guidance & support (phone based)

  3. Background on Service Program Case Study Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation • Evaluation of Information Resource Center(IRC) • Email – initial contact • Program and academic interest & expertise match • Successful response to RFP • Initial discussions including questions identified in RFP response. Additional details expected in the planning process • RFP: Pilot a patient navigation model for IRC follow up services • Navigate health care systems • Overcome barriers • Evaluation of service enhancement-scheduled phone call follow up to select callers. Need identified by program staff • LLS-service oriented and committed to the evaluation process • Evaluate new programs and ongoing evaluations • Internal funding for this evaluation

  4. Background on Service Program Case Study Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation • Initial Planning: • 3 months (email to launch) • Report provided 6 months later • LLS moved to fund a second/larger evaluation on Patient navigation • EPIC model (steps before the pilot study)

  5. Assess the Context Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation Understanding the Political & Organizational Environment • Pre-project-consulted various sources • Internal/external people • Internet/LLS website • LLS Promotional items/collaterals • RFP • LLS Mission Statement • LLS Organizational structure (background materials/site visit) • Commitment to organizational cause. • Political environment associated with advocacy and organizational commitment • Funding-individual/corporate • LLS operations (private sector-accountability) • How will the evaluation findings be used? Future funding from LLS Board of trustees. Internally competitive • Location of IRD within LLS structure. department

  6. Assess the Context Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation Understanding the Political & Organizational Environment • LLS values program evaluation • Objective data to support the IRC program enhancement • IRC – immediate program context • 7 full time information specialists • 80,000 phone calls annually • Navigation model – innovative among other national programs • Navigation to identified subgroup (2 specialists) • National organization • 68 chapters – IRC link to local chapters • Need for cancer information (consumer friendly) • Potential for collaboration “community of cancer information service professional”

  7. Assess the Context Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation Define the Relationship Between Evaluator & Sponsor • Internal/External Evaluator and Funding • LLS External Evaluator/Internal funding • Findings were reported to the LLS Board of Trustees • Primary Point of Contact – LLS Vice President or Patient Services & Disease Programs (Includes IRC) • Advantages/Disadvantages-resistance to int/ext pressures, produce positive findings, objectivity, credibility, timelines(learning/building relationship), potential biases • Clarify roles and the relationship early • Evaluator and Sponsor roles • Funding arrangements • LLS –Participatory Relationship in planning and implementation • LLS – 2 Managers • IRC – 1 Supervisor (Ongoing Liaison) and 1 staff member • Ongoing Education – Responsibilities • LLS Staff – Provide feedback • Evaluator-Drafts, Technical assistance (research process), Assess stakeholder knowledge • Evaluator skill • Relevant experience, experience and interest • Qualitative vs. Quantitative

  8. Assess the context Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation Determine the Level of the Evaluation • Leukemia & Lymphoma Society (LLS) is local • Information Resource Center (IRC) is national. Seek to create the connection with local chapters. • LLS Target population • Newly Diagnosed • Those seeking information on clinical trials • Patients residing in Wyoming • Reasoning for Target Population selection (Where will this enhancement be most helpful?): • Limited LLS resources to offer the program to all callers • Navigation approach –reduction in health disparities to vulnerable populations • Wyoming – no local chapters • Outcome vs. Process Evaluation • Primary focus on outcomes • IRC program enhancement – include and document the process

  9. Gather Reconnaissance Scott Atole UNM-Main PADM 522 Planning for a Service Program Evaluation Specify Evaluation Use • LLS-findings will be used for program sustainability and continued and amount of future funding. • Evaluations used to make decisions. • Important to understand the implication of negative findings • Will this program be duplicated? Will the literature/results be made available nationally? Validate Perspectives • Communications with nearly every level of LLS/IRC • Welcome evaluation • Supportive of the project • Important to understand that it is a program evaluation

  10. Engage Stakeholders John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation • Potential stakeholders for a service program include: • Two main reasons why the evaluator must engage stakeholders in evaluation planning. • Identify and Invite Stakeholders. • Establish rapport. • Participants involved in the evaluation planning. • “Sponsor” role. • Define Stakeholder Roles and Structure for Their Input. • Importance of engaging the few stakeholder groups. • Survey design.

  11. Engage Stakeholders John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation • Establish Group Processes for Ongoing Stakeholder Involvement. • The communication process. • Methods used. • Evaluation plan timeline. • Preliminary reports.

  12. Describe the Program John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation • Establishment of the stakeholder. • Become immersed in the nature of the program to be evaluated. • “Framework for Program Evaluation in Public Health”. • Understanding the IRC. • Conceptualize Program Theory or Rationale. • Usefulness of logic models and conceptual models. • Evaluation Planning Matrix (EPM). (Holden and Zimmerman, pg.78-79) • Lists the evaluation questions considered. • Short-term, Intermediate, or Long-term outcomes and identifies relevant data sources corresponding to each question. • Outcome surveys and questions.

  13. Focus the Evaluation John Toadlena UNM-Gallup PADM 522 Planning for a Service Program Evaluation • Focusing the evaluation. • The planning process. • Study design team. • Preliminary evaluation questions. • Theoretical frameworks. • Ensure Feasibility. • The focus on feasibility. • Control or comparison group. • Assess Potential Data Collection Burden. • Data collection burden an issue on two levels. • How much surveying could the participants tolerate. • What kinds of data collection operation could IRC take on. • Sample size. • Draw conclusions from the results.

  14. Lessons Learned Ervina Castillo-Newton UNM-WS PADM 522 Planning for a Service Program Evaluation • Service Programs are unique • Start with the Mission Statement • Size and scope of the activities • Evaluation • Attitudes • Stakeholders’ knowledge • Stakeholders • Expertise & knowledge • Minimizes resistance • Planning Process • Dissemination • Goals • This case – positive and knowledgeable

  15. Conclusion Ervina Castillo-Newton UNM-WS PADM 522 Planning for a Service Program Evaluation Comparison Group • Based on comparison of callers • Standard Service through IRC • Enroll in intervention • Information Specialists • Qualified • Trained • Experienced • Outcomes

More Related