ASG Cost Model Presentation

1. ASG Cost Model Presentation New Jersey Department of Education October 21, 2009

2. Overview • Model description/Structure • Key inputs and outputs • Databases • Reports • Uses • Questions Assessment Solutions Group www.assessmentgroup.org

3. Description/Structure • The ASG Assessment Cost Model is a variable input, metric based calculation output model • Specific assessment program variables are input to the model and applied against cost factors to derive assessment cost • Several hundred variables are contained in the model • Numerous detailed input variables are assembled for each of the functions and elements involved in constructing an assessment (test development, production and manufacturing, logistics, editing and scoring, reporting, psychometrics, program management, quality assurance, information technology, etc.) • These variables are input into the software and matched to metric based calculations/formulas and/or databases to convert the assessment program variables into dollar costs Assessment Solutions Group www.assessmentgroup.org

4. Description/Structure (cont.) • The cost for the entire assessment is built from the ground up, element by element, function by function • The model also contains “efficient cost metrics” for each key cost area and function so that a state’s assessment program cost data can be compared to those of an “efficient” assessment • This allows states to see where their assessment program is cost effective and where there are opportunities to improve costs Assessment Solutions Group www.assessmentgroup.org

5. Description/Structure (cont.) • Proprietary functions and calculations have been developed based on our years of experience in the industry • Model outputs have been tested against real data • The model is currently built for ASG use only. We are in the process of developing a web based application that could be shared with states. Assessment Solutions Group www.assessmentgroup.org

6. Cost Model-Main Menu Assessment Solutions Group www.assessmentgroup.org

7. Administration • Administrative Center • Key Functions: • - Update Rate Tables • Define Security Settings • Create Custom Reports Assessment Solutions Group www.assessmentgroup.org

8. Assessment Map Assessment Solutions Group www.assessmentgroup.org

9. Key Inputs and Outputs • Numerous assessment component inputs are available for each key area, function etc. • Assessment cost can be developed using detailed data or higher level, summary data • Cost outputs by function, area, cost type, etc. are generated • Key metrics are presented to allow comparison to model efficient cost data for that function/area/component • Canned and custom reports are available at the summary and detailed level Assessment Cost Model Assessment Solutions Group www.assessmentgroup.org

10. Key Inputs This is a sample of the key inputs. There are over 200 different inputs that are captured in the assessment model. Assessment Solutions Group www.assessmentgroup.org

11. Data Input Screens Assessment Solutions Group www.assessmentgroup.org

12. Data Input Screens - cont’d Assessment Solutions Group www.assessmentgroup.org

13. Databases/Tables The Model Contains the Following • 70+ Tables to Manage Rates & Data Capture • Rate Tables for Each Cost Component (examples): • Design/Composition Rates • Printing Rates • Freight/Shipping Rates • Labor Rates – by State, by Occupation • Rates for Delivery of On-Line Assessments • Open Ended Scoring Reader Rates • Item Acquisition/Permission Rates Assessment Solutions Group www.assessmentgroup.org

14. Key Databases - Labor Assessment Solutions Group www.assessmentgroup.org

15. Key Databases - Travel Travel Table - Includes Airfare, Lodging and Meal per diem rates for all 50 States and the District of Columbia Assessment Solutions Group www.assessmentgroup.org

16. Reports Data presented in report and graphical formats • Canned reports include: • Assessment Cost Summary • Assessment Cost by Function • Detail Assessment Functional Cost • Fixed and Variable Cost Data • Assessment Price Composition (direct, indirect and margin) • Key Metrics Summary & Functional Area Metrics Summary • Assessment Costs vs. “efficient costs” • Assessment Metrics vs. “efficient metrics” • Other reports are currently in development • Reports can also be easily customized per customer request Assessment Cost Model Assessment Solutions Group www.assessmentgroup.org

17. Reporting Assessment Solutions Group www.assessmentgroup.org

18. Reporting – cont’d Assessment Solutions Group www.assessmentgroup.org

19. Costs by Functional Area & Cost per Student Year 1 (2012) cost for each functional area and cost per student Assessment Solutions Group www.assessmentgroup.org Confidential

20. State Assessment Program Cost Data Costs for the program by functional area for 2012 - 2014 Assessment Solutions Group www.assessmentgroup.org Confidential

21. Cost per Student Cost per student for the state assessment program for the years 2012-2014 Assessment Solutions Group www.assessmentgroup.org Confidential

22. Cost by Content Area Cost by content area for the years 2012 - 2014 Assessment Solutions Group www.assessmentgroup.org Confidential

23. Cost by Grade Costs for the state assessment program by grade for 2012 - 2014. Note: grades 4 and 8 are tested more often; grade 3 uses consumable test books Assessment Solutions Group www.assessmentgroup.org Confidential

24. Cost by Content Area & Grade Costs for the state assessment program by content area & grade for 2012 – 2014 Assessment Solutions Group www.assessmentgroup.org Confidential

25. Total Costs and Key Metrics by Year Total program costs and key metrics by year for 2012 - 2014 Assessment Solutions Group www.assessmentgroup.org Confidential

26. Summary of Impact of Options/Changes in Costs Chart shows cost savings or increases related to various options/changes to the program for 2012 - 2014

27. Reporting – cont’d Assessment Solutions Group www.assessmentgroup.org

28. Uses • The model can be used in several ways to improve assessment cost and efficiency: • Developing a new RFP • ASG cost sheets key in discerning if vendors understand your program and are proposing apples to apples solutions • Making tradeoffs between various assessment components • Develop the best assessment for the available dollars • Review of an existing program to determine areas of inefficiency and opportunity • Assessment costs tend to be too high because of inefficiencies in the program design • Working with the assessment vendor to reduce/eliminate efficiencies and lower costs is a win/win for both the vendor and customer • Determine the impact of joining a state consortia • Devising scope changes Assessment Solutions Group www.assessmentgroup.org

29. Questions • ASG would be happy to answer any questions or provide a state specific demo of the model on request. • Contact information: • Barry Topol - 210-859-9920 btopol@assessmentgroup.org • Pat Hennon - 210-376-7654 phennon@assessmentgroup.org • John Olson - 210-859-5593 jolson@assessmentgroup.org • Ed Roeber - 517-614-4877 eroeber@assessmentgroup.org Assessment Solutions Group www.assessmentgroup.org