Overview of the Regional Technical Forum Guidelines January 22, 2013

1. Overview of theRegional Technical ForumGuidelinesJanuary 22, 2013

2. After A Decade of Making Decisions the RTF Recently “Codified” Its Decision-Making Process

3. Why Guidelines For Estimating Savings? • RTF Mission • Reliable estimates of savings • Transparent methods for estimating savings • RTF operated for 10 years transparently but its “rule book” evolved with experience • RTF needed to capture its current best practices to ensure more consistent decision-making and operations

4. Brief History and Coming Events • Guidelines Development • RTF identified need to codify guidance on how it does its work • Developed and tested guidelines for savings estimation methods. • Second version adopted 12-11-2013 • Also, developed guidance on how to estimate measure cost and lifetime • Next Steps • Developing an overall Roadmap and integrating all guidelines and supporting tools

5. Guidelines Scope and Intent • Describe how the RTF selects, develops and maintains approved methods for estimating savings from the delivery of energy efficiency measures • Four savings estimation methods: • Unit Energy Savings (UES) – RTF Decides on • Standard Protocols – RTF Decides on • Custom Measure Protocols– RTF provides guidance on • Program Impact Evaluations – RTF provides guidance on • Each method is intended to produce savings estimates of “comparable reliability” at the lowest reasonable cost

6. Key Concepts – Measure and Savings • Measure • …changes in system configuration, equipment specifications or operating practices • …reduces electric power consumption as a result of increases in the efficiency of energy use, production, or distribution • Savings • …difference in energy use between the baseline and post periods • … caused by the delivery of a measure • …terms “net” or “gross” are intentionally not used

7. Savings Reliability and Diligent Review • Savings Reliability • …sampling, data collection and other errors are sufficiently small and unbiased. • Sufficiency is determined by the collective opinion of the RTF • Diligent Review • RTF uses estimates of parameters, e.g., average length of a residential shower or heat/cool interaction factors, from studies performed by other agencies • Diligent review required before use

8. Diligent Review • Diligent review includes, but is not limited to understanding • characteristics of the sample studied • study’s data collection methods and analysis methods • variability of parameter estimates • feasibility of normalizing results to this region

9. Best Practice Savings Estimate • Best Practice Savings Estimate • …relies on the best practical and reliable data collection and estimation methods • …can be carried out with proven techniques and resources deemed reasonable by the RTF • …may rely on parameter values from studies performed by other agencies, if found to be sufficiently reliable based on diligent review.

10. Two Possible Measure Baselines • Current Practice Baseline • …used if the measure affects systems, equipment or practices that are at the end of their useful life • …includes measures delivering new equipment or practices • …baseline is defined by the recent typical choices of eligible end users in purchasing new equipment and services • Pre-Conditions Baseline • …used when the measure-affected equipment or practice still has remaining useful life • …defined by typical existing conditions found among eligible end users • For custom measures choices of individual end users define typical current practice conditions

11. Transparency and Accessibility • All methods have documentation standards • Sources must be cited • Data and documents must be accessible • Must show your work • Estimation algorithms • Derivation of input parameters • Software must be either inherently transparent, such as a Microsoft Excel® workbook or fully documented

12. Method 1 - Unit Energy Savings (UES) • RTF approves measure methods and savings values • Stable, unitized savings that can be reliably forecast through the period defined by the measure’s sunset date • Reduce program delivery costs by simplifying the data that must be collected • Verified count of delivered units is all that is required • Statistical or meta-statistical analysis used to reliably estimate savings • Examples: Res. Insulation CFL

13. Getting to Unitized Savings (UES)* • Statistical or meta-statistical data • Quality judged by relative error of mean savings estimate • Avoid when savings significantly interact with other measures due to large sample needs • Calibrated engineering models • Adjusted to individual cases or to the average characteristics and consumption of groups • Ex. SEEM heating loads calibrated to billing data from representative sample of SF homes • Savings expected to be regionally applicable • Significant interactions need to be dealt with • Planning/Provisional provides a path *The savings formerly known as “deemed”

14. Method 2 - Standard Protocols • RTF approves saving estimation method • Intended for measures with widely varying savings, but where data collection and analysis can be standardized • Site-specific data collection and analysis is the minimum required for a reliable savings estimation • Known as simplest reliable savings estimation method, +/- 20% error that is unbiased compared to best practice • Data collection standardized and savings calculator developed for each protocol • Examples: T8 Lighting VFD

15. Method 3 - Custom Protocols • RTF provides guidance on appropriate methods and documentation standard • Appropriate for measures that require site-specific savings estimation planning, data collection and analysis • Likely numerous components affecting operation, maintenance, and energy use of many separate systems or pieces of equipment • Savings report required • Example: Energy Management System (EMS)

16. Method 4 – Program Impact Evaluation • RTF provides guidance on appropriate methods and documentation standard • Based on reliable sample of program participants (and possibly non-participants) to determine the savings achieved during a program delivery period • UES claim & delivery; Standard Protocol faithfully applied • Rely on the other three methods, plus guidance provided for “Other UES” measures • Examples: O-POWER

17. Measure Category/Quality Standards

18. Guidelines Measure Status

19. Sunset Date • As part of approving a UES or standard protocol measure, the RTF will establish a sunset date for the measure • Beyond this date, the RTF may decide that the measure’s savings estimation method is no longer RTF-approved. • …RTF records factors that justify each sunset date… • revisions to energy codes or federal standards • completion of RTF approved research plans • shifts in current practices of consumers

20. Provisional Category and Research Planning • Defined for UES and Standard Protocol • RTF approves plans for research needed to advance provisional measure to the proven category • Provisional UES measures need data on efficient-case energy use • Provisional standard protocol measures need data to prove simplest reliable method

21. Development of Application of RTF-Savings Estimation Methods

23. Cost and Lifetime Guidelines • RTF also approved guidelines for measure cost and lifetime estimation in 2011 • Capital and O&M costs along with a measure’s expected useful lifetime are used by the RTF in evaluating regional cost-effectiveness using the ProCost model • Program operators may use these values in performing their own cost-effectiveness analyses