Evaluation of Small/Medium Commercial Sector Efficiency Savings: PY2017 Draft Report

1. Small/Medium commercial PY2017 Draft Report Webinar Rachel Harcharik, John Cavalli, Phanindra Pagadala, Jeff Staller, Kris Bradley and Jennifer Fagan, Itron, Inc. Jon Maxwell, Patrick Hewlett, Bryan Kilgore, Kelly O’Connell, ERS Mike Yim, Tierra Resource Consultants, LLC March 15, 2019

2. Agenda • Evaluation Overview • Goals and objectives • Highlights of Approach • ESPI Measure List • Sampling Plan vs Accomplishments • Key Gross Impact Methods • NTG Approach • Gross Impact Results • EUL Results • Net Impact Results • Net Realization Rate Results • Summary of Recommendations

3. Evaluation overview • Small and medium commercial sector • PY2017-PY2019 ESPI uncertain measures • Deemed (i.e., prescriptive) nonresidential measures (excludes lighting and HVAC) • Annual impact evaluation • Gross and net savings estimates • Reporting scheduled to meet EM&V Bus Stops; March 1 each year

4. Goals and objectives • Develop first year and lifecycle net and gross savings (ex-post) • For ESPI uncertain measures • Assigned to the small and medium commercial sector • Develop reliable estimates of key impact parameters, EULs and load shapes (ex-ante) • Support the development of updated deemed savings values • Develop recommendations to improve program performance in delivering energy efficiency savings

5. Highlights of approach • Measure-by-measure data collection plan • Verification on-sites • Targeted short-term monitoring • Telephone surveys • Apply tailored gross impact evaluation methods • Estimate NTGRs • Updates applied to CPUC approved methods • Additional updates anticipated in PY2018 evaluation

6. ESPI measure list • Uncertain Measure List -- Small/Medium Commercial Sector • Source: Hansen, R., 2016. Final 2017 Efficiency Savings and Performance Incentive (ESPI) Uncertain Measures List. December 22, 2016. • * 2017 program participation in the water heating controls measure is all multi-family (MF) and is therefore assigned instead to the residential evaluation. • ** “X” designation indicates ESPI measures that are not being selected for evaluation. Bolded “G” and “N” designations indicate ESPI measures that were selected for evaluation, with “G” identifying gross impact evaluation scope and “N” indicating net impact evaluation scope.

7. Sampling plan vs accomplishments • PY2017 Impact Evaluation Sample Sizes

8. Key gross impact methods

9. Net-to-gross approach • Leveraged existing NTG approach from 2013-15 Evals. • Removed PAI-1 score from algorithm based on recommendations from 2015 reports • Limited variation in scores • 80% of scores between 4.4 and 5.4 (4.9 mean) • Highest program and nonprogram score a 9 or 10, 72% and 80% of the time, respectively • Similarity with PAI-2 scores • Weak correlation with reported ‘no program’ behaviors • Self-Reported Do Nothing – PAI-1 = 4.89 • Self-Reported Do Same Thing – PAI-1 = 4.79 • Revising Workplan in Spring 2019 • Additional revisions to NTG approach are likely

10. Gross Impact results • First Year Impacts • * Gross ex-post evaluation was not performed for Food Service – Other and Water Heating Boiler measures, so ex-ante savings were passed thru. • † The poorer-than-anticipated relative precision is largely due to a single site with ex-post savings over 400% higher than ex-ante. The RP improves to 21% if that site is not considered.

11. Gross Impact results (CONTINUED) • Refrigeration Case LED Lighting Discrepancy Factors for PG&E

12. Gross Impact results (CONTINUED) • Refrigeration Case LED Lighting Discrepancy Factors for SCE

13. Gross Impact results (CONTINUED) • Refrigeration Case LED Lighting Discrepancy Factors for SDG&E

14. Gross Impact results (CONTINUED) • Process Boiler Discrepancy Factors for PG&E

15. Gross Impact results (CONTINUED) • Process Boiler Discrepancy Factors for SCG

16. Gross Impact results (CONTINUED) • Gas Fryer Discrepancy Factors for PG&E

17. Gross Impact results (CONTINUED) • Gas Fryer Discrepancy Factors for SCG

18. Gross Impact results (CONTINUED) • Agricultural Irrigation Discrepancy Factors 5 of 19 sampled projects determined ineligible due to crop type (deciduous) or preexisting irrigation type (flood or furrow).

19. Gross Impact results (CONTINUED) • Pipe Insulation Hot Application Discrepancy Factors Significant differences between workpaper savings and reported first-year savings for all 9 PG&E projects in the sample. SCG FY GRR: 133% PGE FY GRR: 21%

20. Effective useful life results • Ex-Ante vs Ex-Post Sample-Based EUL • * The ratio of PG&E's ex-ante lifecycle savings to first-year savings was 3.7 years, or one-third of the EUL. However, the lifecycle savings appeared to have been originally calculated using an 11-year EUL. This issue led to erroneously high first-year savings and resulted in a low first-year GRR for PG&E projects.

21. Gross Impact results (Continued) • Lifecycle Impacts • * Gross ex-post evaluation was not performed for Food Service – Other and Water Heating Boiler measures, so ex-ante savings were passed thru. • † The poorer-than-anticipated relative precision is largely due to a single site with ex-post savings over 400% higher than ex-ante. The RP improves to 13% if that site is not considered.

22. Net Impact results • Net-to-Gross Ratios • * The NTGR for the SDG&E Downstream Food Service measure was set equal to the weighted average of the PG&E and SCG Downstream NTGRs. • ** The NTGR for the SCE Refrigeration Case LED Lighting measure was set equal to the overall measure level NTGR.

23. Net realization rate results • First Year Impacts

24. Net realization rate results (cont’d) • Lifecycle Impacts

25. Summary of recommendations • Update workpapers based on recent evaluation data and results • Improve the accuracy of reported claims • Revise parameter level assumptions • Correct errors and omissions • Improve screening of projects for eligibility requirements • Enhance verification protocols • Improve EUL estimates • Address tracking system reporting errors

26. Summary of recommendations (Cont’d) • Update program-eligible technologies • Enhance program influence • Research new technologies • Introduce less well-adopted or emerging technologies • Continue to monitor program free ridership (FR), and in response to higher levels of FR: • Adjust program design and promotion • Thus actively discourage FR