Improving the efficiency of editing in ONS business surveys

1. Improving the efficiency of editing in ONS business surveys Rachel Skentelbery Office for National Statistics, UK

2. Overview • Introduction to Eden Project • Selective Editing Methodology – Short term surveys • Annual Business Survey • Overview of SELEKT • Results • Collaborations and next steps

3. Eden Project • New editing strategy for the ONS following visit in 2007 from Australian Bureau of Statistics • Aim to improve the efficiency of editing process for business surveys, based on Australian model • Deliver efficiencies with little impact on quality • Better balance between micro and macro editing • Implementation of new selective editing for RSI and MBS

4. Eden Principles • 1. Holistic Approach • 2. Sound Methods • 3. Maximise Impact of Resources • 4. Process Quality • 5. Informed Data • 6. Continuous Quality Improvement

5. Selective editing methodology - STS • Targets potential errors which have a significant impact on key estimates • Selective editing works by: • assigning a score to each business; • the score reflects the impact that editing the response would have on the estimates; • businesses with a score above their domain threshold are validated; • those businesses with a score below the threshold are not validated.

6. Quality Measures - ARB • Absolute relative bias aims to control the residual bias left in the domain estimates after editing

7. Savings • Savings measure the change in the number of units that will be manually micro edited

8. Results – Short Terms Surveys • The number of forms failing has reduced by approximately 20% on both RSI and MBS • Quality – ARB below 1% for all domains • Monitoring a number of ‘quality indicators’

9. Annual Business Survey (ABS) • Large business survey split by different sectors of the economy • Many questions asked from each business • Large number of edit rules applied to identify suspect values in returned data

10. Alternative approach needed for ABS • Current selective editing is unsuitable for ABS due to • Large number of (key) survey variables • Lack of sufficient predictors needed to calculate scores • Collaboration between ONS, Statistics Sweden and Pedro Silva (University of Southampton) to investigate use of SELEKT

11. SELEKT Tool • Generic software to perform Selective Editing • Developed at Stats Sweden by A. Nordberg and team • Set of SAS macros • Driven by large set of user-specified parameters • Can deal with large, complex surveys like ABS

12. SELEKT Scoring Method • Local scores defined for each combination of: • Domain of interest (d) • Variable (j) • Record / unit (i) • Local score used in SELEKT considers three components: • Potentialimpact • Suspicion • Importance • Global score is obtained by aggregating local scores

13. Results from testing • Production & Construction Sector • Total number of records: 4,164 • Records flagged by original edits: 2,923 • Catering Sector • Total number of records: 784 • Records flagged by original edits: 620

14. Work in progress • Completed feasibility study – looking to implement • Fine-tuning parameters for P&C and Catering • Also testing Statistics Sweden LAB to help choose best performing parameters • Set up parameter table for all other sectors • Plans for use in ABS, 2012 dispatch