ESSnet on Small Area Estimation

1. ESSnet on Small Area Estimation Stefano Falorsi Istat ESSnet on Small Area Estimation

2. When and How SAE? When to use SAE methods: Whenever direct estimators which are based only on sampling units observed for each small area are not reliable (small sample size or sometimes even no observed units) i.e. CVs or other measures of sampling variability of direct estimates are considered to be too high for the target indicator at area level How to use SAE methods: Exploiting relationship among variables by means of explicit or implicit modeling

3. SAE: Borrowing Strength from?

4. Partners of ESSnet SAE project Istituto Nazionale di Statistica (ISTAT) Institut National de la Statistique France et des Etudes Economiques (INSEE) Statistisches Bundesamt (DESTATIS) Centraal Bureau voor de Statistiek (CBS) Statistisk Sentralbyrå (SSB) Główny Urząd Statystyczny (GUS) Instituto Nacional de Estadística de España (INE) Office for National Statistics (ONS) Swiss Federal Statistical Office (FSO) contacts : Stefano Falorsi : coordinator of the project stfalors@istat.it saessnet@istat.it Denisa Florescu : Eurostat denisa.florescu@ec.europa.eu webpage : http://www.ess-net-portal.eu

5. Aims of the ESSnet on SAE The activity of the ESSnet - SAE was directed to analyze methods and previous experiences on SAE, focusing on the application of SAE methods for social surveys. The aim was the promotion of the use of SAE methods in the production of statistical information, through the standardization of SAE estimation process. The activities of the project were grouped into work packages in order to ensure the best interaction and mixing of know-how among the partners.

6. Outcomes of the ESSnet on SAE The project was planned in phases which are a series of theoretical and application activities in order to facilitate and promote the use of small area techniques in the production of statistical information The results of the ESSnet-SAE project will be helpful to detect the best practises to define the guidelines to be followed for the applications of SAE to give some advices to software tools users. Dissemination of results was another important step of the project, therefore a web-site is available in order to share information and the results among the partners be a forum platform in order to boost the communication among NSIs willing to apply SAE methods.

7. Description of the Project The project was composed by 7 work-packages: WP1 - Project management WP2 (GUS,Destatis, INSEE, INE, ONS) State of the art The WP2 was aimed to provide a comprehensive overview of small area estimation in the ESS social surveys with respect to implementation, needs and expectations. Last ten years SAE literature and other SAE project outcomes have been reviewed. Moreover a survey on NSI’s SAE experiences has been carried out.

8. Description of the Project WP3(INE, Istat, CBS, INE, GUS, ONS)Quality assessment This work-package aimed to review and develop suitable criteria to assess the quality of SAE methods. In details methods for: top-down assessment (from larger to smaller domains) determination of accuracy thresholds choosing among different set of auxiliary information model diagnostic (model fitting, error analysis, etc.) overall quality assessment by means of internal validation to check for bias of SAE estimates or by means of external validation using already available information (for instance Census data)

9. Description of the Project WP4 – (Istat, CBS, GUS)Software tools This WP was aimed to provide recommendations for standardization and certification of ESS software tools for SAE. For this reason, a study was devoted to analyze the real capacity of routines to be applied to large scale surveys (characterized by a large number of records and domains). New software routines have been developed in order to provide an integrated set of software tools to produce small area estimates for large scale social surveys. The need of sharing as much as possible the outputs of this WP implied the use of free licensed software and this was the main reason for the choice of the open source software R to develop new software routines.

10. Description of the Project WP4 – (Istat, CBS, GUS)Software tools The main deliverable of the work-package is a collection of R functions to perform SAE estimation, model selection and model diagnostic. The R functions will include : Estimation : unit level : Synthetic, EBLUP (uncorrelated random effects), EBLUP (correlated random effects) area level : Synthetic, EBLUP (uncorrelated random effects) unit level (EBLUP type) logistic mixed model Model choice : conditional AIC and cross validation (unit and area level linear mixed model) unit level : Synthetic, EBLUP (uncorrelated random effects), EBLUP (correlated random effects) Model diagnostic : Bias diagnostic, Goodness of fit diagnostic, Coverage diagnostic, Calibration diagnostic

11. Description of the Project WP5 (CBS – FSO, GUS.INSEE, INE, Istat, SSB)Case studies The activities of this WP focused on applying significant SAE methods acknowledged in WP2 and relevant tools for model diagnostic, model selection, quality assessment identified in WP3. The case studies will also be the ground for training the software or routines developed in WP4. The NSIs’ case studies focus on: CBS: Crime survey FSO: New Census strategy: simulation study from previous census GUS: Labour Force survey INE: New Census strategy: simulation study from previous census INSEE: Labour Force survey ISTAT: Health survey - SSB: Register of deaths: modelling mortality rates

12. Description of the Project WP6 (Istat –CBS,FSO, GUS, INE, ONS, SSB)Guidelines This work-package aimed to summarize the activities and the results produced in the previous WPs and to provide practical guidelines in ESS social surveys context. The guidelines contain: a description of the process that should be followed when applying SAE methods; - standard SAE methods and their main extensions; - main tools to assess the quality of the estimates; recommendations about the use software and routines to be used to produce small area estimates.

13. Main WP6 Outcomes: Guidelines

14. Main WP6 Outcomes: Guidelines

15. Description of the Project WP7 - (Istat - CBS - Destatis, FSO, GUS, Insee, INE, SSB, ONS)Transfer of knowledge and know-how The WP7 aimed to transfer knowledge and know-how to non participating NSIs and to disseminate the results through a course and three trainings on the job. A website devoted to the ESSnet-SAE project was designed within the ESSnet portal. All the final reports and software tools are available at the ESSnet portal http://www.essnet-portal.eu/.

16. Future work? Issues on SAE not exploited in the project but of great interest The ESSnet-SAE covered only univariate cross-sectional methods. The following items were let aside: strategies for SAE at the design stage time correlation for repeated surveys multivariate responses SAE for business surveys (now research activity in WP6 of BLUE-ETS and one of the topic covered by the MEMOBUST project)

17. Future Istat’s work Relevant Istat’s works in progress in this field are: is under studyan extension of web SMART, SMall ARea estimation Tool (http://smart.istat.it/smart/) to include variables related to income and poverty. The tool is available only for LFS ILO variables: each user may perform an online application of SAE estimates for municipalities or their aggregations. The level of aggregation is defined interactively by the user. The tool do not produce official statistics but only domain estimates useful for research or policy. Istat is conducting a study on “Small area estimation of poverty rate at different territorial dissaggregations” The empirical performances of different model based SAE estimators will be compared using EU-SILC data: ELL method (Elbers et al. 2005), EB (Molina and Rao 2010), EBLUP unit level with spatial correlation, furthermore Modified Direct and Bias corrected projection estimators will be considered too; The work will compare the results of the above estimators for different domain levels: NUTS3, LAU1, LAU2.