Home › Harmonizing survey methods

Harmonizing and improving surveys methods

The need for harmonizing survey guidelines has been repeated in various high level meetings over the last decade. In February 2004, the Marrakech Action Plan for Statistics stated: "Surveys become still more valuable when they allow comparisons to be made with other surveys and data sets. Better timing and standardization of surveys would increase the coverage and comparability of the results obtained. Yet there are many examples of the parochial interests of one donor seeming to outweigh the interests of the country (and other users) in maintaining comparability over time or across countries."

The issues raised by the "stove-pipe production" of statistics are not specific to developing countries. The most advanced statistical agencies also have to address them. The following references provide a useful overview of the problem and its possible solutions, as described by Statistics Canada.

See also the National Statistics Code of Practice - Protocol on Statistical Integration and Classification published by the Office for National Statistics of the United Kingdom (Version 1.2, February 2008).

How does IHSN contribute?

To foster harmonization and improvement of data collection methodologies, the IHSN is supporting the work of task forces or experts working on assessment and improvement of existing methods (see section below on our partnerships), and is maintaining a Question Bank.

The Question Bank is a central repository of indicators, classifications, concepts and questions, and reference materials. It provides international guidelines on survey design and indicators calculation, from various international sources. The Question Bank does not contain any data or actual survey questionnaires (see the IHSN Catalog of Survey Questionnaires).

The Question Bank application was developed by the IHSN; the maintenance of its content is a collaborative effort of IHSN member agencies and partners. The materials available in the Question Bank come from two main sources:

  • Existing international guidelines (e.g., from well established household survey programs such as the Demographic and Health Surveys (DHS), the Multiple Indicator Cluster Surveys (MICS), or the Living Standards Measurement Studies (LSMS).
  • New guidelines produced as part of the IHSN work program, in collaboration with specialised agencies, partners and experts. In such cases, the approach of the IHSN is to undertake a comprehensive assessment of existing survey methods (including analysis of the microdata generated by these surveys), and to seek a consensus with key stakeholders. This work on "assessment, harmonization and improvement of survey modules" has become a key component of the IHSN work program in late 2009.

Partnerships for the development of improved survey guidelines

  • Credit and debt

    Consultants have been contracted to produce draft guidelines, which are being reviewed.

  • Education

    The IHSN contracted the Education Policy and Data Center (EPDC) to undertake an in-depth assessment of the education module from a diverse collection of household surveys. In close collaboration with the Unesco Institute for Statistics (UIS), EPDC is now developing new guidelines for the collection of education data, to be included in the IHSN Question Bank. The assessment of existing education modules will soon be available as an IHSN Working Paper, and the new guidelines will be made available in the IHSN Question Bank.

  • Food security

    IHSN is working with the World Food Program on the assessment of food security surveys.

  • Health

    The IHSN will collaborate with WHO and other partners (UNICEF, World bank, macro International and others) on the improvement of survey modules on household health expenditures (Out-of-Pocket expenditure). An expert Group meeting will take place in Geneva in january 2010 to define a work program.

  • Household consumption

    The World Bank Data Group has recently undertaken a large-scale assessment of household consumption surveys, and has attempted to standardize consumption data from numerous countries. This work has revealed multiple issues in the design of consumption or expenditure survey modules. With IHSN support, a report on common issues is being produced, and recommendations for the harmonization and improvement of household consumption survey modules will be produced. The output of this work will include an IHSN Working Paper and new guidelines in the IHSN Question Bank. This work, led by the World Bank Data Group, is suppoorted by various stakeholders (poverty analysts, national and hgealth accounts specialists, ICP Global Office, and others).

  • Labor force

    The IHSN contracted a labor force survey expert to develop new guidelines on labor force modules. The expert worked with support from the ILO statistics division. A detailed review by ILO will complete the work, which will be published in the IHSN Question bank.

  • Migration and remittances

    For many countries, remittances represent a sizeable source of funds that sometimes exceed official aid or financial inflows from foreign direct investment, and they affect economic growth and poverty reduction in these economies. Improvements in the data used to measure international remittances were first requested by Heads of State at the G8 Sea Island Summit in 2004. An international working group, coordinated by the World Bank, was formed at the request of G7 Finance Ministers.

    Improving the availability and quality of remittance data collected through household surveys is one objective of this international effort. Given the nature of remittances – relatively small amounts of money sent and received by individuals, often using informal channels – household surveys are an attractive data source, particularly as they allow the study and analysis of related socio-economic variables, such as those associated with poverty, health, and education. There are, however, specific difficulties when collecting remittance data through surveys. For surveys of remittance-senders, finding respondents may be a relatively rare occurrence, requiring special sampling techniques. For surveys involving remittance-recipients, respondents may be reluctant to divulge information, or may make recall errors. Few regular household surveys are implemented with the sole and primary objective of collecting information on remittances and migration, and a more common approach is to integrate questions on remittances and migration into household surveys designed to collect data on multiple indicators.

    The need for improved household survey data on migration and remittances was also identified by the Conference of European Statisticians, first in 2005, and also (for example) at the regional meeting on migration statistics in December 2007. Following a meeting in 2008, jointly organized by the World Bank, the US Census Bureau and the UN Economic Commission for Europe, a technical working group of interested partners and experts was formed to work on issues relating to household surveys and migration (including remittances). This is known as the Suitland Group, after the location of the US Census Bureau where the first meeting was held.

    The work plan of the Suitland Group includes several projects, but of specific interest to Thematic Area 1 (Data) of the GWRG is the development of an on-line repository of surveys which have collected data on migration and remittances, and the preparation of guidance material for survey practitioners; currently there is no survey repository, and practical guidance is available but scattered and uncoordinated. The International Household Survey Network has been approach by the Suitland Group to build an on-line repository of existing household survey questionnaires that that have been used to collect remittance and migration data. Work on this repository is expected to be complete by June 2010. Users will be able to browse information, post comments and upload new content. The on-line repository will not contain any survey data files, but will provide relevant metadata including a listing of relevant surveys, examples of questionnaires, interviewer instructions, classifications, concepts and indicators. This resource will also provide guidelines, recommendations and research aimed at improving the quality and comparability of household survey migration and remittance data.

  • Social capital

    Consultants have been contracted to produce draft guidelines, which are being reviewed.

  • Water and sanitation

    The IHSN collaborates with the Joint Monitoring Program on Water and Sanitation (UNICEF, WHO and other partners) on the extraction of water and sanitation-related variables from a large number of survey datasets. This work will allow the JMP to improve its estimates of MDG indicators, and will provide a detailed assessment of a variety of survey questionnaires.