Chapter 14. Surveillance systems, health facility assessments and other sources of data

Overview

The two major classes of data sources for health inequality monitoring are population-based sources (e.g. household health surveys, civil registration and vital statistics (CRVS) systems, censuses) and institution-based sources (e.g. administrative records from health or other sectors). There are, however, many other sources that may be used for health inequality monitoring. Surveillance systems and health facility assessments contain elements of both population-based and institution-based sources. Relevant data may also be derived from health and academic research, nongovernmental organizations, corporate entities and elsewhere. For certain applications of inequality monitoring, such sources contain data about health indicators and/or dimensions of inequality, helping to fill information gaps.

Surveillance systems draw from a range of data sources to monitor a specific disease or condition (e.g. public health emergencies), aimed at triggering a response. They may collect data actively or passively; may represent the population through comprehensive or sentinel designs; and may apply different case definition criteria. Health facility assessments, including health facility sample surveys and health facility censuses, provide detailed information relevant to health service delivery at health facilities. Other sources of data, such as those derived from health research, health-care financing analyses and modelling exercises, use various methods to collect data for specified purposes and topics of interest.

This chapter discusses the main characteristics of surveillance systems and health facility assessments, highlighting how they may be used for health inequality monitoring. It also acknowledges the possibility of using data from a variety of other sources.

Surveillance systems

The purpose of surveillance systems is to detect, report and respond to specific notifiable or reportable conditions. They rely on inputs about health events from population-based and institution-based data sources and may also integrate other sources of ad hoc data. Data may be collected actively (e.g. in clinics and camps serving refugees and displaced groups, or during outbreaks from known diseases) or passively through other established data sources. In addition to information about the specific condition, the minimum essential surveillance data required to guide response and prevention efforts include the time, place and basic characteristics about the affected person, which are often limited to age, place of residence and sex (although surveillance systems may also collect information about a wider range of dimensions of inequality). Surveillance systems may be designed as sentinel or comprehensive, or a combination of both (Box 14.1) (1).

BOX 14.1. Sentinel and comprehensive surveillance systems

Sentinel surveillance systems identify a selection of health facilities that are required to report, often with intensified timely data collection. This approach allows for tracking patterns in reporting cases and is appropriate for common diseases that do not require immediate public health action (e.g. diseases that are not targeted for eradication or elimination, such as influenza and other viral respiratory diseases).

Comprehensive (also known as universal) surveillance systems require that all sources report diseases or hazards that are subject to mandatory notification under notifiable diseases lists. This approach enables immediate public health action, because a single case may be sufficient to warrant action. It is suited for situations such as diseases targeted for eradication or elimination, contaminated food or medicine, severe diseases with a high potential for spread, and severe adverse reactions or death following use of medicines or vaccines.

Surveillance systems may be a combination of sentinel and comprehensive approaches, such as across different parts of a country or during different times of the year (2).

Surveillance systems encompass a broad set of methods for handling data from a range of sources. They may rely on established standard case definitions, systematic reporting protocols, laboratory capacity, centralized reporting and analysis, and responses to early warning signals, as applicable to their function (Box 14.2). Although surveillance systems have traditionally been designed for epidemic-prone communicable diseases, they have also been developed for other purposes, such as monitoring public health trends, vital events, chronic diseases, risk factors and demographic information. Digital public health surveillance, which relies on data from digital sources such as social media, news media, discussion forums, internet search engines and other web-based sources, is covered in Chapter 16.

BOX 14.2. Indicator-based and event-based surveillance functions

Conventional surveillance systems tend to rely on indicator-based surveillance (1). This is a form of passive surveillance that makes use of the routine reporting of health information according to standardized case definitions. These data, which tend to have high reliability, are typically obtained passively through the health-care system, such as through local public health units, laboratories, registers or surveys. Indicator-based surveillance is used to monitor the frequency, origin and distribution of reportable diseases. Data obtained through passive surveillance may include information on inequality dimensions and are likely to be useful for inequality monitoring over time. For example, in the United States of America, the Notifiable Infectious Disease Data Tables, prepared by the Centers for Disease Control and Prevention, report weekly-aggregated data for national notifiable infectious diseases and conditions (3).

Event-based surveillance is a hallmark of early warning systems (1). It relies on a more directed approach to detect emerging events and public health threats through ad hoc data sources such as the internet, media, informal networks or big data. Event-based data can enhance the sensitivity of a surveillance system because they can provide information before human cases occur or before an event is detected or reported through conventional means. Information detected through event-based surveillance may be unstructured and unreliable, and it does not necessarily adhere to case definitions. This form of surveillance is usually put in place when faced with an outbreak. It may have more limited applications for health inequality monitoring, especially if information about inequality dimensions is not available. At the onset of the COVID-19 pandemic, many event-based surveillance systems emerged, including wastewater monitoring, and data collected for small geographical areas, which enabled early prediction of case increases (4).

A patchwork of surveillance systems exists nationally and internationally, addressing diverse aspects of public health. These systems are part of emergency responses. They cover a variety of diseases and threats. In some contexts, they may play a role in collecting demographic information. For example, outbreak disease surveillance systems aim to track cases of epidemic-prone diseases and their risk factors. They rely on frequent reporting by health facilities, including laboratories, as the main source of data. Risk factor surveillance collects information on noncommunicable diseases, often focusing on data obtained through surveys. Demographic surveillance systems are common in countries where the coverage of CRVS systems is very low. Although they are not representative of the wider population, they help to provide an overview of basic information, such as population-level deaths and causes of death (5).

Some of the key challenges associated with surveillance systems stem from financial and health workforce demands, because these systems are costly to establish and maintain and require well-trained epidemiologists. Working with country partners, WHO has developed numerous surveillance systems. The WHO COVID-19 Detailed Surveillance Data Dashboard, for example, includes data related to COVID-19 cases, deaths and case fatality ratios, disaggregated by age and sex (6, 7). In the wake of the COVID-19 pandemic, and with the aim of strengthening global health emergency preparedness, response and resilience, attention has turned to the concept of collaborative surveillance, which emphasizes establishing intentional collaboration across diverse surveillance systems as part of strengthening coordinated actions (8).

Use of surveillance system data for inequality monitoring

Surveillance systems can generate standardized data according to specific case definitions, yielding data that can be compared across settings, over time and between populations. Data from high-quality surveillance systems tend to be up to date and produced frequently across multiple sites or settings. This can enable regular, repeated inequality monitoring and benchmarking, provided data on dimensions of inequality are collected alongside health data or the data can be linked to other sources containing information about dimensions of inequality. See Chapter 15 for more information on data source linking.

Surveillance systems can provide data for certain health indicators (e.g. notified cases), but corresponding information on dimensions of inequality may be more limited. For example, the WHO State of inequality: HIV, tuberculosis and malaria report included data on tuberculosis (TB) case detection rate, sourced from national TB surveillance systems. The available dimensions of inequality data enabled disaggregation by age and sex, but not socioeconomic dimensions, because data were not systematically available for all countries (9). In some cases, surveillance systems may provide a stream of data that can be linked to other data sources to enable inequality monitoring.

In general, the inclusion of small-area identifiers alongside data from surveillance systems may enable linkages and expanded use for health inequality monitoring. For example, information derived from surveillance systems about the numbers of disease cases in subgroups (numerators) would need to be combined with information about the population sizes of those subgroups (denominators, derived from a census or other source) to yield population rates, which could then be used for inequality analysis. Likewise, if a surveillance system is monitoring interventions, it may be necessary to calculate intervention coverage by linking to another data source for denominator values. For more information about defining and constructing health indicators, see Chapter 17.

In some cases, surveillance systems may yield data about small numbers of people and/or short reporting periods, making them less useful for inequality monitoring because the data are non-reportable or provide imprecise estimates due to insufficient sample sizes. This may be addressed by aggregating data across population subgroups and/or time periods before analysis. Users of surveillance data should consider data gaps and data quality when planning and interpreting inequality analysis.

Ideally, inequality monitoring requirements should be considered in the design and operation of surveillance systems to ensure they are fit for that purpose. The accessibility of surveillance system data, and therefore their usefulness for health inequality monitoring, is enhanced when national, regional or global dashboards are maintained.

Health facility assessments

Health facility assessments include health facility sample surveys and health facility censuses. These assessments periodically collect information about health facilities and the services they provide. Using trained enumerators, they gather data through various inventories such as health resource inventories, interviews with staff and clients, and observations of service deliveries. Whereas health facility surveys are conducted on a representative sample of health facilities within a country (ideally including both public and private), health facility censuses include all facilities in the country. Health facility assessments rely on a current master facility list as a reference list for facility censuses and as a facility sampling frame for facility surveys.

For the purposes of this book, health facility assessments are discussed separately from data sources that collect data at health facilities in the course of routine administrative and operational activities (see Chapter 13). There are four main ways that health facility assessments differ: data collection occurs on a periodic rather than an ongoing basis; data are collected by external enumerators rather than facility self-reporting; the assessments encompass a greater scope of information on health system inputs and outputs; and the assessments can provide information on staff and client satisfaction and client consultation processes (10). Therefore, data from health facility assessments can be used to complement and validate routinely collected data from health facility records. A health facility assessment should be conducted every three to five years using standardized methodologies and instruments (Box 14.3).

BOX 14.3. Standardized methodologies and instruments for conducting health facility assessments

The WHO Harmonized Health Facility Assessment (HHFA) is a comprehensive health facility survey that assesses the availability of facility services and the capacity of facilities to provide the services at required standards of quality (11). It covers a range of key primary health-care services and basic hospital services. The HHFA generates objective information on services offered, key resources (including infrastructure, trained staff, guidelines, equipment, diagnostic capacity, essential medicines and commodities), and management, finance and quality assurance systems. The HHFA updates and expands on the previous WHO facility assessment, the Service Availability and Readiness Assessment (SARA) (12).

The WHO Health Resources and Services Availability Monitoring System (HeRAMS) is a rapidly deployable and scalable system that supports countries with the standardized and continuous collection, analysis and dissemination of information on the availability of and accessibility to essential health resources and services (13). HeRAMS is intended for contexts where limits to access, security, time and resources do not favour traditional means of assessment and monitoring, such as the HHFA.

The United States Agency for International Development Service Provision Assessment (SPA) is a health facility survey that collects data on service availability and quality of care measures, including physical and human resources, provision of care, and experiences of care through direct observations of consultations and post-consultation interviews with clients. SPA includes a focus on antenatal care, family planning, maternity care, and services for children who are unwell (14).

World Bank Service Delivery Indicators (SDI) surveys measure primary health-care service delivery, with an emphasis on capturing the experience of the “average” citizen. In addition to measuring the availability and functioning of key medicine, equipment and infrastructure at health facilities, SDI surveys also measure health-care provider knowledge and ability using standardized clinical vignettes, absenteeism and caseload (15).

Use of health facility assessment data for inequality monitoring

If standard methodologies and instruments are used, health facility assessments yield comparable data across settings. These sources contain rich data about health facilities and services. Ideally, health facility assessment data should be available and representative at the subnational level, making them useful for within-country inequality monitoring.

Data obtained through health facility assessments are typically available at the facility or small-area level and suited to ecological analysis (i.e. analysis based on aggregated or grouped rather than individual-level data). For example, district-level health information may be combined with district-level socioeconomic information to assess socioeconomic-related inequalities in health across districts (see Chapter 25 for more about ecological analysis). Linkages with other data sources at a small-area level can further enhance possibilities for inequality monitoring. Combining facility-level data with household survey data, for example, can enable more complex inequality analysis by allowing for adjustments for the type or quality of facility that people report receiving care from (which may account for some difference in care-seeking behaviours). Linking may be done in combination with geospatial data, such as travel distance or catchment areas around health facilities, to assess inequalities in health-care distribution and access (see Chapter 15).

In some cases, it may be possible to link at the individual level through exact-match linking of individuals in population data to the exact health facility they attended, allowing for assessment of systematic differences in care-seeking behaviour between individuals with different characteristics, such as economic status.Box 14.4 contains examples of how health facility assessment data have been used to monitor inequalities.

BOX 14.4. Examples of health facility assessment data used in health inequality monitoring

Across 17 low- and middle-income countries, health facility data from SARA and SPA were linked with household health survey data from the Demographic and Health Survey (DHS) and the Multiple Indicator Cluster Survey (MICS) to explore obstetric service availability, readiness and coverage within and between countries (16).

A study in Malawi examined the relationship between distance to services and immunization coverage in a rural population, using facility data from the 2013–2014 Malawi SPA, linked with individual data from the 2015–2016 DHS (17).

In Côte d’Ivoire, a health provider assessment was conducted in a health facility census using adapted questionnaires from SARA and SPA (18). Information was linked to care-seeking data from MICS.

In Mali, data from DHS and SARA were combined to assess the service environment and service use in the country at the regional level (19).

A study conducted in rural Ethiopia explored the association between distance from health facility and early neonatal mortality in rural areas. The study used health facility data from the Ethiopian Emergency Obstetric and Newborn Care Needs Assessment, a cross-sectional facility-based census of nearly all public hospitals, health centres and private clinics in the country, and data from the DHS (20).

The Gavi, the Vaccine Alliance Full Country Evaluations Project in Bangladesh, Mozambique, Uganda and Zambia conducted joint health facility surveys with the aim of understanding and quantifying the barriers to and drivers of immunization programme performance. Data collection methods included interviews of health providers, direct observation of facility areas, direct observation of child vaccinations, and assisted observation of immunization sessions (21).

Other data sources

Data from other sources not covered in earlier sections of this chapter and in Chapters 12 and 13 are sometimes used to fill information gaps when monitoring health inequalities. These data sources may present possibilities for inequality monitoring in populations excluded from or underrepresented in other sources, or possibilities for inequality monitoring pertaining to understudied health topics or inequality dimensions. Such data may derive from health and academic research, social media, corporate entities or elsewhere, and include exploratory studies, large-scale established research programmes, monitoring and evaluation, client surveys and client feedback.

For example, at the beginning of the COVID-19 pandemic, WHO monitored country and ministry of health reporting on social media platforms to track cases and deaths, before mandatory weekly country reporting was put in place. As another example, throughout the COVID-19 pandemic, Canada leveraged nontraditional data sources and explored the potential of artificial intelligence web scraping to overcome timeliness issues and data gaps in traditional case-based information for provinces and territories.

Estimates modelled or triangulated from multiple data sources are another potential source of data for health inequality monitoring, especially if reliable direct measures are not available and if estimates are available across population subgroups. Modelled estimates draw from diverse types of information, such as epidemiological and programmatic data, taking into account the quality of available data sources (especially routine surveillance and surveys), expert opinions, and other factors such as underreporting, overdiagnosis and underdiagnosis. Modelling is most often used to generate prevalence, incidence, mortality and morbidity estimates and to create estimates that are comparable across countries. If estimates are available by population subgroup, inequality dimensions are usually limited to age or sex. In some applications, modelling exercises have been used to derive estimates across wealth quintiles (22). For more discussion on the use of modelled estimates, see Chapter 15.

These sources are varied and diverse in terms of their scope, methods and quality and therefore will not be covered here in depth. The data requirements and attributes of high-quality data sources (see Chapter 11) can be used to help assess their suitability for use in health inequality monitoring. For further discussion of novel and emerging data sources for inequality monitoring, including geospatial data, mobile and web-based surveys, health tracking applications and digital public health surveillance, see Chapter 16.

References

1. Early detection, assessment and response to acute public health events: implementation of early warning and response with a focus on event-based surveillance – interim version. Geneva: World Health Organization; 2014 (https://iris.who.int/handle/10665/112667, accessed 15 May 2024).

2. Beauté J, Ciancio BC, Panagiotopoulos T. Infectious disease surveillance system descriptors: proposal for a comprehensive set. Euro Surveill. 2020;25(27):1900708. doi:10.2807/1560-7917.ES.2020.25.27.1900708.

3. National Notifiable Diseases Surveillance System (NNDSS). Notifiable infectious disease data tables. Atlanta, GA: Centers for Disease Control and Prevention; 2023 (https://www.cdc.gov/nndss/infectious-disease/?CDC_AAref_Val=https://www.cdc.gov/nndss/data-statistics/infectious-tables/, accessed 15 May 2024).

4. Naughton CC, Roman FA, Alvarado AGF, Tariqi AQ, Deeming MA, Kadonsky F, et al. Show us the data: global COVID-19 wastewater monitoring efforts, equity, and gaps. FEMS Microbes. 2023;4:xtad003. doi:10.1093/femsmc/xtad003.

5. Health Metrics Network. Framework and standards for country health information systems, 2nd edition. Geneva: World Health Organization; 2008 (https://iris.who.int/handle/10665/43872, accessed 15 May 2024).

6. Allan M, Lièvre M, Laurenson-Schafer H, De Barros S, Jinnai Y, Andrews S, et al. The World Health Organization COVID-19 surveillance database. Int J Equity Health. 2022;21(Suppl. 3):167. doi:10.1186/s12939-022-01767-5.

7. WHO COVID-19 detailed surveillance data dashboard. Geneva: World Health Organization (https://app.powerbi.com/view?r=eyJrIjoiY2UyNmQ0MWQtYjdiZC00MmIyLWI5YmYtZmRiZWJkZDcyMDMwIiwidCI6ImY2MTBjMGI3LWJkMjQtNGIzOS04MTBiLTNkYzI4MGFmYjU5MCIsImMiOjh9, accessed 22 June 2024).

8. Defining collaborative surveillance: a core concept for strengthening the global architecture for health emergency preparedness, response, and resilience (HEPR). Geneva: World Health Organization; 2023 (https://iris.who.int/handle/10665/367927, accessed 15 May 2024).

9. State of inequality: HIV, tuberculosis and malaria. Geneva: World Health Organization; 2021 (https://iris.who.int/handle/10665/350198, accessed 15 May 2024).

10. Greenwell F, Salentine S. Health information system strengthening: standards and best practices for data sources. Chapel Hill, NC: United States Agency for International Development; 2018 (https://www.measureevaluation.org/resources/publications/tr-17-225/at_download/document, accessed 5 June 2024).

11. Harmonized health facility assessment (HHFA): comprehensive guide. Geneva: World Health Organization; 2022 (https://iris.who.int/handle/10665/365534, accessed 15 May 2024).

12. Service availability and readiness assessment (SARA). Geneva: World Health Organization (https://www.who.int/data/data-collection-tools/service-availability-and-readiness-assessment-(sara), accessed 15 May 2024).

13. Health Resources and Services Availability Monitoring System (HeRAMS). Geneva: World Health Organization (https://www.who.int/initiatives/herams, accessed 15 May 2024).

14. Service provision assessment overview. Chapel Hill, NC: United States Agency for International Development (https://dhsprogram.com/methodology/Survey-Types/SPA.cfm, accessed 15 May 2024).

15. Service delivery indicators. Washington, DC: World Bank (https://www.worldbank.org/en/programs/service-delivery-indicators, accessed 15 May 2024).

16. Kanyangarara M, Chou VB, Creanga AA, Walker N. Linking household and health facility surveys to assess obstetric service availability, readiness and coverage: evidence from 17 low- and middle-income countries. J Glob Health. 2018;8(1):010603. doi:10.7189/jogh.08.010603.

17. Johns NE, Hosseinpoor AR, Chisema M, Danovaro-Holliday MC, Kirkby K, Schlotheuber A, et al. Association between childhood immunisation coverage and proximity to health facilities in rural settings: a cross-sectional analysis of Service Provision Assessment 2013–2014 facility data and Demographic and Health Survey 2015–2016 individual data in Malawi. BMJ Open. 2022;12(7):e061346. doi:10.1136/bmjopen-2022-061346.

18. Munos MK, Maiga A, Do M, Sika GL, Carter ED, Mosso R, et al. Linking household survey and health facility data for effective coverage measures: a comparison of ecological and individual linking methods using the Multiple Indicator Cluster Survey in Côte d’Ivoire. J Glob Health. 2018;8(2):020803. doi:10.7189/jogh.08.020803.

19. Choi Y, Boiré S, Diabaté M, Temsah G, Wang W. Availability, readiness, and utilization of services in Mali: analysis of the Mali Demographic and Health Survey and service availability and readiness assessment 2018. Rockville, MD: United States Agency for International Development; 2020 (https://dhsprogram.com/publications/publication-fa136-further-analysis.cfm, accessed 15 August 2024).

20. McKinnon B, Harper S, Kaufman JS, Abdullah M. Distance to emergency obstetric services and early neonatal mortality in Ethiopia. Trop Med Int Health. 2014;19(7):780–790. doi:10.1111/tmi.12323.

21. Gavi Full Country Evaluations (FCE) Project. Seattle, WA: Institute for Health Metrics and Evaluation (https://ghdx.healthdata.org/series/gavi-full-country-evaluations-fce-project, accessed 15 May 2024).

22. Chao F, You D, Pedersen J, Hug L, Alkema L. National and regional under-5 mortality rate by economic status for low-income and middle-income countries: a systematic assessment. Lancet Glob Health. 2018;6(5):e535–e547. doi:10.1016/S2214-109X(18)30059-7.