Poor data hamstrings gender equity reporting in India

Author: Vikas Kumar, Azim Premji University

Achieving gender equity is a major development challenge facing countries such as India. The sex ratio — the ratio of the number of females per thousand males — is a key measure of the scale of the challenge. In the late twentieth century, there was a precipitous decline in the child sex ratio in India — from 962 in 1981 to 918 in 2011.

During this period, the overall sex ratio fluctuated between 927 and 943. The latest National Family Health Survey (NFHS) suggests that India’s overall sex ratio increased from 991 in 2015–16 (NFHS-4) to 1020 in 2019–21 (NFHS-5). The ratio was 1000 in 2005–06 (NFHS-3).

The change in sex ratio between 2005–2006 and 2019–2021 is not large and the NFHS has always reported a higher ratio than the Census of India. But the latest NFHS data received media attention because for the ‘first time on record’ a major survey may have suggested that there are more women than men in India.

Prime Minister Narendra Modi credited the improved sex ratio to government policies, even though the NFHS is not designed to evaluate the impact of specific policies and researchers had already questioned the quality of the data. Currently, the NFHS is possibly the only official source of information on the sex ratio, as the 2021 census has been delayed due to the pandemic. It is, therefore, important to assess the reliability of the NFHS-5 findings.

An improvement in the estimated sex ratio can be attributed to better counting of women, poorer accounting of men, misclassification of sex, an improvement in sex ratio at birth and a relative improvement in female life expectancy. The reported improvement will have to be apportioned among the above categories to identify the actual increase and rule out the possibility that it is an artefact of variation in data quality.

The NFHS is not meant to generate reliable sex ratio estimates. The NFHS failed to release conceptual metadata that would have cautioned readers against misleading comparisons with the census, which follows a different reference date and definition of population.

The NFHS conducts field interviews in two phases for logistical reasons. The reference period for questions that are reference-period specific is different between the two phases. It is not clear how the NFHS arrives at the national-level estimates using non-synchronous state-level estimates. In 2019–2021, the two phases had different reference periods and also differed vis-à-vis seasonal effects on migration that can affect estimates of the sex ratio. This affects the comparability of the NFHS and the census, different rounds of NFHS and even the two phases of NFHS-5.

The NFHS non-synchronously samples the de facto population — ‘all persons who stayed in the selected households the night before the interview, whether usual residents or visitors.’ The census follows an extended de facto (synchronous) method that simultaneously counts people (where they are found during enumeration) across the country over three weeks. The NFHS definition biases the sex ratio upwards by undercounting migrants, who are often male.

The NFHS reports are also deficient with respect to metadata on methodological issues and the quality of data. In 2019–2021, 22 states/union territories (UTs) accounting for half of the country’s population were covered in the first phase (2019–20) with the remaining fourteen states/UTs being covered in the second phase (2020–21). The second phase fieldwork had to be divided into two parts due to the pandemic.

NFHS interviewer’s manuals and reports are silent on how the pandemic, which necessitated social distancing and mobility restrictions, affected field interviews. This affects the comparability of the NFHS-5 and earlier rounds.

NFHS-5 fact sheets did not mention the extent of sample non-coverage and non-response, which obscured the systematic differences in data quality between the two phases. As per the state reports, there was a drop in response rates in the second-phase states between 2015–2016 and 2019–2021. For instance, compared to 2015–2016, the median response rate for men in 2019–2021 was more than two percentage points higher for first-phase states, but it was more than two and a half percentage points less for second-phase states.

In short, the optimism regarding the improvement in sex ratio is marred by the poor quality of metadata. Ad hoc clarifications issued later failed to address the core issues. For instance, it was suggested that the NFHS reports a higher sex ratio because it may have undercounted male outmigrants and does not cover institutions (hostels, hotels, rescue homes and jails) that have a low sex ratio.

In 2011, institutional households accounted for 0.68 per cent of India’s population. So, their exclusion can only account for a small part of the higher sex ratio in 2019–2021. The sex ratio of second-phase states such as Uttar Pradesh, which in recent decades saw high outmigration, should have decreased due to the pandemic-induced reverse migration. But all but one large second-phase states reported an increase in their sex ratio since 2016. Another ad hoc explanation relates to higher mortality among men due to COVID-19. Even if we assume that men accounted for all pandemic deaths, that can at best explain a 2.5-point increase in the sex ratio reported by the NFHS.

To conclude, NFHS is not designed to generate precise estimates of the sex ratio. Its estimates are not comparable with the census, and the poor quality of metadata impairs even comparisons between different rounds of the NFHS. The manner of release of the 2019–2021 NFHS results draws attention to the risks of publishing decontextualised statistics and highlights how policymaking in India continues to be hamstrung by poor quality data.

Vikas Kumar is Associate Professor at Azim Premji University, Bengaluru.

The post Poor data hamstrings gender equity reporting in India first appeared on East Asia Forum.

from East Asia Forum