Iraq Family Health Survey

On January 9, 2008 the World Health Organization reported the results of the "Iraq Family Health Survey" published in the New England Journal of Medicine. The study surveyed 9,345 households across Iraq and was carried out in 2006 and 2007. It estimated 151,000 deaths due to violence (95% uncertainty range, 104,000 to 223,000) from March 2003 through June 2006.

The study was done by the "Iraq Family Health Survey Study Group", a collaborative effort of six organizations: the Federal Ministry of Health, Baghdad; Kurdistan Ministry of Planning, Erbil; Kurdistan Ministry of Health, Erbil; Central Organization for Statistics and Information Technology, Baghdad; World Health Organization Iraq office, Amman, Jordan; World Health Organization, Geneva.

Methods

The Iraq Family Health Survey (IFHS) was a cross-sectional, nationally representative survey of 9345 households that was conducted by relevant federal and regional ministries in Iraq in collaboration with the World Health Organization (WHO). The sampling frame that was used in the southern and central provinces was derived from the 1997 Iraq census, which had been updated for the 2004 Iraq Living Conditions Survey. The sampling frame used in Kurdistan was based on information provided by the Statistical Offices in the region. Population estimates for Iraq for the survey period were projected by Iraq's Central Organization for Statistics and Information Technology (COSIT).

Training of central and local supervisors from all 18 governorates was conducted in Amman, Jordan. Training of interviewers was done separately in each governorate for one week during May and June 2006 and a one-day refresher training session was conducted the day before the start of the survey in each governorate. Following interviewer training, the survey instruments and procedures were pilot tested in all governorates. The survey fieldwork was conducted during August and September 2006 in the 14 South/Centre governorates. In Anbar governorate, the fieldwork was conducted in October and November 2006, while fieldwork for the Kurdistan region was conducted during February and March 2007. Overall, 407 personnel participated in the implementation of the survey, consisting of 100 central, local and field supervisors, 224 interviewers evenly split between males and females, and 83 central editors and data entry personnel.

The survey had an original target sample size of 10,080 households. Interviewers visited 89.4% of 1086 household clusters during the study period and the household response rate was 96.2%. 115 clusters (10.6%) were not visited due to security problems. The IFHS argues that past mortality is likely to be higher in these missed clusters. Ratios derived from comparing their surveyed clusters to corresponding data from Iraq Body Count were used to impute elevated mortality rates to these missed clusters. Using this method, the IFHS derives death rates for the missed clusters in Baghdad that are 4.0 times as high as the clusters visited by the survey in Baghdad. The same procedure in Anbar derived rates that are 1.7 times as high as clusters visited in Anbar.

The IFHS also argues that underreporting is likely to be common in household surveys, particularly due to household dissolution after the death of a household member. On this basis the authors used several demographic assessments such as the growth balance method to derive an adjustment for reporting bias. This adjustment raised the violence-related death rate by roughly 50%, from 1.09 (95% CI, 0.81 to 1.50) to 1.67 (95% uncertainty range 1.24 to 2.30).

Results

The number of violent deaths derived from the household survey, plus the adjustments for missing clusters and reporting bias, was estimated to be 151,000 (95% uncertainty range, 104,000 to 223,000) from March 2003 through June 2006. This estimate suggests that violence was a leading cause of death for Iraqi adults and was the main cause of death in men between the ages of 15 and 59 during the reporting period.

The results were also compared to the 2006 survey by Burnham et al. reported in the Lancet, and to the Iraq Body Count project (IBC). Naeema Al Gaseer, the WHO Representative to Iraq noted, "Our survey estimate is three times higher than the death toll detected through careful screening of media reports by the Iraq Body Count project and about four times lower than a smaller-scale household survey conducted earlier in 2006," referring to the survey by Burnham et al. reported in the Lancet journal in October 2006.

The NEJM article on the survey states that both the IFHS and the IBC "indicate that the 2006 study by Burnham et al. considerably overestimated the number of violent deaths. For example, to reach the 925 violent deaths per day reported by Burnham et al. for June 2005 through June 2006, as many as 87% of violent deaths would have been missed in the IFHS and more than 90% in the Iraq Body Count. This level of underreporting is highly improbable, given the internal and external consistency of the data and the much larger sample size and quality-control measures taken in the implementation of the IFHS." The article also notes that the IFHS and IBC are consistent with each other on both the distribution of violent deaths by province and on the trend in levels of violent deaths over time, while both sources are inconsistent with the results of the Burnham et al. survey on these issues.

The IFHS also collected data on health issues beyond mortality. One such notable finding of the survey was that a worryingly low 57% of the women surveyed said they had heard of AIDS, as compared with 84% of women in Turkey and Egypt, 91% in Morocco and 97% in Jordan.

Reactions and Criticisms

An accompanying commentary appeared in the editorial 'Perspectives' section of the New England Journal of Medicine issue that contained the study. The authors commend the IFHS study group for its attempt to capture the highest-quality results, but also discuss "substantial limitations" of the IFHS and more generally of the use of household surveys to estimate mortality in circumstances such as Iraq, saying:

The sampling frame was based on a 2004 count, but the population has been changing rapidly and dramatically because of sectarian violence, the flight of refugees, and overall population migration. Another source of bias in household surveys is underreporting due to the dissolution of some households after a death, so that no one remains to tell the former inhabitants' story. Mortality estimates that are derived from surveying deaths of siblings were also calculated, but this method may also be subject to such underreporting. ... Under the current conditions in Iraq, it is difficult to envision a study that would not have substantial limitations. The circumstances that are required to produce high-quality public health statistics contrast starkly with those under which the IFHS study group worked. Indeed, it must be mentioned that one of the authors of the survey was shot and killed on his way to work.

Paul Spiegel, a medical epidemiologist at the United Nations High Commission on Refugees in Geneva, commented, "Overall, this is a very good study," adding that "this does seem more believable to me" than the earlier Lancet survey, which estimated 601,000 deaths from violence over the same period.

Officials in the Iraqi government had differing reactions to the report. The Iraqi Health Minister Saleh al-Hasnawi described the survey as "very sound" and said the survey indicated "a massive death toll since the beginning of the conflict," and "I believe in these numbers". However, a senior official in the Iraq Health Ministry's inspector general's office cast doubt on the findings, saying 151,000 was far too high because the numbers cited by the study were much larger than figures tracked by the ministry. On the other hand, Jalil Hadi al-Shimmari, who oversees the western Baghdad health department, said the 151,000 total seems roughly accurate but is probably a "modest" one and that "the real number might be bigger than this."

Many criticisms leveled at the IFHS estimate have originated from authors of the Lancet survey or others associated with that survey. Some of these criticisms relate to an idea that respondents would have a fear of giving information about violence-related deaths to government interviewers who represented "one side of the conflict" and the reliance on Iraq Body Count's data in dangerous areas (Anbar and Nineveh provinces and parts of Baghdad).

In a January 11, 2008 article Les Roberts, co-author of the Lancet study, is quoted as saying:

They roughly found a steady rate of violence from 2003 to 2006. Baghdad morgue data, Najaf burial data, Pentagon attack data, and our data all show a dramatic increase over 2005 and 2006.

On February 24, 2009 Morning Edition discussed what a Baghdad central morgue statistics office worker reported to them:

the number of deaths the morgue registers never corresponds with numbers from the Ministry of Health or the Ministry of Interior. "They do it on purpose," he says. "I would go home and look at the news. The ministry would say 10 people got killed all over Iraq, while I had received in that day more than 50 dead bodies just in Baghdad. It's always been like that — they would say one thing but the reality was much worse."

400,000 excess deaths?

A July 2008 article in MedPage Today asserts that the IFHS survey actually estimated 400,000 excess Iraqi deaths as a result of the invasion, with 151,000 being violence-related deaths (95% uncertainty range, 104,000 to 223,000) and 249,000 from non-violent causes (unknown uncertainty range). However, the Q&A published on the IFHS by the World Health Organization states that it did not estimate excess deaths:

Q: Does this estimate represent "excess" violent deaths -- those attributable to the invasion?

A: No. It is an estimate of how many violent deaths occurred between the March 2003 invasion and June 2006. The study did not measure whether or not those deaths would have occurred had there been no invasion. However, the mortality rates for 2002 and early 2003 showed that mortality due to violent causes was low before the invasion.

The main NEJM article only briefly discusses its measured increase in non-violent mortality rates from the pre-invasion to post-invasion period:

Overall mortality from nonviolent causes was about 60% higher in the post-invasion period than in the pre-invasion period. Although recall bias may contribute to the increase, since deaths before 2003 were less likely to be reported than more recent deaths, this finding warrants further analysis.

The Q&A elaborates:

Q: What happened to mortality due to causes other than violence?

A: The non-violent mortality rate increased by about 60%, from 3.07 deaths per 1000 people per year before the invasion to 4.92 deaths per 1000 people per year in the post-invasion period. This was not further addressed in this analysis, which focused on mortality due to violent deaths. Further analysis would be needed to calculate an estimate of the number of such deaths and to assess how large the mortality increase due to non-violent causes is, after taking into account that reporting of deaths longer ago is less complete....

John Tirman, who commissioned and directed the funding for the 2nd Lancet study, stated in a January 21, 2008 AlterNet article:

a little digging would have revealed much more: The total deaths attributable to the war, nonviolent as well as violent, was about 400,000 for that period, now 19 months ago. If the same trends continued, that total today would be more than 600,000.... Interviewers identified themselves as employees of the Ministry of Health, then under the control of Shiite cleric Moktada al Sadr. Those interviewed, therefore, would be wary of saying a brother or son or husband had been killed by violence, fearing retribution. And, indeed, there are nonviolent categories in the survey that suggest just such equivocation: 'Unintentional injuries' would equal about 40 percent of the death-by-violence toll, for example. Road accidents were ten times their pre-war totals-if someone is run off a highway by a U.S. convoy, is that a "nonviolent" death?

Les Roberts said Friday, January 10, 2008:

The NEJM article found a doubling of mortality after the invasion, we found a 2.4-fold increase. Thus, we roughly agree on the number of excess deaths. The big difference is that we found almost all the increase from violence, they found one-third of the increase from violence....

They roughly found a steady rate of violence from 2003-2006. Baghdad morgue data, Najaf burial data, and our data all show a dramatic increase over 2005 and 2006. ...

It is likely that people would be unwilling to admit violent deaths to the study workers who were government employees. ...

Finally, their data suggests one-sixth of deaths over the occupation through June 2006 were from violence. Our data suggests a majority of deaths were from violence. The morgue and graveyard data I have seen is more in keeping with our results.

John Tirman wrote on (February 14, 2008) in Editor and Publisher:

Yet another, a much larger house-to-house survey was conducted by the Iraq Ministry of Health (MoH). This also found a sizable mortality figure—400,000 "excess deaths" (the number above the pre-war death rate), but estimated 151,000 killed by violence. The period covered was the same as the survey published in The Lancet, but was not released until January 2008. The ORB results were almost totally ignored in the American press, and the MoH numbers, which did get one-day play, were covered incompletely. Virtually no newspaper report dug into the data tables of the Iraqi MoH report, published in the New England Journal of Medicine, for that total excess mortality figure, or to ask why the MoH report showed a flat rate for killing throughout the war when every other account shows sharp increases through 2005 and 2006. The logical explanation for this discrepancy is that people responding to interviewers from the government, and a ministry controlled by Moktada al Sadr, would not want to admit that their loved one died by violence. There were, instead, very large numbers of dead by road accidents and "unintentional injuries." The American press completely missed this.

Timothy R. Gulden, Ph.D., of the University of Maryland School of Public Policy in College Park, asserted that the Iraq Family Health Survey authors "acknowledge and attempt to correct for underreporting of deaths from nonviolent causes, but they make no allowance for the more serious underreporting of violence-related deaths to government-affiliated survey takers."

Earlier Iraqi Health Minister estimate in November 2006

When the Iraq Family Health Survey was published in January 2008 the Iraqi health minister was Dr Salih al-Hasnawi.

The previous Iraqi health minister, Ali al-Shemari, in early November 2006 estimated between 100,000 and 150,000 people had been killed since the March 2003 U.S.-led invasion. The Taipei Times reported on his methodology: "Al-Shemari said on Thursday [Nov. 9, 2006] that he based his figure on an estimate of 100 bodies per day brought to morgues and hospitals -- though such a calculation would come out closer to 130,000 in total."

The Washington Post reported: "As al-Shemari issued the startling new estimate, the head of the Baghdad central morgue said Thursday he was receiving as many as 60 violent death victims each day at his facility alone. Dr. Abdul-Razzaq al-Obaidi said those deaths did not include victims of violence whose bodies were taken to the city's many hospital morgues or those who were removed from attack scenes by relatives and quickly buried according to Muslim custom."

From a November 9, 2006 International Herald Tribune article:

"Each day we lost 100 persons, that means per month 3,000, per year it's 36,000, plus or minus 10 percent," al-Shemari said. "So by three years, 120,000, half year 20,000, that means 140,000, plus or minus 10 percent," he said, explaining how he came to the figures. "This includes all Iraqis killed — police, ordinary people, children," he said, adding that people who were kidnapped and later found dead were also included in his estimate. He said the figures were compiled by counting bodies brought to "forensic institutes" or hospitals.

From the November 11, 2006 Taipei Times article:

An official with the ministry also confirmed the figure yesterday [Nov. 10, 2006], but later said that the estimated deaths ranged between 100,000 and 150,000. "The minister was misquoted. He said between 100,000-150,000 people were killed in three-and-a-half years," the official said.

See also

• Council of Ministers of Iraq
• Casualties of the Iraq War. An overview of many casualty estimates.
• Iraq War. Infobox there has the most up-to-date casualty figures.