The Automated Coding of Causes of Death in the Netherlands

Introduction

Cause-of-death statistics are an important source of information for epidemiological research or policy decisions. In 2013, Statistics Netherlands started to use IRIS, free software for the automated coding of causes of death, in the routine production process of cause-of- death statistics [1]. From 1901-2012, death certificates were coded manually. A medical coder read the death certificate and assigned a code for the underlying cause of death by applying and interpreting ICD rules and guidelines. In due course, the international ICD rules were supplemented by almost 1 000 local rules supporting the coding process. Due to technical restrictions a coder could assign a maximum of three contributing causes of death per death certificate. IRIS is expected to change the coding process and its outcomes in a fundamental way. All diagnostic expressions on a death certificate are coded and an underlying cause of death is selected by strict adherence to ICD-10 rules and guidelines [2]. However, IRIS cannot interpret obvious mistakes on a death certificate as the medical coders can. Death certificates can show a wrong order of diagnostic expressions, an underlying cause of death in a wrong position (part two of the death certificate (contributing causes) instead of on part one (causal chain of morbid events) arrows or signs used by the certifier, and local habits of reporting a direct cause of death [3]. IRIS selects an underlying cause of death by its position on a death certificate as the lowest used line is supposed to contain the underlying cause of death. The software has some additional rules for correcting errors at its disposal, but is more dependent on the quality of the input (death certificates) than medical coders are. Thus the change of manual to automated coding is expected to cause changes in cause-of-death statistics. Therefore we compared the two different methods before implementing IRIS in the routine production process on a representative sample of death certificates.

Methods and Materials

In order to study differences between automated and manual coding, we performed a so called bridge coding study. A bridge or double coding study is a comparison of two different methods on the same data set. The year 2009 provided the data. This was an average year without epidemics (flu or nor virus) coded manually in 2009-2010 without any awareness of the change to come. All death certificates of the year 2009 (n = 134 262) were entered into IRIS (version 4.4.1) in 2011- 2012. In this way, we obtained a set of death certificates coded by two different methods independently of each other. IRIS is the name of (free) software for automated coding of causes of death, developed around the year

2000 by Lars Age Johansson (Sweden) and Gerard Pavilion (France) [4]. It is a language independent version of the American system for the Automated Coding of Medical Entities (ACME) [5, 6]. After data entry, medical terms encountered on a death certificate are translated into ICD-10 codes by the use of a dictionary, enabling users to adapt the system to their own language. Then codes are (I) combined or modified as prescribed by the ICD-10, and (ii) the underlying cause of death is selected according to ICD-10 rules [7]. Cause-of-death statistics is a tabulation of these underlying causes of death. IRIS can code about 65 per cent of the death certificates without any manual intervention. About 27 per cent of the death certificates are rejected by IRIS, because of spelling errors on the death certificate or the absence of a diagnostic expression in the dictionary. A human intervention is necessary to code a cause of death. About 8 per cent of the death certificates could not be coded by IRIS, because the software is not (yet) suitable for handling the records (external causes of death, maternal deaths, perinatal deaths and stillbirths). The rejected certificates were excluded from our study in order to avoid manual interventions. In this way, we obtained a set of 86.893 death certificates coded both manually and automatically, independently of each other, suitable for comparison. The comparability ratio and the perfect compatibility percentage are common expressions of the outcome of bridge coding studies. A Comparability Ratio (CR) is defined as: the frequency of an ICD-10 code (x) as underlying cause of death when coded automatically (IRIS) divided by the frequency of that ICD-10 code (x) coded manually in the same sample of (n) death certificates: The CR indicates the expected shift in frequency of causes of death when we change from manual to automatic coding. It is a measure of reproducibility, not of validity. For example. A CR of (close to) 1, 00 indicates no difference in change of frequency of an underlying cause of death. Nothing seems to change. However, such a CR of 1, 00 can mask a change of coding practice, when the inflow of cases coded differently, equals the outflow of cases coded differently. So, for individual death certificates there can be changes not captured by the CR. Therefore a measure of validity is needed. A Perfect Compatibility Percentage (PCP) is defined as: the percentage of death certificates with exactly the same ICD-10 code (x) when coded manually or automatically:

The PCP is a measure of validity. However, there is no golden standard for serving as denominator. Because we change from manual to automatic coding, the manually coded death certificates in the sample seem to be the obvious denominator of choice. Thus, we compare the new method with the method in use. Deviations of 1, 00 should be analysed by ICD-10 code to see if the new method is an improvement in coding or not.

Results

Of the death certificates coded both manually and automatically, 75 per cent showed exactly the same underlying cause of death (ICD-10, four digits). The perfect agreement (PCP) was 84 per cent on the three- digit level of ICD-10 codes and 89 per cent on ICD-10 chapter level. The percentage of agreement between automated and manual coding decreased significantly with an increase in age of the deceased, an increase in the number of codes on the death certificate and with an increase in detail of the ICD-10 code, i.e. in general with an increase in the complexity of the death certificate (Table 1).

Table 1: Perfect Compatibility Percentage (PCP) by age and mean number of codes on a death certificate in manual (MC) and automated coding (AC). *MC: due to technical restrictions a maximum of 4 codes per case could be assigned, AC: all terms on a death certificate are coded. **different pattern of deaths with a prominent role of R54, old age as cause of death The introduction of automated coding caused a significant increase of infectious diseases (47%), non- malignant neoplasms (41%), endocrine diseases (16%), mental disorders (32%) and diseases of the nervous system (18%) as underlying cause of death. There was a significant decrease of diseases of the respiratory

Table 2: Comparability Ratio (CR): manual versus automated coding of causes of death (n = 86 930). * Significant difference on double sided T-test of percentages The PCP-s showed a strong agreement between coders and IRIS for neoplasms (98%) and cardiovascular disorders (94%) as underlying cause of death [8]. Low agreement was found for diseases of the blood forming organs (67%) and diseases of the genitourinary tract (65%) as underlying cause of death. Very low agreement was found diseases of the skin (49%) as underlying cause of death (Table 3). The table 3 shows an exchange of cases between ICD- 10 chapters coded manually and automatically. Major shifts of deaths were observed from respiratory diseases (chapter J) to mental disorders (chapter F) or cardiovascular diseases (chapter I), and from diseases of the genitourinary system (chapter N) to mental disorders (chapter F). The preference of IRIS for selecting dementia (F01-F03), Alzheimer’s disease (G30) and the sequelae of cerebrovascular accidents (I69) as underlying cause of death at the expense of COPD (J44), pneumonia (J18) or urogenital infections (N39) underlies this pattern. The shift of deaths from endocrine disorders (chapter E) to cardiovascular diseases (chapter I) was mainly caused by an exchange of diabetes (E10-E14) and the myocardial infarction (I21) or cerebrovascular accidents (I60-I69) as a cause of death. Different views on cardiovascular complications of diabetes underlie this difference in selection. The shift of deaths from diseases of the genitourinary tract (chapter N) to endocrine disorders (chapter E) was due to a different view on renal complications of diabetes. The shift of deaths from the chapter on symptoms and signs (R) to cardiovascular diseases (chapter I) and endocrine disorders (chapter E) was mainly due to a preference of IRIS for cardiac arrest (I46) or dehydration (E86) - often reported as direct causes of death - as an underlying cause of death above old age (R54). The tendency to code dehydration or cardiac arrest as cause of death can be considered an artefact of automated coding. IRIS is not able to identify causal connections with more significant causes of death such as Parkinson’s disease, diabetes or heart failure placed next to or under dehydration on the same death certificate. The exchange of cases between the ICD-10 chapters on digestive disorders (chapter K) and infectious diseases (chapter A-B) was due to a change in the coding of diarrhoea not otherwise specified (A09 instead of K52) as prescribed by ICD-10 updates.

Major causes of death such as cerebrovascular accidents, dementia, cardiac arrest, Alzheimer’s disease and sepsis showed a significant increase of respectively 11, 26, 13, 26 and 41 per cent as an underlying cause of death in automated coding. Heart failure, COPD and Pneumonia showed a significant decrease of respectively 6, 5 and 44 per cent as underlying cause of death in automated coding. The PCP-s of cerebrovascular diseases, dementia and Alzheimer’s disease are high (PCP > 90 per cent), indicating a strong agreement between manual and automated coding on the cases coded manually. However, IRIS adds cases. With respect to cardiac arrest, sepsis, heart failure, COPD and pneumonia there was not only a significant change in frequency of occurrence, but a considerable disagreement between manual and automated coding of cases as well (table 4).

Table 4: Automated (AC) versus manual coding (MC) for leading causes of death in the Netherlands. * - : No (significant) effect of changing method; bold = significant Major changes in frequency of occurrence were caused by the implementation of ICD-10 updates (gastro-enteritis), artefacts of automated coding (brain anoxia, dehydration), the (absence of) querying (brain tumours), a change of coding practice (aortic aneurysm), and (most often) by a different selection of the underlying cause of death. IRIS showed a preference towards degenerative diseases (Parkinson, multiple sclerosis) and risk factors (hypercholesterolemia) as underlying cause of death (table 5).

Discussion

Our bridge coding study showed major differences between manual an automated coding. There are several explanations for these differences. First of all, our bridge coding study showed the effect of implementing ICD-10 updates with the introduction of IRIS. The Netherlands is among the many countries that could not implement ICD-10 updates while coding manually. The ICD-10 was used as it was released by WHO in 1993 [9]. IRIS is updated every year incorporating the (yearly) ICD-10 updates. Thus, the introduction of IRIS in the Netherlands implied the implementation of all ICD-10 updates as released by WHO since 1996 at once resulting in a strong increase of infectious diseases. The ICD-10 update of January 2010 prescribes the coding of gastroenteritis (diarrhoea) not otherwise specified as A09.9 instead of K52.9. Thus the shift of deaths from the chapter on digestive system (K) diseases to the chapter of infectious diseases (A-B) is due to a change of coding in accordance with a change of opinion on the nature of the cause of death. The decrease of skin disease is also due to the implementation of an ICD-10 update with the introduction of automated coding. The update of the ICD-10 from January 2006 prescribes M72.6 as code for Fasciitis Necroticans, classifying it as a disease of the musculoskeletal system. Before the release of the update, there was no fixed code for Fasciitis Necroticans in the ICD-10. It used to be coded as L89.9, a skin disease, by a convention among medical coders. The ICD-10 update overruled this convention. The observed changes are in accordance with ICD-10 updates not applied by the medical coders, but included in IRIS. It shows how ICD-10 updates can influence mortality statistics. Apart from the Netherlands, Eurostat metadata show that most European member states coding manually, do not implement ICD-10 updates. Such countries can expect shifts in statistics because of ICD-10 updates when implementing IRIS. Next, there are artefacts of automated coding. The increase of endocrine disorders in our sample is such an artefact. IRIS prefers dehydration, classified in this chapter (E86), as cause of death above others mentioned on the same death certificate. However, dehydration is usually a direct cause of death. Therefore it is not selected as an underlying cause of death in manual coding. The same holds for cachexia, brain anoxia and cardiac arrest. They are part of the death process. When there is no clear connection with other causes of death mentioned IRIS selects these aspects of dying as underlying cause of death. However, this is not in accordance with the intention of the certifier. Medical coders tend to follow this intention. Thus manual coding is better adapted to the local habits of certifiers than an automated coding system is. Another explanation for observed differences is the querying system used for specification of codes by asking for more detailed information of the certifier. For example. When the certifier writes “lungtumor” on a death certificate, the medical coder will send a letter asking for the nature and location of the tumour. The code will be specified according to the answer of the clinician. While waiting for this answer, it will be coded as a non-malignant neoplasm (D chapter). In the absence of a querying system the number of non- specific ICD-10 codes will increase at the expense of more specific ICD-10 codes. During the bridge coding study IRIS was not able to use the information of these queries. Thus, the bridge coding study shows the effect of querying, to be reinstalled and adapted to IRIS when used in the routine production process of coding. The most important explanation for the difference between manual and automated coding is a different selection of the underlying cause of death. When a death certificate contains more than one cause of death (about 75 per cent of the cases) a selection is made according to ICD-10 rules and guidelines (Part II ICD-10) [10]. A different interpretation of these guidelines explains most of the differences between IRIS and medical coders with respect to the selection of underlying causes of death. The increase of mental disorders and diseases of the nervous system is due to the preference of dementia and Alzheimer respectively as underlying causes of death in favour of pneumonia and urinary tract infections. IRIS tends to prefer dementia or Alzheimer when mentioned on a death certificate, regardless its position. Medical coders tend to prefer diagnostic expressions mentioned in part 1 of the death certificate, the part holding the causal sequence of causes of death. Dementia and Alzheimer are often encountered, i.e. mentioned on part 2 of the death certificate, as contributing causes (about 30 per cent of the cases). They become underlying cause of death when coded by IRIS while the medical coder selected an underlying cause of death according to the position of the death certificate or according to context. Thus, a different interpretation of the death certificate causes a difference in coding. The decrease of deaths from symptoms and sign is not attributable to a specific cause of death. In general, the software prefers more specific medical diagnoses, reported anywhere on the death certificate, above signs or symptoms as cause of death. The number of codes in this chapter is seen as an indicator for the quality of mortality statistics. The less specific the reporting or coding is, the more codes will be in the signs or symptoms chapter. So the decrease of deaths in this chapter mirrors the improvement of the quality of our cause-of-death statistics by implementing IRIS. There are hardly any bridge coding studies comparing manual and automated coding in medical literature. The United States of America (USA) switched from manual to automated coding (ACME) in 1968 [11]. A report shows 96 per cent agreement of automated and manual coding [12]. The increase in detail of the ICD - ICD-8 versus ICD-10 - and the application of the system in another language and culture requires reconsideration of this figure. When comparing automated and manual coding in 1995, England and Wales found 94 per cent of the records had identical four digit codes for underlying causes of death.

However, Italy and Serbia found 70-75 per cent agreement between the manual and automated coding of death certificates [13, 14, 15]. Our finding of 75 per cent agreement is in line with these figures. This difference between the USA, UK and other countries can be explained by the necessity to use a dictionary by non- English speaking countries. Such a dictionary is not only a translation of terms to codes, but also incorporates an interpretation of medical expression against a local context. The way to report direct causes of death, medical interventions or risk factors are examples of local certification habits, not incorporated in the standard version of IRIS. Our study showed major differences between manual and automated coding for important causes of death. These findings might be influenced by local certifying habits, but indicate the need to distinguish between manual and automated coding when comparing causes of death in different countries. Bridge coding studies should be performed and published to inform users of death statistics in making such distinctions.

Conclusions

The introduction of automated coding causes major shifts in the frequency of occurrence of cause of death in death statistics. For studying trends in time or regional/international variations of causes of death, the method of coding (manual or automated) should be taken into account. The user of the system can either accept these changes or decide to correct the system. The preference of IRIS for degenerative diseases above infectious disorders seems to be in accordance with an international consensus and should be accepted for the sake of international comparability of data. Shifts caused by the implementation of ICD-10 updates, included in IRIS, should also be accepted for reasons of international comparability, although the data may not fit local circumstances. Data produced by IRIS as artefact should be corrected by manual control or by a pre selection of records expected to be handled in a wrong way, such as: external causes of death, stillbirths, non-malignant neoplasm and signs of dying like cardiac arrest, respiratory failure, dehydration, and cachexia or brain anoxia. For these categories, the medical coder is better in interpreting the intention of the certifier. So although an automated coding system can be expected to improve gradually and handle the majority of death certificates in the future, a manual control of data produced by the system will remain necessary. IRIS holds the future, but the human coder does not belong to the past.

Acknowledgements

I would like to thank the four coders of Statistics Netherlands for their participation in this study. I also would like to thank the staff members of Health Statistics for their critical comments on earlier drafts of this paper and the IRIS working group for their help in using and implementing IRIS.

Conflict of interest

There was no conflict of interest. IRIS is open source software for coding causes of death. It operates under Windows XP®. The software is available by (free) membership of the IRIS user group, a platform for further development and quality assurance of IRIS (see: www.Iris -institute.org).