Study population

The National Data Protection Authority (Commission Nationale de l’Informatique et des Libertés) approved the study on October 23, 2015 (n° 1859704). All data were fully anonymized before we accessed them.

The unit of analysis was the hospital stay, integrated an episode of hospital admission in gynecological unit. All hospital stays of women between 18 and 45 years old, admitted in two tertiary teaching gynecological units for acute pelvic pain or/and vaginal bleeding in 2012, were included. Hospital stays in a clinical context of post-partum, post-surgery or post-abortion, less than six weeks of the event, and of women with a history of chronic pelvic pain or whose pregnancy was beyond 15 weeks of amenorrhea, were excluded. During the study period, some women were hospitalized twice either for two different medical events (i.e., one CEPSB and one ovarian cyst rupture) or for the same pathology (i.e., one pelvic infection treated by antibiotics and a second hospitalization for laparoscopy). We counted each hospitalization as a hospital stays.

For our population, we recorded the specific clinical situation that justified each hospital stay. In case of several hospitalizations for the same patient, the administrative data (i.e., date of hospitalization or duration of hospitalization) helped us select the right medical record.

Data sources

Data processing and statistical analysis

Identification of CEPSB from HDD.

Two algorithms were built to extract hospital stays related to CEPSB from the Medical Information Department.

The first algorithm (Table 1), called the “predefined algorithm” was constructed a priori from the coding guidelines of the MISP. This algorithm was composed of the following items: 1) administrative information related to inclusion criteria (sex: female; age: between 18 and 45 years; hospital stay during the year 2012 in gynecological unit); 2) diagnosed pathologies based on specific ICD-10 codes (EP for the main diagnosis and pregnancy-associated complications for the associated diagnosis); 3) specific procedures related to the EP treatment using the CCAM thesaurus (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Items integrated into the algorithms used to identify hospital stays for complicated ectopic pregnancy with severe bleeding (CEPSB) in the HDD.

https://doi.org/10.1371/journal.pone.0217674.t001

After this first data extraction, the association between the ICD-10 or CCAM codes and the data from hospital stays classified as CEPSB in the HDD was used to define the second algorithm. This “pragmatic algorithm” was constructed from codes with statistically significant associations and a positive likelihood-ratio > 4 (Table 1).

The pragmatic algorithm (Table 1) was composed of two items: 1) the diagnosed pathology, either the main or associated diagnosis with specific ICD-10 codes for EP; 2) EP-related procedures coded using the CCAM thesaurus.

Main findings

The principal finding of our study is that the HDD has a poor performance to detect cases of CEPSB. Our method, which involved the successive application of two algorithms, improved detection performance. Most of the errors in the two tertiary teaching gynecological units were due to incorrect coding practices, despite the presence of specific codes in ICD-10 and surgery procedures for CESPB.

Strengths and limitations

We present a pragmatic method to improve detection of CEPSB with a view to evaluating quality of care. Our research took into account both coding guidelines and observed coding practices to build two algorithms applied successively as mentioned above. The first algorithm measured the gap between data from the HDD and the medical records, while the second improved the capability of the HDD to detect CEPSB. Our results justify the use of a rational approach, evaluating the inclusion or not of diagnosis and procedures according to coding guidelines on the one hand, and according to coding practices on the other.

Similarly to our study, other studies have found that the validity of an HDD varies according to the construction of the algorithm [22] and the pathology of interest [15]. For instance, a study on the detection of breast cancer found that the inclusion of procedure codes into the algorithm improved the specificity of HDD [23]. Other studies have used several algorithms to determine the reliability and the accuracy of the HDD, but without explaining how the algorithms were built [23, 24]. Our approach, which uses coding guidelines and miscoding based on the health providers’ coding practices, would appear to be relevant.

Our study optimized the validity of HDD in detecting CEPSB among our population by applying a pragmatic algorithm. The main difference between the predefined and pragmatic algorithm is the absence of a second mandatory ICD-10 code. This could increase false negatives and fail to identify complications of EP. Furthermore, the algorithms are likely to work accurately if the coding errors were eliminated. This is a critical situation since the performance of HDD depends on coding practices, medical quality data in hospitals [25, 26, 27], or recording data practices [13]. The improved validity of HDD using the pragmatic algorithm must be confirmed with another population at a regional and national level, taking into account different types of gynecological emergencies. Nevertheless, this is the first study to examine the performance of HDD to detect CEPSB with a view to evaluating quality of care [18, 22].

Interpretation

The difficulty in detecting complicated medical situations with routinely collected HDD data throws into question its utilization as a database to identify near-miss cases to evaluate quality of care. Other studies have also assessed the use of complication codes in HDD. To evaluate the accuracy of diagnosis-type indicators for flagging complications, Quan et al. [17] attempted to select hospital stays based on complications to perform a clinical audit. They reported poor validity for some complications from HDDs for 1996 and 1997. In 1994, Hartz et al. [9] studied complications of coronary artery bypass surgery and data sources, and concluded that assessing quality of care using an HDD might be difficult. Our study comes to the same conclusion 20 years later: EP complications were under-coded and were unable to select hospital stays of interest. Although the healthcare providers’ understanding of HDD quality has improved over the last 10 years, coding practices of medical complications remains to be improved.

An analysis of coding errors may provide an understanding of miscoding practices and the reasons for misreporting medical information. As previously specified [18], data quality depends on medical information collected (diagnosis, procedures) and administrative data. In our study, coding errors were mainly due to the misuse of ICD-10 codes. Lombrail et al [28] discussed the lack of reliability of HDD in the use of uncertain diagnosis codes. Clinical difficulties in establishing an accurate diagnosis could explain the misuse of ICD-10 codes. Nevertheless, the diagnosis of CEPSB is confirmed by a surgical procedure that determines the final diagnosis during the hospital stay. Some coding errors could also be due to the use of two classifications, one for the diagnosis and one for the procedure. Other errors are due to the shortening or lengthing wording as previously described [29, 30].

In a study of the validity of severe maternal morbidity in HDD, Chantry et al. [18] reported that false positives are mainly due to the excessive use of diagnosis codes. They observed that some professionals used ICD-10 to describe a more severe medical situation than the real one. Furthermore, false negatives were mainly due to the use of non-specific procedure codes. Our study gave similar results though the proportion of coding errors was higher than in Chantry et al.’s study and the false negatives were due to inaccurate use of ICD-10 and procedure codes.

Chantry et al. [15] describe a link between the organization of medical information production and data quality in HDD. Computerization of medical data and information control in a simultaneous intermediary step were associated with an improvement in data quality. The authors show that various organizations of medical information production create differences in sensitivity and specificity of HDD in four hospitals. In our study, a significant difference was also observed between the performances of the two hospitals HDD, which could be explained by their organization.

This study is a first step for a possible program of clinical audit of pregnancy loss, such as EP, among gynecological units, since CEPSB is associated with substandard cares. Our study highlights that further improvements in coding practices need to be implemented. The implementation of a specific thesaurus with ICD-10 and procedure codes from the CCAM could help clinicians to standardize medical information collection, even if rare and severe medical situations could still be difficult to track [18, 26]. Improving the quality of HDD requires data quality assessment, including coding procedures for various clinical cases and staff training [18, 22, 29, 31, 32]. These different actions necessitate dedicated human resources [29] to ensure accurate routine care data.

These results reinforce the need for high standards in quality health data collection to be able to use HDDs for public health purposes [31].