Learning from Hospitals' Stage 1 Meaningful Use Performance

[Anita Karcz]

We recently reviewed the data from the CMS Web site regarding individual hospital performance for Meaningful Use Stage 1, before the FFY 2014 requirements were in effect[i].

3814 hospitals were listed; some twice or three times depending on the number of years they attested for Stage 1 prior to FFY 2014. We analyzed Stage 1 performance for the core and menu measures that required reporting a numerator and a denominator.

 Core measures

Hospitals performed very well on the core measures:

The reported compliance with measure 1 and 3 was striking, given conventional wisdom and experience with physician use of electronic systems. 46% of hospitals had a 90% or greater compliance rate with measure 1. Only 13% of hospitals had a compliance rate less than 60%. The scatter plot below of number in denominator versus compliance percentage does not show any obvious relationship of volume (a proxy for hospital size) with CPOE compliance.

Meaningful Use incentives have increased CPOE installations significantly in a relatively short time. HIMSS Analytics had reported in 2009[ii] that only 45% of hospitals had installed CPOE. The 3814 hospitals in this sample must have had CPOE installed to have attested and they represent 78.3% of U.S. hospitals[iii].  The data reported by CMS for these hospitals is from 2011, 2012 and 2013. This is an undercount of the true number of installations since there are many hospitals that have installed CPOE and are not included in this attestation list.

Compliance with measure 3 is difficult to interpret since there was ambiguity in the measure and its interpretation.  In Stage 1, some hospitals used ICD9 codes to designate problems while others focused on the use of a specific problem list function.

The following table displays the mean and median of the number of cases in the denominator reported by the 3814 hospitals:

Measure 11 and 12 are strikingly low in volume. These measures were dependent on patients requesting the relevant documents in electronic format. 2583 of the hospitals reported a zero denominator for measure 11 and 2464 of the hospitals reported a zero denominator for measure 12. The median denominator for year 2 hospital data (2440 hospitals) was 4 and 8 respectively for these two measures and the average denominator was 203 and 945 respectively.

Menu measures

The following table displays similar calculations for the menu measures requiring a numerator and denominator to be reported.

While we expected compliance to be high for measure 2 and 3, compliance was over 80% for the other three menu measures. Only 8.4% of hospitals reported the transfer of care summary measure.

The following table displays the mean and median of the number in the denominator for the hospitals reporting these menu measures:

Compliance Change from Year 1 to Year 2

We compared year 1 and year 2 core measure performance and found that compliance with the CPOE and smoking status measures improved.  For the menu measures, providing patient specific education resources improved. The rest of the core and menu measures showed no change in performance between year 1 and year 2.

We examined the proportion of hospitals reporting 100% compliance on core measures and how that changed from Year 1 to Year 2. There was a significant drop in the percentage of 100% compliance hospitals from Year 1 to Year 2 for Core 1, 4, 5, 6 and 7.  Since the Year 1 reporting period was only 90 days and the year 2 reporting period was a full year, this is not surprising.

Implications

1. Evaluating your hospital’s performance rank on these measures can help identify problem areas for the 2014 measures. See the reference link below to download the detailed spreadsheet.
2. Very few patients in Stage 1, less than 1% of the median denominator, requested e-copies of the specified documents.  This low percentage argues for active strategies to encourage patient portal access.
3. Compliance is so high on some of these measures that policy makers will likely have the same conversations as in the past on certain quality metrics: If there is sustained high performance, discard those metrics and move on to others that need improvement. Hopefully this will not happen soon since the pace of change in this program has been rapid enough for the moment.

Anita Karcz, MD is the Chief Medical Officer at IHM Services