Effects of implementing Computerized Provider Order Entry with various levels of Clinical Decision Support System: a systematic review

Effects of implementing Computerized Provider Order Entry with various levels of Clinical Decision Support System: a systematic

review

Prosjektoppgave av

Hallvard Solbø Hagen kull: Høst -11

Effects of implementing Computerized Provider Order Entry with various levels of Clinical Decision Support Systems: a systematic review

Hallvard Solbø Hagen, stud. med.

Abstract

Context: Errors in the phases of medication ordering have been shown to pose a huge threat to patient safety in health care. One of the means to combat these errors have been the

implementation of information technology in health care.

Objective: To analyze effects of implementation on medication errors (MEs) after stratifying trials based on the level of Clinical Decision Support System (CDSS).

Methods: Systematic search in Medline and Embase for papers describing the effect on

implementation of Computerized Physician Order Entry (CPOE) with or without (CDSS) in an inpatient setting.

Results: An increase in the effect with the increasing level of complexity from 2/6=33% for “no CDSS”, 3/6=50% for “Basic CDSS” and 8/12=67% for “Complex CDSS” was found. The effect was found both in procedural errors and clinical errors.

Conclusion: An inverse relationship between the level of CDSS and medication errors can be found.

Introduction

Medication errors (MEs) have been repeatedly demonstrated to pose a huge threat to patient safety (1) for a long time. MEs can occur in all phases from prescribing, transcription, administration, dispensing and monitoring.

One of the means to combat this challenge is the introduction of

information technology in health care. This includes, but is not limited to, the introduction of Computerized Physician Order Entry (CPOE) systems replacing the handwritten prescriptions, with a varying levels of Clinical Decision Support System (CDSS) conveying an additional effect (2, 3).

The CDSSs are constantly being developed and can include features such as drug-drug interaction checks, drug-laboratory checks, order sets and predefined clinical pathways.

These features can be adapted to the needs of the specific setting of the targeted institution. Several review articles have demonstrated the positive effect implementation can have on ME rates (3-5).

While initially gaining enthusiasm, several drawbacks have been highlighted in the recent years.

Several of these concern the level of CDSS where examples such as alert fatigue, increased workload and overdependence on technology (6) have been reported. The aim of this paper is to try to shed a light on the effect on the medication errors at each level of CDSS. To my knowledge such a stratified approach has not previously been performed through a systematic way.

Methods

The search was initiated by formulating a PICO question creating the stem for a systematic search through Ovid Medline (In-Process & Other Non-indexed Citations, 1946 to September 2016) and Embase and Embase Classic (1947 to September 2016) using free-text with synonyms, MeSH-terms and Emtree-terms (listed in appendix 1 and 2) on 26 August 2016. Bibliographies of all relevant review articles were scanned for relevant articles.

Inclusion and exclusion criteria

The criteria for inclusion were: language (english or scandinavian), analytical (7) primary study, inpatient study population, implementation of CPOE with or without CDSS, effect on medication errors.

Abbreviations ME: Medication Errors ADE: Adverse Drug Effect CPOE: Computerized

Provider/Physician Order Entry System

CDSS: Clinical Decision Support System

PE: Prescribing Error

The criteria for exclusion were: no abstract, text not available text through university library, mixed population of inpatient and outpatient, theoretical implementation of CPOEs.

Articles with a main focus on the implementation of a CPOE with MEs as one of the main endpoints were included. Previous reviews have limited its scope to prescribing errors (PE) but several papers included used a wider definition and thus included errors

committed in the phases of transcription, dispensing, administration and monitoring.

The selected articles were then classified according to a set of variables to identify the heterogeneous nature of software, study design, type of ward, length of data collection etc.

As a consequence of the aforementioned heterogeneity it was found futile to perform a meta-analysis, but rather perform a qualitative assessment after classifying the articles using level of CDSS as independent variable to examine the effect on medication errors on each level.

CDSS was defined as ”Computer-based information systems used to integrate clinical and patient information and provide support for decision-making in patient care” as proposed by the National Library of Medicine (8). Three distinct groups were created based on the descriptions in the included articles; ”No CDSS”, ”basic level of CDSS” and ”complex level of CDSS”. See table 1.

Where the functionality of the system to be implemented was described, but no indication of CDSS was given, the ”No CDSS” was chosen.

Order sets were defined as being a part of the CDSS as these would be defined by the diagnosis given to the specific patient.

Study design

Based on a classification developed by Kaushal et al (9) the study designs were classified as either 1) Randomized Controlled Trial 2) Non-

randomized controlled trial 3) Observational Study with control and 4) Observational study without control. See table 2.

Classification of effect

The classification was based on a tabulation published in a previous review article by Eslami et al (10) indexing the outcomes into positive effect, no effect, negative effect and mixed effect. The positive and negative effects were subdivided into “demonstrated effect”,

“statistically significant” and “mixed effect”. As weighting every single outcome would be

extremely time-consuming (if not to say impossible due to the heterogeneity encountered) the summarized classification was based on the effects on overall error rate (when available) or a qualitatively assumed impact on patient safety (n=2).

Table 2

Study design Desctiption 1) Randomized

Controlled Trial

Includes quasi-randomized processes such as alternate allocation

include intervention and control groups

2) Non-randomized controlled trial

Includes prospectively planned studies with predetermined eligibility criteria and outcome measures or prospective cohort studies that

3) Observational Study with control and

Includes retrospective, interrupted time series (a change in trend attributable to the intervention), case-control studies, cohort studies with controls, and health services research that includes adjustment for likely confounding variables

4) Observational study without control

Includes cohort studies without controls and case series

1. No CDSS

2. basic level of CDSS -drug-allergy check

-drug-drug interaction check -drug-laboratory check -former drug complications -duplicate check

-pregnancy warnings -dosage and/or frequency recommendation based on age or weight

-overdosing alert 3. Complex level of CDSS -order sets

-clinical pathways or

->4 of the mechanisms in the basic level

Table 1

Results

The systematic search resulted in 499 articles, 204 and 295 from Medline and Embase respectively. See figure 1. Fifty three of these were duplications and 48 did not contain an abstract. 18 review articles were found and a manual search through the review bibliographies was performed which resulted in the inclusion of 29 articles. A total of 409 articles were thereafter excluded on the basis of title and abstract. Four papers were excluded because they included data collection from outpatients or primary care patients.

Some papers collected data from previously existing CPOEs with an additional upgrade (n=13). Papers that failed to relay information on the level of CDSS were excluded (n=6).

Preliminary reports, research letters, letter to editor, commentaries, correspondence, short communications, editorials, clinical face-offs, ”in reply”, roundup, case studies, poster presentations, supplements and conference abstracts were excluded (n=32).

Of the 77 articles requested, 17 were not available through the subscriptions of university library at the University of Oslo and 36 were excluded based on the aforementioned exclusion and inclusion criteria leaving a total number of 24 articles regarded as pertinent.

The bulk of articles were from the USA (11/24=46%) with the UK (4/24=17%) and Spain (3/24=13%) as runner-ups.

CPOE

A total of 19 different CPOEs were implemented, some commercial, some homegrown. In most cases a single system was being compared to the previous handwritten system. Some trials compared two different systems to baseline (11, 12). No clinical single system was tested in more than 2 trials.

Departments

9 trials were performed at pediatric wards, 1 trial at both pediatric and adult wards, while 14 were performed at adult wards.

The types of departments were very heterogeneous with a mix of medical and surgical wards where no type of department was represented in more than 3 papers. Some papers were limited to only one ward, whereas some trials performed a hospital-wide implementation.

Study design

As previously reviews have reported (10, 13) , very few of the studies were of the RCT design (n=1) while the majority were crafted in a prospective before-after fashion and therefore categorized in group 1) (n=10). Four trials were purely retrospective and were classified in group 3). No studies fitted the criteria of group 4).

See table 2, table 3 and appendix 3.

Outcomes

A wide range of outcomes during all the phases of medication ordering were measured with the majority assessing several outcomes during the same trial. The ones concerning MEs can be placed in

Figure 1

two broad groups as suggested by Westbrook et al (11) in an attempt to classify errors committed during the prescription phase.

 procedural error: e.g. legal/procedural (local policy on the filling of prescriptions), unclear orders and incomplete orders. The group can be defined as all errors concerning the flow of information.

 clinical errors: e.g. duplication, wrong strength, wrong dose/volume, wrong rate/frequency, wrong route, wrong drug, drug not prescribed, drug-drug interaction, not indicated, wrong formulation, inadequate monitoring, allergy, wrong patient, Adverse Drug Events (ADEs) with the subdivisions such as potential ADEs, preventable ADEs. This group can be defined as all errors concerning the adherence to best practice for the specific patient.

Many of the papers also focused on the interceptability of the errors, defining the as either intercepted and non-intercepted error, meaning these could be placed in either of the above mentioned error groups depending on the nature of the error.

Table 3

Effects

Although no certain conclusion can be drawn from this material a certain increase in the effect is suggested with the increasing level of complexity from 2/6=33% for “no CDSS”, 3/6=50% for “Basic CDSS” and 8/12=67% for “Complex CDSS”.

No CDSS

In six of the papers a CPOE without CDSS was implemented (14-19) where all of them were classified as non-randomized trials with controls. Four trials showed a predominantly positive effect on

Level of CDSS

Study design

Total Positive No

effectⁱ

Negative Mixedⁱⁱ

Demon- stratedⁱⁱⁱ

Stat. sig. Mxd^iv Demon-

strated^v

Stat.

sig.

Mxd^vi

No CDSS 1)^vii - - - -

2)^viii 6 - 2 2 - 1 - - 1

3)^ix - - - -

Basic level

1) - - - -

2) 4 - 2 1 - - - - 1

3) 2 - 1 - - - 1

Complex level

1) 1 - 1 - - - -

2) 9 1 6 - - - 2

3) 2 1 - - 1 - - - -

i When reported as such by the authors, with or without statistical arguments

ii Mix of positive, absence of, and negative effects

iii When the authors report a predominantly positive effect but without reporting statistical significance.

iv Mix of statistically significant and demonstrated positive effects.

v When the authors report a predominantly negative effect but without reporting statistical significance.

vi Mix of statistically significant and demonstrated negative effects.

vii Randomized controlled trial

viii Non-randomized controlled trial

ix Observational study with control

medication errors where the trials of Shawahna et al (17) and Shulman et al (17) showed a statistical significant level whereas the trials of Warrick et al (19) and Franklin et al (16) showed a mix of demonstrated and significant effect. The only trial to show a predominantly negative effect of all papers included was Evans et al (20) while the results King et al (21) were of a mixed nature.

Basic level of CDSS

Six trials implemented a basic level of CDSS (12, 22-26). Four of the study designs were classified as non-randomized trials with control and two classified as observational studies with controls. The studies of Bates et al, Bizovi et al, Holdsworth et al and Delgado Silveira et al (22-25) showed a predominantly positive effect, all statistically significant with the exception of Bates et al which was classified as mixed level of evidence. The studies of Upperman et al and van Doormal et al(12, 26) showed mixed results.

Complex level of CDSS

Twelve trials implemented a complex level of CDSS (2, 6, 11, 27-32). The only trail, of all the papers included, performed in a randomzed controlled fashion was that of Colpaert et al(33). Nine of trials were rated non-randomized controlled trial and two as observational studies with controls. Eight papers showed a predominantly positive effect (2, 6, 11, 27-31, 33) where six of these were

statistically significant. The papers of Zucker et al and Hernandez showed a mixed effect while that of Walsh et al showed no effect post-implementation.

Discussion Trends

When classifying the outcomes as either procedural error or clinical error a trend of increased positive impact on both variables with increasing level of complexity is suggested. Especially

interesting are the reports on positive impact on the level of ADEs as one study showed that clinical error rates in fact increased, though masked by a larger reduction of procedural errors (11), yielding the impression of an overall reduction. See appendix 3.

Heterogeneity

As shown in the descriptive part of the results the clinical situations analyzed constitute a high degree of complexity in terms of age group, type of CPOE and type of department. It should come as no surprise that the rate of errors vary from ward to ward (17). Another factor, controlled for by some of the papers, is the rate of errors made dependent on the experience of the prescriber (17, 34). For example, Shawahna et al classified physicians as either junior or senior and showed that the junior doctors committed more medication errors than their senior counterparts and that the difference in demographics at the wards under scrutiny were large.

The same level of complexity is found in the design of CDSS functionality where features such as the ability to override alerts and the timing (synchronous vs asynchronous alerts(35)) vary.

Most of the studies where before-after studies without a non-intervention control ward.

While this may be considered a design weakness some trials including concomitant control wards did not experience any reduction in the control ward (11), although others did (21).

Bates et al (2) collected data through several periods and were therefore able to identify and correct system-related errors with a significant impact on error rates while other trials only collected data at two time-points and were thus unable to adjust the software based in initial audit.

Hopefully, by defining a clear research question, performing a systematic research and standardizing data extraction, some of jumbled nature of the question at hand has been counteracted throughout this paper.

Confounding factors

In some of the papers the implementation of the CPOE with or without CDSS was one of several concomitant measures intended to increase patient safety such as validation by clinical pharmacists (36) informational campaigns (17), lectures and pamphlets. One might assume that raised awareness might have contributed to a reduction in errors. (37)

There were also major discrepancies in the time between implementation to data collection ranging from 1 week to 13 months (12, 19, 26, 32, 37) though some studies collected data at several time points showing a clear trend in decrease in overall error reduction (2). One of the few papers to control for the time of year was Walsh et al showing a clear correlation between error rate and month of the year (32) where rates were higher at the beginning of the academic year.

Other facets of information technology in health care

A lot of research has also been undertaken on the implementation of other mechanisms more concerned with administration phase of medication ordering. Some of the mechanisms encountered during this search includes Radio-frequency identification, bar-code medication administration, automatic dispensing, handheld devices to increase availability on ward-rounds.

Other outcomes

Other outcomes were being measured such as time from admission to dispensing of medication, level of reconciliation of medications, radiology procedure completion time and laboratory result turn-around time (30) must be assumed to make an impact. Some papers also measured length of stay and this could be interpreted as an indirect indicator of prescribing errors(30).

Systems-related unintended consequences

Several studies commented on systems-related errors including, but not limited to Evans et al (20), King et al (15) and Warrick et al(19). Even medication errors such as increase in duplication,

overdosing, lack of prescription, the very phenomena that were being targeted, were reported (20).

Other consequences such as alert fatigue, increased workload and overdependence on technology (6) were also reported. This perspective has been highlighted by Ash et al (38), but a systematic review of was beyond the scope of this paper.

Conclusion

The overwhelming majority of implementations proved to be predominantly positive on a range of outcomes, regardless of the level of CDSS. A certain inverse relationship between the level of CDSS and medication errors was found, though one should be careful in extrapolating. Digitalizing health information technology is likely to have consequences beyond the narrow perspective of this review.

References

1. Institute of Medicine Committee on Quality of Health Care in A. To Err is Human:

Building a Safer Health System. In: Kohn LT, Corrigan JM, Donaldson MS, editors. To Err is Human: Building a Safer Health System. Washington (DC): National Academies Press (US) Copyright 2000 by the National Academy of Sciences. All rights reserved.; 2000.

2. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, et al. The Impact of Computerized Physician Order Entry on Medication Error Prevention. J Am Med Inform Assoc. 1999;6(4):313-21.

3. Rothschild J. Computerized physician order entry in the critical care and general

inpatient setting: A narrative review. Journal of Critical Care. 2004;19(4):271-8.

4. Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: a systematic review. Journal of the American Medical Informatics Association. 2008;15(5):585-600.

5. Bubalo J, Warden BA, Wiegel JJ, Nishida T, Handel E, Svoboda LM, et al. Does applying technology throughout the medication use process improve patient safety with antineoplastics? Journal of Oncology Pharmacy Practice. 2014;20(6):445-60.

6. Armada ER, Villamanan E, Lopez-de-Sa E, Rosillo S, Rey-Blas JR, Testillano ML, et al.

Computerized physician order entry in the cardiac intensive care unit: effects on prescription errors and workflow conditions. J Crit Care. 2014;29(2):188-93.

7.

2009;14(2):37-8.

8. MeSH 2016

[Internet]. 2016. Available from: https://www.nlm.nih.gov/mesh/MBrowser.html.

9. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003;163(12):1409-16.

10. Eslami S, Keizer NFd, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patients-A systematic review. International Journal of Medical Informatics. 2008;77(6):365-76.

11. Westbrook JI, Reckmann M, Li L, Runciman WB, Burke R, Lo C, et al. Effects of two commercial electronic prescribing systems on prescribing error rates in hospital in-patients:

A before and after study. PLoS Medicine. 2012;9 (1) (no pagination)(e1001164).

12. van Doormaal JE, van den Bemt PMLA, Zaal RJ, Egberts ACG, Lenderink BW, Kosterink JGW, et al. The Influence that Electronic Prescribing Has on Medication Errors and

Preventable Adverse Drug Events: an Interrupted Time-series Study. Journal of the American Medical Informatics Association. 2009;16(6):816-25.

13. Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW, et al.

What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? Journal of the American Medical Informatics Association.

2009;16(4):531-8.

14. Evans KD, Benham SW, Garrard CS. A comparison of handwritten and computer- assisted prescriptions in an intensive care unit. Critical care (London, England). 1998;2(2):73- 8.

15. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect of computerized physician order entry on medication errors and adverse drug events in pediatric inpatients.

Pediatrics. 2003;112(3 I):506-9.

16. Franklin BD, O'Grady K, Donyai P, Jacklin A, Barber N. The impact of a closed-loop electronic prescribing and administration system on prescribing errors, administration errors and staff time: A before-and-after study. Quality and Safety in Health Care. 2007;16(4):279- 84.

17. Shawahna R, Rahman NU, Ahmad M, Debray M, Yliperttula M, Decleves X. Electronic prescribing reduces prescribing error in public hospitals. Journal of Clinical Nursing.

2011;20(21-22):3233-45.

18. Shulman R, Singer M, Goldstone J, Bellingan G. Medication errors: a prospective

cohort study of hand-written and computerised physician order entry in the intensive care

unit. Critical care (London, England). 2005;9(5):R516-21.

19. Warrick C, Naik H, Avis S, Fletcher P, Franklin BD, Inwald D. A clinical information system reduces medication errors in paediatric intensive care. Intensive Care Medicine.

2011;37(4):691-4.

20. Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF, Jr., et al. A computer-assisted management program for antibiotics and other antiinfective agents. The New England journal of medicine. 1998;338(4):232-8.

21. King C, Hackett E, McKenchie E, Higgins KS, Gallagher A. An inpatient audit of insulin dose omission. Diabetic Medicine. 2015;32:113-4.

22. Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. Jama. 1998;280(15):1311-6.

23. Bizovi KE, Beckley BE, McDade MC, Adams AL, Lowe RA, Zechnich AD, et al. The effect of computer-assisted prescription writing on emergency department prescription errors.

Academic emergency medicine : official journal of the Society for Academic Emergency Medicine. 2002;9(11):1168-75.

24. Delgado Silveira E, Soler Vigil M, Perez Menendez-Conde C, Delgado Tellez De Cepeda L, Bermejo Vicedo T. Prescription errors after the implementation of an electronic

prescribing system. [Spanish]. Farmacia Hospitalaria. 2007;31(4):223-30.

25. Holdsworth MT, Fichtl RE, Raisch DW, Hewryk A, Behta M, Mendez-Rico E, et al.

Impact of Computerized Prescriber Order Entry on the Incidence of Adverse Drug Events in Pediatric Inpatients. Pediatrics. 2007;120(5):1058-66.

26. Upperman JS, Staley P, Friend K, Neches W, Kazimer D, Benes J, et al. The impact of hospitalwide computerized physician order entry on medical errors in a pediatric hospital.

Journal of Pediatric Surgery. 2005;40(1):57-9.

27. Cordero L, Kuehn L, Kumar RR, Mekhjian HS. Impact of computerized physician order entry on clinical practice in a newborn intensive care unit. Journal of perinatology : official journal of the California Perinatal Association. 2004;24(2):88-93.

28. Huertas Fernandez MJ, Baena-Canada JM, Martinez Bautista MJ, Arriola Arellano E, Garcia Palacios MV. Impact of computerised chemotherapy prescriptions on the prevention of medication errors. Clinical & translational oncology : official publication of the Federation of Spanish Oncology Societies and of the National Cancer Institute of Mexico.

2006;8(11):821-5.

29. Kim GR, Chen AR, Arceci RJ, Mitchell SH, Kokoszka KM, Daniel D, et al. Error reduction in pediatric chemotherapy: computerized order entry and failure modes and effects analysis.

Archives of pediatrics & adolescent medicine. 2006;160(5):495-8.

30. Mekhjian HS, Kumar RR, Kuehn L, Bentley TD, Teater P, Thomas A, et al. Immediate benefits realized following implementation of physician order entry at an academic medical center. J Am Med Inform Assoc. 2002;9(5):529-39.

31. Potts AL, Barr FE, Gregory DF, Wright L, Patel NR. Computerized physician order entry and medication errors in a pediatric critical care unit. Pediatrics. 2004;113(1 Pt 1):59-63.

32. Walsh KE, Landrigan CP, Adams WG, Vinci RJ, Chessare JB, Cooper MR, et al. Effect of computer order entry on prevention of serious medication errors in hospitalized children.

Pediatrics. 2008;121(3):e421-e7.

33. Colpaert K, Claus B, Somers A, Vandewoude K, Robays H, Decruyenaere J. Impact of

computerized physician order entry on medication prescription errors in the intensive care

unit: a controlled cross-sectional trial. Critical care (London, England). 2006;10(1):R21.

34. Nelson CE, Selbst SM. Electronic prescription writing errors in the pediatric emergency department. Pediatric Emergency Care. 2015;31(5):368-72.

35. Beuscart-Zephir MC, Pelayo S, Anceaux F, Meaux JJ, Degroisse M, Degoulet P. Impact of CPOE on doctor-nurse cooperation for the medication ordering and administration process. Int J Med Inform. 2005;74(7-8):629-41.

36. Jimenez Munoz AB, Muino Miguez A, Rodriguez Perez MP, Duran Garcia ME, Sanjurjo Saez M. Comparison of medication error rates and clinical effects in three medication

prescription-dispensation systems. International Journal of Health Care Quality Assurance.

2011;24(3):238-48.

37. Zucker J, Mittal J, Jen SP, Cheng L, Cennimo D. Impact of Stewardship Interventions on Antiretroviral Medication Errors in an Urban Medical Center: A 3-Year, Multiphase Study.

Pharmacotherapy. 2016;36(3):245-51.

38. Ash JS, Sittig DF, Campbell EM, Guappone KP, Dykstra RH. Some unintended

consequences of clinical decision support systems. AMIA Annual Symposium proceedings /

AMIA Symposium AMIA Symposium. 2007:26-30.