"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead
Original article

The Impact of Automated Electronic Surveillance of Electronic Medical Records on Pediatric Inpatient Care



To assess the impact of the automated surveillance of the electronic medical record process on clinical interventions among hospitalized children at a tertiary care pediatric center.


A retrospective chart review of the alerts triggered for central line-associated blood stream infections (CLABSIs), catheter-associated urinary tract infections (CAUTIs), neonatal sepsis, or clinical deterioration through elevated pediatric early warning scores (PEWS) by automated electronic surveillance of the hospital electronic medical record (EMR) over a five-month period among hospitalized children. Interventions instituted in response to the alerts were reviewed from the hospital EMR. Fisher’s exact test was performed to detect any significant difference in the proportion of interventions performed for alerts triggered between groups.


A total of 244 alerts were collected (27 CAUTI, 55 CLABSI, 10 neonatal sepsis, and 152 PEWS alerts). A significant difference in the proportion of interventions instituted after neonatal sepsis and PEWS alerts (9/162, 5.6%) as compared to CLABSI and CAUTI alerts (20/82, 24.4%) was observed (p<0.001; Odds ratio (95% CI): 0.182 (0.079-0.422)). Neonatal sepsis triggered the least number of alerts (10/244, 4.1%) and proportionately fewer interventions than the other clinical alerts.


Alerts for potential device-associated infections resulted in more clinical intervention than less-specific alerts. Neonatal sepsis alerts resulted in minimal interventions undertaken in response to the alert. Identifying and focusing on alerts benefitting the patient can serve as a better allocation of time and resources. Future studies should explore which newer alerts and their accompanying interventions improve patient outcomes.


Hospital-acquired infections pose an increased burden for patients and the healthcare system. Arterial catheters or central lines, while often necessary for the management of critically ill children, increase the risk of acquiring bloodstream infections [1-2]. Not only are these infections associated with increased morbidity and mortality but also result in increased healthcare cost and length of hospital stay [3-5]. A study that analyzed the 2008 to 2011 Nationwide Inpatient Sample (NIS) databases concluded that among pediatric patients with central line-associated blood stream infections (CLABSIs), the difference in the mean attributable cost and the length of stay for these patients as compared to patients without a CLABSI were $55,646 and 19 days, respectively [4]. Preventing these secondary infections can have a significant impact on patients and the healthcare system.

Automated electronic surveillance for identifying potential nosocomial infections offers more efficiency than manual surveillance [6-8]. Shepard et al. showed that detecting catheter-associated urinary tract infections (CAUTIs) utilizing an automated electronic algorithm reduced the surveillance requirement by 97.01% [6]. Traditional means of surveillance were time-consuming and resource-intensive. The study concluded that the increased efficiency offered by the automated electronic surveillance to detect healthcare-acquired infections should allow for more time to be spent on implementing appropriate interventions and improving patient care [6]. Another study that assessed the effectiveness of a fully automated surveillance system to detect intensive care unit-acquired infections found that electronic surveillance (sensitivity of 87% and specificity of 99%) was more effective than manual surveillance (sensitivity of 40% and specificity of 94%) and required fewer personnel resources by reducing the personnel time spent on surveillance by up to 85% (82.5 hours spent for manual surveillance versus 12.5 hours for electronic surveillance) [8]. While prior studies have shown the efficiency of the automated electronic approach in the surveillance of healthcare-associated infections, there are few data addressing the impact of such surveillance on the care of the patient.

At our institution, an automated electronic surveillance tool was implemented in 2012 to aid clinicians in detecting hospital-acquired infections, sepsis, or rapid clinical deterioration among hospitalized children. The automated surveillance system continuously monitored the Electronic Medical Record (EMR) of our pediatric inpatients to alert the clinician for the following: CLABSI, CAUTI, neonatal sepsis, and patients at risk for rapid clinical deterioration reflected by elevated pediatric early warning Scores (PEWS). CLABSI is defined by the presence of a laboratory-confirmed bloodstream infection after >2 calendar days of the placement of a central line or umbilical catheter and still in place on the date of the event of infection or removed the day before the event [9]. CAUTI describes patients who presented with fever and a positive bacterial culture of urine while an indwelling urinary tract catheter was in place [10]. Neonatal sepsis is the clinical syndrome caused by bacterial invasion of the bloodstream during the first month of life [11]. Since the immune system of neonates is functionally immature, neonatal sepsis can be rapidly fatal if left untreated or if the provision of antibiotic therapy is delayed [11]. PEWS is an objective scoring tool based on behavior, cardiovascular and respiratory status, nebulizer use, and a persistent post-surgical vomiting of the pediatric patient [12]. PEWS has been used to detect early changes in a patient’s physiological status so that appropriate and early interventions can be implemented for patients at risk of rapid critical deterioration. PEWS served as an early indicator of clinical deterioration for >11 hours prior to a code blue event in one study, forewarning the care team to respond to the change in status and possibly averting a code [13]. The automated surveillance system was enacted at our institution to assist in preventing potentially adverse outcomes.

This retrospective chart review was conducted to determine the impact of the automated electronic surveillance for CLABSIs, CAUTIs, neonatal sepsis, and PEWS alerts on the care of the patient by assessing the interventions instituted in response to the alerts. We postulated that this study should provide insight into which alerts were effective and necessary to be monitored and which alerts may be modified or removed due to a lack of patient benefit. We hypothesized that alerts triggered for neonatal sepsis and PEWS would have a higher proportion of interventions taken by the clinical staff compared to alerts triggered for CLABSIs and CAUTIs, as neonatal sepsis and PEWS could lead to rapid clinical deterioration if timely interventions were not performed [11,13-16].

Materials & Methods

Study design

Our hospital, a freestanding, 100-bed pediatric tertiary care center in Southeastern US, housed a specialized department known as the Clinical Logistics Center (CLC). Staffed by paramedics with expertise in pediatric emergency medicine for 24 hours a day, seven days a week, the CLC served to ensure patient safety by monitoring the digital input, e.g., patient vital signs, EMR, and patient location, of all admitted patients of ages 0 to 18 years from a remote location. The CLC utilized the EPIC Monitor software program to conduct continuous automated electronic surveillance of the EMR (Epic Hyperspace, Epic Systems Cor., Verona, Wisconsin, USA) in real time of all patients admitted to inpatient units, pediatric and neonatal intensive care units, as well as the cardiac critical care unit. Customized clinical rules were followed by the surveillance system to alert for CAUTIs, CLABSIs, neonatal sepsis, or elevated PEWS (see Appendix).

The trigger of an alert for a CLABSI, CAUTI, neonatal sepsis, or PEWS displayed a color-coded score on a monitor for the paramedics to view. The paramedics were then instructed to follow a protocol-driven response to ensure that the patient received appropriate care. For example, when a CLABSI was triggered, the EPIC Monitor displayed a score of 0, 1, 2, or 3 if the patient satisfied the criteria set by the customized clinical rules. A CLABSI score of 3 was triggered if the patient had a central venous catheter (central line), a documented fever of 100.4˚F or greater, and an elevated white blood cell count. A score of 0 was displayed if the patient did not have a central line in place or if the patient met the criteria for a CLABSI score of 3 but was being treated with appropriate antibiotics. No responses were taken by the paramedics for a CLABSI score of 0. Patients were placed on a watch list if a score of 1 or 2 was displayed and were monitored for score escalations. A score of 3 prompted the paramedic to further triage the alert. The alert was not only updated on the patient’s EMR for access to the patient’s attending physician but the physician was also immediately notified via text message in the event that appropriate antibiotics were not in place for the patient at risk for infection [17]. Similarly, other customized clinical rules were used to alert for CAUTIs, neonatal sepsis, or PEWS, and alert-specific protocol-driven responses were followed by the paramedics to further triage the alerts.

This retrospective study was approved by the local institutional review board. The database platform, Qlik Sense (v.2.2, QlikTech International AB, Lund, Sweden), was used to record and track the alerts triggered by Epic Monitor starting on October 13, 2015. The total alerts triggered from October 13, 2015, to the start of the study on March 11, 2016, were reviewed. However, only the alerts that satisfied the criteria for physician notification and documentation in the EMR were extracted for further investigation. The sample size, therefore, consisted of all the physician-notified alerts that occurred within the five-month time period. The patient information obtained from each alert was used to retrospectively review the EMR to identify the interventions instituted within a two-hour time frame after the alert onset. The two-hour window was selected to ensure that any interventions not immediately documented were included. The outcomes measured consisted of any clinical interventions, such as a diagnostic test, therapy change, or transfer to a higher level of care, instituted within the two-hour time period after the trigger of a CLABSI, CAUTI, neonatal sepsis, or PEWS alert. We excluded any interventions implemented prior to the alert onset since we wanted to ensure that the alerts themselves resulted in a change in clinical practice. Each patient’s reason for admission was also recorded. The collected data was extracted to an electronic spreadsheet on a password-protected computer, and any patient identifiers were deleted from the electronic spreadsheet after the completion of the data analysis.

Statistical analysis

Statistical analyses were conducted using the software SPSS 23.0 (IBM Corp., Armonk, NY, US). Fisher’s exact test was performed to assess the association between the categorical variables alert type (PEWS/neonatal sepsis and CLABSI/CAUTI) and intervention instituted (yes or no). Alert and intervention frequency, as well as the type of interventions instituted for individual alert and the patient’s reason for admission, are reported in frequency and percentage. Confidence interval and odds ratio are reported. All tests were two-tailed. An alpha level of 0.05 was considered statistically significant.


A total of 244 alerts triggered by Epic Monitor were notified to the attending physician and documented in the EMR from October 13, 2015, to March 11, 2016. Twenty-nine interventions resulted from the total 244 alerts (29/244, 12%). Of these, 162/244 (66.4%) alerts were recorded for neonatal sepsis and PEWS and 82/244 (33.6%) alerts were identified with CLABSI and CAUTI. There was a significantly (p < 0.001) lower proportion of interventions instituted after neonatal sepsis and PEWS alerts (9/162, 5.6%) as compared to interventions instituted after CLABSI and CAUTI alerts (20/82, 24.4%) (Odds ratio (95% CI): 0.182 (0.079-0.422)).

CLABSI alerts produced the highest frequency of interventions. PEWS was the most frequently triggered alert (152/244, 62%). Neonatal sepsis alerts were the least triggered and resulted in the lowest proportion of interventions as compared to the other clinical alerts studied (Figure 1). Fifteen out of 55 (27%) CLABSI alerts resulted in diagnostic and therapeutic interventions, predominantly blood culture and antibiotic administration. And, 6/8 (75%) of PEWS interventions consisted of respiratory therapeutic interventions (Table 1). Patients who activated the CLABSI alert were most often hematology/oncology patients (30/55, 54%). PEWS alerts were often triggered by patients with respiratory conditions (Table 2).

Type of Intervention Instituted in Response to Each Alert
  Neonatal Sepsis PEWS CLABSI CAUTI
Respiratory intervention 0 6 0 0
Transfer to higher level of care 0 1 0 0
Respiratory intervention & transfer to higher level of care 0 1 0 0
Antibiotics issued 1 0 4 1
Blood culture 0 0 2 0
Blood culture and antibiotics issued 0 0 8 2
Blood culture & antibiotics issued & transfer to higher level of care 0 0 1 0
Removal of catheter 0 0 0 2
Total interventions 1 (3%) 8 (28%) 15(52%) 5 (17%)

Triggered Alerts and Reason for Admission

  Neonatal Sepsis PEWS CLABSI CAUTI
Hematology/Oncology 0 (0%) 16 (10%) 30 (54%) 2 (7%)
Respiratory 6 (60%) 116 (76%) 13 (24%) 1 (4%)
Gastrointestinal 1 (10%) 3 (2%) 0 (0%) 1 (4%)
Infectious Disease 3 (30%) 1 (1%) 4 (7%) 0 (0%)
Neurology 0 (0%) 4 (3%) 2 (4%) 2 (7%)
Other 0 (0%) 12 (8%) 6 (11%) 21 (78%)
Total alerts 10 152 55 27


CLABSI and CAUTI alerts resulted in a significantly higher proportion of interventions than neonatal sepsis and PEWS alerts. We hypothesized that neonatal sepsis and PEWS alerts would result in a higher proportion of interventions based on the severity of the conditions associated with these alerts and the potentially rapid clinical deterioration that may occur should there be a delay in interventions [11,13-16].

CLABSI alerts, by themselves, resulted in the highest proportion of interventions compared to interventions instituted after CAUTI, PEWS, and neonatal sepsis alerts combined. Blood cultures, antibiotic administration, or transfer to a higher level of care were often performed after a CLABSI alert was notified to the physician. The interventions generated by a CLABSI alert were most commonly implemented for hematology-oncology patients; typically, the immunocompromised patient with a central venous catheter in place. Clinicians seemed to be cautious of the increased susceptibility to sepsis from a central line infection in these patients. In effect, the onset of a CLABSI alert would yield a high probability of a diagnostic or therapeutic intervention in these patients to prevent the occurrence of sepsis.

PEWS objectively scores the pediatric patient based on behavior, cardiovascular and respiratory status, nebulizer use, and persistent post-surgical vomiting to identify patients who are at risk of clinical deterioration [12]. Yet, out of 152 PEWS alerts, only 8/152 (5%) alerts resulted in an intervention after the alert onset. Patients who triggered the PEWS alert were often admitted for respiratory conditions such as status asthmaticus. Clinicians seemed to be aware of the deteriorating condition of the patient even before the alert onset, as appropriate interventions were already in place or were being considered in addition to the careful and frequent monitoring of the patient prior to the activation of the alert.

Neonatal sepsis was the least triggered alert and resulted in proportionately fewer interventions than CLABSI, CAUTI, and PEWS alerts. Patients who triggered neonatal sepsis alerts were often carrying a diagnosis of a respiratory illness, such as bronchiolitis, and monitored frequently. Since neonatal sepsis alerts notified to the physician resulted in few patient care interventions, identifying and focusing on other types of clinical alerts may be more beneficial to the patient. For example, abnormal heart rate characteristics, such as reduced heart rate variability and decelerations up to 24 hours preceding an abrupt clinical deterioration from neonatal sepsis, has demonstrated to provide important and more timely information than clinical laboratory values [18-20]. In fact, a continuous monitoring of heart rate characteristics was associated with reduced sepsis-related mortality among very low birth weight infants [21]. Developing a clinical alert for neonatal sepsis utilizing the continuous automated surveillance of heart rate characteristics could potentially provide more benefit to the patient and lead to an earlier diagnosis and prevention of neonatal sepsis [18-22].

Although this study was able to shed some light on the alerts that prompted interventions, several limitations have been identified that may be resolved in future studies. First, interventions documented within the two-hour time period may not have been in response to an alert and may have been placed independently. Also, the duration of this retrospective pilot study was limited to a five-month time period during the peak season of respiratory syncytial virus (RSV)/influenza infections. Patients were frequently admitted for respiratory distress, which may have influenced the larger proportion of PEWS alerts triggered during this time period. Another potential limitation was that data were only collected at a single center. Evaluating similar automated clinical alerts may elicit different results at another institution with different volumes and types of patients.

While only 12% of the total alerts resulted in interventions in this study, studies have shown that despite the growth of medical technology, approximately 5% of patient alarms actually become clinically relevant [23-24]. Additional studies reviewing the clinical circumstances of the non-actionable alerts can contribute to a further improvement of the clinical criteria set for the alerts. Furthermore, physician surveys to gauge the clinician’s perceptions toward the alerts can help understand the contribution of alarm burden and alarm fatigue to the alert response. There is definitely an increased interest in incorporating fully automated surveillance to detect healthcare-acquired infections and early signs of clinical deterioration [6-8,25-26]. Comparing alerts and interventions in other pediatric inpatient healthcare institutions that incorporate automated electronic medical record surveillance as well as analyzing their patient care outcomes would result in the ability to collect a more robust sample of data that can help identify the type of alerts greatly impacting the care of the patient.


Alerts of potential device-associated infections (CLABSI and CAUTIs) resulted in significantly more clinical interventions than less-specific alerts (neonatal sepsis and PEWS). Future studies comparing newer alerts and assessing patient care interventions may provide valuable information to clinicians on the alerts that should continue to be monitored and can improve the patient's clinical outcome.


  1. Yogaraj JS, Elward AM, Fraser VJ: Rate risk factors, and outcomes of nosocomial primary bloodstream infection in pediatric intensive care unit patients. Pediatrics. 2002, 110:481-485. 10.1542/peds.110.3.481
  2. AL-Rawajfah OM, Stetzer F, Hewitt JB: Incidence of and risk factors for nosocomial bloodstream infections in adults in the United States, 2003. Infect Control Hosp Epidemiol. 2009, 30:1036-1044. 10.1086/606167
  3. Al-Rawajfah OM, Hewitt JB, Stetzer F, Cheema J: Length of stay and charges associated with health care-acquired bloodstream infections. Am J Infect Control. 2012, 40:227-232. 10.1016/j.ajic.2011.03.014
  4. Goudie A, Dynan L, Brady PW, Rettiganti M: Attributable cost and length of stay for central line-associated bloodstream infections. Pediatrics. 2014, 133:1525-1532. 10.1542/peds.2013-3795
  5. Pittet D, Tarara D, Wenzel RP: Nosocomial bloodstream infection in critically ill patients: Excess length of stay, extra costs, and attributable mortality. JAMA. 1994, 271:1598-1601. 10.1001/jama.1994.03510440058033
  6. Shepard J, Hadhazy E, Frederick J, et al.: Using electronic medical records to increase the efficiency of catheter-associated urinary tract infection surveillance for National Health and Safety Network reporting. Am J Infect Control. 2014, 42:33-36. 10.1016/j.ajic.2013.12.005
  7. Woeltje KF, McMullen KM, Butler AM, Goris AJ, Doherty JA: Electronic surveillance for healthcare-associated central line-associated bloodstream infections outside the intensive care unit. Infect Control Hosp Epidemiol. 2011, 32:1086-1090. 10.1086/662181
  8. de Bruin JS, Adlassnig KP, Blacky A, Mandl H, Fehre K, Koller W: Effectiveness of an automated surveillance system for intensive care unit-acquired infections. J Am Med Inform Assoc. 2013, 20:369-372. 10.1136/amiajnl-2012-000898
  9. Pepin CS, Thom KA, Sorkin JD, et al.: Risk factors for central-line-associated bloodstream infections: a focus on comorbid conditions. Infect Control Hosp. Epidemiol, 2015:479-481. 10.1017/ice.2014.81
  10. Lo E, Nicolle LE, Coffin SE, et al.: Strategies to prevent catheter-associated urinary tract infections in acute care hospitals: 2014 update. Infect Control Hosp Epidemiol. 2014, 35:464-479. 10.1086/675718
  11. Edmond K, Zaidi A: New approaches to preventing, diagnosing, and treating neonatal sepsis. PLoS Med. 2010, 7: e1000213. 10.1371/journal.pmed.1000213
  12. Monaghan A: Detecting and managing deterioration in children. Paediatr Nurs. 2005, 17:32-35. 10.7748/paed.17.1.32.s27
  13. Akre M, Finkelstein M, Erickson M, Liu M, Vanderbilt L, Billman G: Sensitivity of the pediatric early warning score to identify patient deterioration. Pediatrics. 2010, 125:763-769. 10.1542/peds.2009-0338
  14. Agulnik A, Mora Robles LN, Forbes PW, et al.: Improved outcomes after successful implementation of a pediatric early warning system (PEWS) in a resource-limited pediatric oncology hospital. Cancer. Cancer, 2017:2965-2974. 10.1002/cncr.30664
  15. Demmel KM, Williams L, Flesch L: Implementation of the pediatric early warning scoring system on a pediatric hematology/oncology unit. J Pediatr Oncol Nurs. 2010, 27:229-240. 10.1177/1043454209358410
  16. Parshuram CS, Duncan HP, Joffe AR, et al.: Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children. Crit Care. 2011, 15:184. 10.1186/cc10337
  17. Torres A, Milov DE, Melendez D, Negron J, Zhao JJ, Lawless ST: A new approach to alarm management: mitigating failure-prone systems. J Hosp Adm. 2014, 3:79. 10.5430/jha.v3n6p79
  18. Griffin MP, Moorman JR: Toward the early diagnosis of neonatal sepsis and sepsis-like illness using novel heart rate analysis. Pediatrics. 2001, 107:97-104. 10.1542/peds.107.1.97
  19. Griffin MP, Lake DE, Moorman JR: Heart rate characteristics and laboratory tests in neonatal sepsis. Pediatrics. 2005, 115:937-941. 10.1542/peds.2004-1393
  20. Griffin MP, Lake DE, Bissonette EA, Harrell FE, Jr., O'Shea TM, Moorman JR: Heart rate characteristics: novel physiomarkers to predict neonatal infection and death. Pediatrics. 2005, 116:1070-1074. 10.1542/peds.2004-2461
  21. Moorman JR, Carlo WA, Kattwinkel J, et al.: Mortality reduction by heart rate characteristic monitoring in very low birth weight neonates: a randomized trial. J Pediatr. 2011, 159:900-906. 10.1016/j.jpeds.2011.06.044
  22. Fairchild KD: Predictive monitoring for early detection of sepsis in neonatal ICU patients. Curr Opin Pediatr. 2013, 25:172-179. 10.1097/MOP.0b013e32835e8fe6
  23. Lawless ST: Crying wolf: false alarms in a pediatric intensive care unit. Crit Care Med. 1994, 22:981-985. 10.1097/00003246-199406000-00017
  24. de Man FR, Greuters S, Boer C, Veerman DP, Loer SA: Intra-operative monitoring-many alarms with minor impact. Anaesthesia. 2013, 68:804-810. 10.1111/anae.12289
  25. Snyders RE, Goris AJ, Gase KA, Leone CL, Doherty JA, Woeltje KF: Increasing the reliability of fully automated surveillance for central line-associated bloodstream infections. Infect Control Hosp Epidemiol. 2015, 36:1396-1400. 10.1017/ice.2015.199
  26. Venable A, Dissanaike S: Is automated electronic surveillance for healthcare-associated infections accurate in the burn unit?. J Burn Care Res. 2013, 34:591-597. 10.1097/BCR.0b013e3182a2aa0f


Figures 2-3 provide detailed information on the customized rules that trigger a CLABSI, CAUTI, neonatal sepsis, or PEWS alert and the specific protocol response. 

Original article

The Impact of Automated Electronic Surveillance of Electronic Medical Records on Pediatric Inpatient Care

Author Information

Jais Emmanuel Corresponding Author

Pediatrics, University of Central Florida, Orlando, USA

Adalberto Torres

Pediatrics, Nemours Children's Hospital, Orlando, USA

Ethics Statement and Conflict of Interest Disclosures

Human subjects: Consent was obtained by all participants in this study. Caroline Schierle issued approval 891248-1. Please note that Nemours IRB 2 has taken the following action on IRBNet:

Project Title: [891248-1] The Impact of Automated Electronic Surveillance of the Electronic Medical Record on Pediatric Inpatient Care
Principal Investigator: Adalberto Torres, MD

Submission Type: New Project
Date Submitted: May 20, 2016
Effective Date: June 8, 2016
Review Type: Expedited Review

Should you have any questions you may contact Caroline Schierle at [email protected]

Thank you,
The IRBNet Support Team. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.


This research project was completed as part of the FIRE Module at the University of Central Florida, College of Medicine. We also extend our sincere gratitude to Dr. Madhuradhar Chegondi for his thoughtful review of the manuscript.

Original article

The Impact of Automated Electronic Surveillance of Electronic Medical Records on Pediatric Inpatient Care

Figures etc.


Scholary Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.