A Systemic Review on the Diagnostic Accuracy of Point-of-Care Ultrasound in Patients With Undifferentiated Shock in the Emergency Department

Early identification of the shock type and correct diagnosis is associated with better outcomes. Previous studies have suggested that point-of-care ultrasound (POCUS) increases the diagnostic accuracy of patients in undifferentiated shock. However, a complete overview of the diagnostic accuracy of POCUS and the related treatment changes when compared to standard care is still limited. Our objective was to compare POCUS against standard practice regarding the diagnostic accuracy and specific therapeutic management changes (fluid volume administration and vasopressor use) in patients with undifferentiated shock in the emergency department (ED). We conducted a systematic review in concordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. A systematic search was performed using Embase, PubMed, Cochrane Central Register for Controlled Trials, and clinicaltrials.gov. Two physicians independently selected the articles and assessed the quality of the studies independently with the Quadas-2 tool. All included studies used POCUS in adult patients in undifferentiated shock and described diagnostic accuracy or specific therapeutic management changes (fluid volume administration or vasopressor use) and compared this to standard care. The primary outcome was diagnostic accuracy. Secondary outcomes were the amount of fluid administered and vasopressor use in the ED. Only articles published after 1996 were included. There were 10,805 articles found of which 6 articles were included. Four out of six studies reported diagnostic accuracy, three reported on fluid administration and vasopressors. We found that the diagnostic accuracy improved through the use of POCUS when compared to the standard care group, increasing overall diagnostic accuracy from 45-60% to 80-89% when combined with clinical information. There was no significant difference in fluid administration or vasopressor use between the groups. In our systematic review, we found that the use of POCUS in patients that presented with undifferentiated shock in the ED improved the diagnostic accuracy of the shock type and final diagnosis. POCUS resulted in no changes in fluid administration or vasopressor use when compared to standard care. However, the results should be interpreted within the limitations of some of the studies that were included in the review.


Introduction And Background
Shock represents 0.4% to 1.3% of all emergency department (ED) presentations and up to one-third of all intensive care unit (ICU) admissions [1][2][3]. It is associated with high morbidity and in-hospital mortality of up to 48% [1,[4][5][6]. Early recognition by the use of shock alerting systems has been shown to decrease mortality. Therefore, it seems reasonable to assume that rapid and accurate detection of the cause of shock has the potential to improve patient outcomes further [7].
While physical examination alone is unreliable to accurately determine the correct cause of hypotension [8,9], evidence suggests that point-of-care ultrasound (POCUS) has the potential to obtain good diagnostic accuracy in patients with hypotension in the ED [10,11]. The use of POCUS has gained widespread acceptance in recent years and is progressively becoming the standard of care in the evaluation of critically ill patients [12]. Pneumothorax, pericardial tamponade, fluid hypovolemia, left ventricular failure, and right ventricular strain can all be detected by POCUS [13,14]. As a result, many approaches to optimize and organize the use of POCUS in shock have been described [4,[15][16][17][18][19][20][21][22]. However, there is a lack of overview of the diagnostic accuracy of POCUS in undifferentiated shock patients that present to the ED. Therefore, the objective of this systematic review is to compare POCUS against standard practice regarding the diagnostic accuracy and specific therapeutic management changes (fluid volume administration and vasopressor use) in patients with undifferentiated shock in the ED.

Literature Review
The reporting of the present review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [23].

Search Strategy
A search strategy was constructed with medical subject headings and keywords focusing on "POCUS," "shock," and "emergency department" (Appendices: Table 4). An initial search was conducted on September 14, 2015, and a follow-up search was conducted on November 27, 2020. The following databases were searched: EMBASE through OVID (January 1, 1996 to November 27, 2020), PubMed (January 1, 1996 to November 27, 2020), and Cochrane Central Register for Controlled Trials (May 24, 2019). We searched the reference lists of appropriate studies, most relevant guidelines and consulted the clinicaltrials.gov registry (April 12, 2020), after which we contacted the authors of all ongoing trials on this topic for preliminary results.

Inclusion and Exclusion Criteria
We included studies that used POCUS in adult patients in undifferentiated shock, described diagnostic accuracy or specific therapeutic management changes (fluid volume administration or vasopressor use), and compared this to standard care. The following studies were excluded: (i) studies conducted outside of an ED setting, (ii) differentiated shock (e.g. trauma, septic shock), (iii) studies that included pregnant patients or patients <18 years old, (iv) studies that did not use ultrasound as a diagnostic tool to determine or exclude the cause of shock in a clinical setting, (v) studies that examined trans-esophageal ultrasound, and (vi) studies lacking a control group where ultrasound was (initially) not performed.

Data Collection and Processing
The primary outcome was the difference in diagnostic accuracy between the POCUS and standard-care groups. The diagnostic accuracy was defined as the percentage of occurrences of the correct diagnosis with or without POCUS. The correct diagnosis was defined as the gold standard that was used in the article (e.g., final diagnosis at discharge). Secondary outcomes were differences in IV fluid administration (total mL of fluids administered during the ED stay) and vasopressor use in the ED between the POCUS and standard care groups. For studies to be eligible, data related to at least one of these outcomes needs to be available for extraction. The search was limited to studies published in 1996 or later. Because of the advancements in POCUS in the last 25 years, we did not expect any relevant studies to have been conducted before 1996. No search limits were placed on the language of publication. Six authors selected the articles and extracted the data. Each step in the selection and data extraction process was done independently by two of these authors ( Figure 1). The reviewers were not blinded to the authorship, journal, or year. Disagreements were resolved by consensus-based discussion, and when necessary, a third author adjudicated any disagreements. We extracted data regarding study design, study location, sample size, characteristics of participants, intervention, control group, reference standard, and outcome measures. Two authors independently assessed the quality of the studies with the Quadas-2 tool [24] for determining the risk of bias ( Table 1).

FIGURE 1: Preferred Reporting Items for Systematic Reviews and Metaanalyses (PRISMA) flow diagram.
PICO: Population, Intervention, Comparison, Outcomes of study as mentioned in methods section [23].   [28,30]. A post-hoc analysis of the 2018 prospective study was published in 2019, dividing patient groups into cardiogenic or non-cardiogenic shock types [24].

Outcome Measures and Data Analysis
A meta-analysis was not feasible because of the heterogeneity between the included studies. Therefore, study results were directly compared and critically appraised against the primary and secondary outcomes of the study.
The removal of duplicate studies resulted in 10,805 unique citations. After excluding 10,714 articles by screening the titles and abstracts, 91 articles were analyzed in more depth to assess their suitability. A further 85 articles that did not meet the inclusion criteria were therefore excluded (Appendices: Table 5). A flow diagram of the literature search is presented in Figure 1.

Study Characteristics
A total of six studies met the inclusion criteria [25][26][27][28][29][30]. The number of patients included varied from 100 to 270 patients per study, with a total of 852 patients in all studies together. Two studies were original randomized controlled trials (RCT) [25,28], and two studies had a prospective before-after design [26,27].
One study [30] was a post hoc analysis of a prospective trial [28], which is also included in this review. One study was a prospective explorative study [29]. All studies were published in English. Two studies were conducted in the USA [25,26], one study in both Canada and South Africa [28,30], one study in Turkey [27], and one study in India [29]. There was a moderate degree of variability in the quality of the included studies ( Table 1). Three studies were judged to have a low risk of bias [25,28,30]. The three others were considered moderate to high-risk in one or more domains [26,27,29]. An overview of the study characteristics is presented in Table 2.  All studies took place in the ED [25][26][27][28][29]. Four of those were single-center studies, and Atkinson et al. [28,30] was a multicenter study. One study had a control group where no ultrasound was performed [28,30], and one study had a control group that received an ultrasound at a later stage after collecting the initial data [25]. The three other studies collected pre-and post-ultrasound data in the same patient group [26,27,29]. The mean age of the included patients varied from 52 to 63 years. The results of the included studies are summarized in Table 3.

Analysis of Outcomes
Diagnostic accuracy: Four out of six studies reported the diagnostic accuracy of a POCUS protocol for shock etiology in patients with undifferentiated shock in the ED and compared it to the diagnostic accuracy of a physician who did not use ultrasound (initially) as part of the workup [25][26][27]29]. Jones [26,29]. An overview of the diagnostic accuracy in the No POCUS versus the POCUS group is presented in Figure 2.

FIGURE 2: Diagnostic Accuracy No POCUS vs POCUS.
Studies in figure from left to right: Jones [25], Javali [29], Sasmaz [27], Shookohi [26]. Three studies marked the difference in diagnostic accuracy between the two groups as significant (*) [25][26][27]. Javali [29] did not report significance but found a Cohen's kappa coefficient (#) of 0.89, correlating with an almost perfect agreement with the final diagnosis. The diagnostic accuracy was defined as the percentage of occurrence of the correct diagnosis with or without POCUS.
Jones et al. found that using POCUS in patients with undifferentiated shock, the diagnostic accuracy was 80%, compared to 50% in the control group that received no ultrasound at that point [25]. The 30% difference was significant (95% CI, 16-42%). The control group also received a POCUS exam after the first round of data collection, resulting in an increase in correct diagnoses from 50% to 78%. Similarly, Sasmaz et al. also found that diagnostic accuracy significantly increased from 61% before POCUS to 85% after POCUS [27].
Javali et al. reported that the accuracy in diagnosing the type of shock increased from 45% to 89% when adding POCUS by a trained emergency physician to the clinical information alone to make the diagnosis (an overall kappa correlation of 0.89) [29]. Shokoohi et al. [26] found a significant increase in patients with a definitive diagnosis for the type of shock from 0.8% before to 12.7% after POCUS was performed by an ultrasound-trained attending physician (Diff.: 11.9%; 95% CI, 5.6-18.1%). When they compared the final diagnosis with the leading POCUS diagnosis, it matched the discharge diagnosis in 86% of the cases (Cohen κ of 0.80; 95% CI, 0.73-0.88).
Change of management: Four out of six studies reported on management changes [26][27][28]30]. Of these studies, however, only Atkinson et al. specified the difference in mean fluid volume administration and vasopressor use in the ED, both in the original study and its post-hoc analysis [28,30].
Fluid administration: Three out of six studies reported changes in ED fluid administration in patients after the use of POCUS [26,28,30]. One study mentioned changes in fluid regimens, yet did not report on statistical significance [26]. Atkinson et al. found no significant difference in the mean fluid volume administered during the first four hours between the POCUS and the standard care group [28]. A subgroup analysis that looked specifically at patients in cardiogenic shock also showed no significant difference in the mean amount of fluid administered between the POCUS and standard care groups [30].
Vasopressors: Three out of six studies reported on the use of vasoactive agents [26,28,30]. Atkinson et al. saw no significant difference in vasopressor usage, both in the original paper and in the post-hoc subgroup analysis [28,30]. Shokoohi et al. did report increased use of vasopressors after POCUS, ranging from 25 to 36%. This change, however, was not reported to be statistically significant [26].

Discussion
Evidence from the six available studies suggests that the use of POCUS in patients who presented to the ED with undifferentiated shock resulted in an increase in diagnostic accuracy of the shock type and final diagnosis, as well as a reduction in viable differential diagnoses and improved diagnostic confidence. However, we found no evidence of a change in fluid volume administration or use of vasopressors between the two groups.
In our review, diagnostic accuracy improved significantly from 45-60% to 80-89% when combined with clinical information. These results correlate well with those from other studies [31], including a recent systematic review and meta-analysis assessing the diagnostic accuracy of the RUSH exam for shock type in undifferentiated shock in the ED [11]. This study reported positive likelihood ratios (LR+) that ranged from 8.2 to 40.5, yielding clinically useful information, especially when ruling in a shock subtype. The positive likelihood ratios were highest for obstructive and lowest for mixed-etiology types of shock. A recent study published shortly after our search found that POCUS, when compared to standard examination, increased the accuracy of the cause of shock and altered the proposed treatment [32].
This high diagnostic accuracy was also expressed by the high concordance values between the diagnosis post POCUS and the final diagnosis in three of our studies, with overall Cohen's kappa coefficients ranging between 0.80 and 0.89 [26,27,29]. These results were supported by previous studies where good to excellent concordance was found among the POCUS diagnosis, type of shock, and final diagnosis with inter-rater reliability Kappa coefficient values ranging between 0.70 and 0.97 [10,11,[33][34][35][36].
In contrast to the other studies included in this review, accuracy numbers appeared low in Shokoohi et al., who had a strict protocol for diagnosing the type of shock where a diagnosis was termed definitive when a single diagnosis remained on the differential diagnosis sheet [26]. This appears to explain the lower accuracy finding of 0.8% before POCUS and 12.7% after POCUS introduction. However, when the initial leading POCUS diagnosis was compared to the final diagnosis, the diagnostic accuracy increased to 86%, which is comparable to the results found in the other studies [25,27,29].
Apart from the observed improvement in diagnostic accuracy, Jones et al. also found that the use of POCUS in patients with undifferentiated shock resulted in fewer viable diagnostic etiologies, with a median number of 4 in the POCUS group versus 9 in the control group (p<0.01) [25]. Furthermore, other studies found that POCUS led to higher physicians' certainty regarding the diagnosis and cause of vital sign abnormalities in sepsis, chest pain, dyspnea, and symptomatic hypotension [26,33,37]. A similar increase in diagnostic confidence has also been reported in the ICU setting [38].
Therapeutic management changes were reported in four of the six selected studies [26][27][28]30]. Two studies reported treatment changes in 25% to 50% of cases [26,27]. However, these changes were not specified and may not have been significant or beneficial to the patient. Only Atkinson et al. investigated IV fluid volume administration and inotrope use in patients with undifferentiated shock in the ED and found no significant difference between the POCUS and standard care groups [28]. The same study's post hoc analysis also failed to notice any treatment differences within both the cardiogenic and non-cardiogenic shock types when POCUS was compared against standard care [30]. These results are in contrast with findings from other ED studies that showed a change in treatment in patients that presented with sepsis [37] and hypotension [31]. In addition, many ICU studies also noticed significant treatment alterations brought on by the use of POCUS in patients that presented with shock [39], sepsis [38,40], and undifferentiated hypotension [41].
A plausible explanation for the lack of treatment changes in Atkinson et al. [28] could be the limited number of patients with POCUS-sensitive diagnoses. More than half of the patients included in this study were diagnosed with sepsis, which can lead to variable findings from hyper-to hypodynamic left ventricular function, variable inferior vena cava size and collapsibility, and findings such as ascites and pleural effusions. These findings make it difficult to make a correct diagnosis early. Other possible explanations for the lack of difference between the groups within this study were that comprehensive laboratory and advanced imaging resources were used in both groups, the high skill of emergency physicians and thus level of care, and that the definition of undifferentiated shock is still not accurate enough. An unclear definition could have led to the exclusion of patients before a final diagnosis was made. These patients could possibly have had the benefit of POCUS and could have contained POCUS-sensitive diagnoses. The hypothesis that POCUS-sensitive diagnoses occur rarely but can change treatment is supported by the findings of Shokoohi et al., who reported that there was a drastic change in management in only 5.1% of the cases in the population of ED patients with undifferentiated shock [42].
A recent study by Mosier et al. suggested that POCUS could lead to a delay in treatment and found higher mortality in the POCUS group [39]. However, this study has been reported to contain potential methodological weaknesses and, therefore, should be interpreted with caution when looking at the effect of POCUS in shock patients in the ED. In a letter to the editor, Amini et al. commented that the study had an unclear definition of POCUS, with inappropriate data inclusion and collection, and overstated conclusions. They reported that the study included educational studies that were not used for medical decision-making or related to interventions, thus introducing a bias [43]. In the studies included in this review, two studies reported on discordant diagnoses and potential harm. Both studies reported no indication of harm in the POCUS group compared to standard care, and no ultrasound findings were reported to lead to further unnecessary invasive procedures [25,28].

Clinical Implications and Future Perspectives
Since all studies were conducted in advanced care settings where other imaging modalities are widely available, the added value of POCUS in increasing diagnostic accuracy in some studies could be underestimated when compared to a medium-to low-resource setting. However, the same could be expected for treatment changes, but a sub-analysis comparing the South-African cohort with the Canadian cohort showed no difference between the POCUS and standard care group [28]. Further studies could potentially focus on diagnostic and therapeutic changes in medium-to low-resource settings. Another possible advantage is that POCUS can provide a more accurate diagnosis early and diminish the number of viable diagnoses. This could lead to less advanced imaging and examinations and, thereby, lower healthcare costs and time spent in the ED.
The most substantial added value of POCUS, therefore, seems to lie in its potential to increase both the diagnostic accuracy of the final diagnosis and shock type. In specific cases, POCUS seems to have the potential to shorten the time to a diagnosis, tailor and accelerate the workup and start of correct treatment, and prevent adverse outcomes. Future research on POCUS on patients in shock in advanced healthcare systems could focus on specific populations, patient-and setting tailored use of POCUS, and outcomes such as time-to-correct diagnosis, correct classification of shock type and diagnosis, and time to the next diagnostic examination. As a consequence, the time to appropriate treatment, treatment effects, and prevention of errors by using POCUS in high-risk decisions and patients could be influenced. Also, repetitive examinations during fluid and inotrope administration could positively guide treatment and outcomes in specific cases. Patient outcome benefits such as mortality and morbidity are of interest in future studies, and differences are more likely to be found in specific cases and high-risk settings, in contrast to expecting that one protocol would be useful for the entire population of undifferentiated shock patients. This is supported by reports of selected cases where it does seem likely that ultrasound can rapidly change diagnosis and treatment, lowering mortality and morbidity in shock patients, as described in a case series by Shokoohi et al. [42]. In a study by Gaspari et al., it was found that patients with pulseless electrical activity during cardiac arrest with organized activity, visualized on ultrasound, demonstrated an increased survival to hospital admission when started on adrenergic agents during resuscitation, compared to the group with disorganized activity [44]. Similar findings have been reported by Atkinson et al., who reported that patients with cardiac activity on POCUS had longer resuscitation times, were more likely to achieve ROSC, and had better survival to hospital discharge when compared to those without cardiac activity on POCUS [45]. Although these patients are at the far end of the shock spectrum, these findings support the hypothesis that ultrasoundguided therapeutic decisions might reduce mortality in patients in shock. However, looking at the undifferentiated shock in the ED, Atkinson et al. found no difference in the 30-day survival rate [28], nor in resuscitation outcome markers such as lactate, bicarbonate, Modified Early Warning Score (MEWS), or Shock Index (SI) [30].
Studying applications of POCUS within specific groups within the undifferentiated shock population can guide us to a tailored and time-effective approach for each scenario and care of each patient in undifferentiated shock. We, therefore, support an etiology-based prioritization of POCUS views as proposed in the IFEM consensus statement ShoC [22].

Limitations
Besides its strengths, this systematic review also has several limiting factors. We do not think that a selection or retrieval bias affected the results. Our study did not include specific populations such as children or pregnant patients and therefore may not accurately represent these patients. We did observe significant heterogeneity in reporting methods and a marked difference in the POCUS windows used between the included studies. Although there was a difference in POCUS windows, the included studies all included cardiac, IVC, aorta, and peritoneal views, and five out of six studies included thoracic views to assess for pneumothorax, pleural fluid, and pulmonary edema [26][27][28][29][30]. Due to the heterogeneity of the studies and varying definitions of diagnostic accuracy, we concluded that a meta-analysis would not add any value. Also, only two out of the six studies had sonographers who were not directly involved in patient care [26,29], and none of these provided quality assurances by reviewing the ultrasound images by a blinded assessor. The quality of reporting for the included studies was modest, with three of the six studies having a low risk of bias [25,28,30], while the other three studies had a moderate to high risk [26,27,29]. Moreover, two out of the six studies did not provide sample size calculations [26,27]. Therefore, the findings of this review might be influenced by the lack of power of the individual studies to detect a difference in the outcomes of interest. A publication bias could be a possible concern because studies, where no POCUS benefit was found, may not have been published.

Conclusions
This systematic review demonstrates that POCUS improved the diagnostic accuracy of the underlying shock type in patients presenting with undifferentiated shock in the ED, compared to when clinical assessment without POCUS was used. Furthermore, POCUS use also improved the diagnostic accuracy of the underlying cause(s) of the shock type. POCUS use made no difference in intravenous fluid therapy or vasopressor management of patients presenting with undifferentiated shock. A subgroup analysis that looked specifically at patients in cardiogenic shock and non-cardiogenic shock also showed no significant difference in the mean amount of fluid administered between the POCUS and control groups. Since all studies were conducted in advanced care settings where other imaging modalities are widely available, the added value of POCUS could possibly be underestimated in a medium-to low-resource setting. These results should be interpreted within the scope of the limitations of the six studies included in the systematic review. Search Limits: -Tab 'search manager'. (((shock NOT ("shock wave" OR "shock waves")) OR circulatory NEXT failure* OR circulatory NEXT collaps* OR circulation NEXT collaps* OR collapsed NEXT circulation* OR collapse NEXT of NEXT circulation* OR critical NEXT ill* OR critically NEXT ill* OR hypotens* OR hypo-tens* OR (low AND (blood NEXTpressure* OR bloodpressure*)) OR hemodynamically NEXT unstab* OR hemo-dynamically NEXT unstab* OR haemodynamically NEXT unstab* OR haemo-dynamically NEXT unstab* OR hemodynamic NEXT instab* OR hemo-dynamic NEXT instab* OR haemodynamic NEXT instab* OR haemo-dynamic NEXTinstab* OR hemodynamic NEXT unstab* OR hemodynamic NEXT unstab* OR haemodynamic NEXT unstab* OR haemo-dynamic NEXT unstab* OR hypovolem* OR hypo-volem* OR hypovolaem* OR hypo-volaem* OR septic* OR sepsis*) AND (ultraso* OR ultra-so* OR echo OR echos OR echo's OR echoc* OR echo-c* OR echog* OR echo-g* OR echoscop* OR echo-scop* OR echoso* OR echo-so* OR Gezocht in: 'search manager'. Ingeperkt op: -(((shock NOT ("shock wave" OR "shock waves")) OR circulatory NEXT failure* OR circulatory NEXT collaps* ORcirculation NEXT collaps* OR collapsed NEXT circulation* OR collapse NEXT of NEXT circulation* OR critical NEXT ill* OR critically NEXT ill* OR hypotens* OR hypo-tens* OR (low AND (blood NEXT pressure* OR bloodpressure*)) OR hemodynamically NEXT unstab* OR hemo-dynamically NEXT unstab* OR haemodynamically NEXT unstab* OR haemo-dynamically NEXT

Conflicts of interest:
In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.