Prediction of Noninvasive Ventilation Failure in COVID-19 Patients: When Shall We Stop?

Introduction: In coronavirus disease 2019 (COVID-19), there are no tools available for the difficult task of recognizing which patients do not benefit from maintaining respiratory support, such as noninvasive ventilation (NIV). Identifying treatment failure is crucial to provide the best possible care and optimizing resources. Therefore, this study aimed to build a model that predicts NIV failure in patients who did not progress to invasive mechanical ventilation (IMV). Methods: This retrospective observational study included critical COVID-19 patients treated with NIV who did not progress to IMV. Patients were admitted to a Portuguese tertiary hospital between October 1, 2020, and March 31, 2021. The outcome of interest was NIV failure, defined as COVID-19-related in-hospital death. A binary logistic regression was performed, where the outcome (mortality) was the dependent variable. Using the independent variables of the logistic regression a decision-tree classification model was implemented. Results: The study sample, composed of 103 patients, had a mean age of 66.3 years (SD=14.9), of which 38.8% (40 patients) were female. Most patients (82.5%) were autonomous for basic activities of daily living. The prediction model was statistically significant with an area under the curve of 0.994 and a precision of 0.950. Higher age, a higher number of days with increases in the fraction of inspired oxygen (FiO2), a higher number of days of maximum expiratory positive airway pressure, a lower number of days on NIV, and a lower number of days from disease onset to hospital admission were, with statistical significance, associated with increased odds of death. A decision-tree classification model was then obtained to achieve the best combination of variables to predict the outcome of interest. Conclusions: This study presents a model to predict death in COVID-19 patients treated with NIV in patients who did not progress to IMV, based on easily applicable variables that mainly reflect patients’ evolution during hospitalization. Along with the decision-tree classification model, these original findings may help clinicians define the best therapeutical approach to each patient, prioritizing life-comforting measures when adequate, and optimizing resources, which is crucial within limited or overloaded healthcare systems. Further research is needed on this subject of treatment failure, not only to understand if these results are reproducible but also, in a broader sense, helping to fill this gap in modern medicine guidelines.


Introduction
Coronavirus disease 2019 (COVID-19) has affected millions of people worldwide [1]. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) may still spread exponentially, especially when preventive measures are not fully adopted. Although most infected individuals do not require hospitalization [2], we continue to observe a large group of patients who need advanced respiratory support [3]. The dramatic lack of ventilators felt worldwide during the COVID-19 pandemic confirmed that, in a setting of limited or overloaded healthcare systems, optimizing the use of medical resources is paramount [4,5].
In the first months of the pandemic, the idea of early intubation was widespread, based on previous experience with acute respiratory distress syndrome (ARDS) [6,7], which resulted in overstrained intensive care units (ICU). Over time, other forms of respiratory support that were more available and less demanding than invasive mechanical ventilation (IMV), such as high-flow nasal oxygen (HFNO) and noninvasive ventilation (NIV), became advocated [8,9]. 1 1 1 1 1 1 NIV efficacy is clearly validated in various diseases that cause respiratory failure, such as cardiogenic pulmonary edema or exacerbation of chronic obstructive pulmonary disease (COPD) [10]. Although controversial, the use of NIV has also been supported in cases of ARDS and community-acquired pneumonia, by improving clinically relevant outcomes such as progression to intubation [11,12].
The use of NIV in COVID-19 was initially limited by a lack of information regarding its benefit and concerns about aerosolization [13]. However, when it started to be used due to necessity and lack of better alternatives, studies showed an improvement in mortality rates, hospitalization time, and a decrease in the severity of symptoms related to COVID-19 when comparing NIV against IMV [14]. Moreover, no difference in mortality was found between patients who started NIV and later progressed to IMV and those who started on IMV [9]. A complementary study found that patients who commenced continuous positive airway pressure (CPAP) therapy earlier had lower mortality than patients commencing CPAP later [15]. As more research emerged, the usage of helmet NIV instead of HFNO was linked to a reduction in the number of patients who needed IMV [16].
Due to the success of those strategies, a sequential model of care (conventional oxygen, HFNO, NIV, and finally IMV) became recommended and standardized [17]. Optimal criteria to step up, however, remained difficult to accomplish in most hospitals due to the high influx of patients, and lack of equipment or personnel. It became mandatory to allocate resources to patients that could benefit the most of them, meaning limitation of care in certain cases and recognition of treatment failure in others, especially in those not candidates for IMV. Very few studies have addressed the risk of NIV failure in COVID-19 and those that had, mainly focus on analytical data, comorbidities, age, and some clinical parameters [18][19][20][21][22][23][24]. To our knowledge, no work postulates a model to predict NIV failure in COVID-19, which includes the evolution of NIV parameters, such as the fraction of inspired oxygen (FiO 2 ) and end positive airway pressure (EPAP), which could help guide clinical decisions.
In modern medicine, despite the high frequency of treatment failure in many scenarios, current clinical guidelines are mainly oriented toward adding treatments and rarely provide support to decisions involving the suspension of a therapy that has apparently become futile in a certain patient. Sadly, that is also happening in COVID-19, where studies and orientations on when to change focus and reinforce lifecomforting measures are scarce [25,26]. As sensible practitioners, we need to keep in mind that the urge to prolong a patient's life may negatively impact its quality, particularly in the last moments of it. Insisting, perhaps obsessively, on NIV when the patient will ultimately die, will lead to dysthanasia and, in a limited resource setting, preclude other patients from receiving beneficial therapies.
The main purpose of this study is to elaborate a model that, in a reliable and timely manner, predicts NIV failure in patients who did not progress to IMV. These results may help to assess the futility of therapy, avoiding life-prolonging measures of no benefit to the patient. After all, we were taught that first, we shall do no harm.
This article was previously posted on the Research Square preprint server on February 9, 2022 [27].

Participants' selection and formation of groups for analysis
The medical records of all patients hospitalized for COVID-19 during the study period were consulted to exclusively select patients undergoing NIV during hospitalization. Subsequently, compliance with inclusion and exclusion criteria was verified for each case. Patients treated with IMV were excluded so that the group of surviving patients was unequivocally a therapeutically successful group of NIV and not of other supplementary strategies. Likewise, the group of patients who died, as they were not treated with IMV, unequivocally corresponds to NIV failure and not to the concomitant failure of NIV and IMV.

Participants' data collection
Patients' medical records were used to obtain baseline data on demographic features and comorbidities (age, sex, functional and cognitive status, tabagism and alcohol abuse, obesity, arterial hypertension, diabetes mellitus, dyslipidemia, heart failure, chronic liver disease, chronic kidney disease, previous stroke, immunemediated diseases, active cancer (stages I and II), COPD, asthma, pulmonary embolism, pulmonary hypertension, other respiratory diseases, previous long term oxygen therapy), as well as information regarding the course of the disease and the use of NIV during the hospital stay. Details on CPAP and BPAP modes, namely the values of EPAP and inspiratory positive airway pressure (IPAP), as well as FiO 2 , were collected from the time the patient started NIV until the last day of its utilization. More specifically, for each 24-hour period of NIV, the maximum CPAP/ EPAP value (referred now on as EPAP for simplification) was collected as well as the maximum FiO 2 and IPAP used with that EPAP value. All data were collected and inserted into a database by the medical doctors involved in this work.

Outcome measure
The outcome measure of interest was NIV failure, defined as COVID-19-related in-hospital death.

Statistical analysis
Descriptive statistics were obtained for sociodemographic and clinical variables. The assumption of normality was assessed with the Shapiro-Wilk test. Sociodemographic features with a prevalence higher than 10% and clinical variables were compared between groups of patients (alive vs dead) using the independent samples t-test, the Mann-Whitney, or the chi-square (X 2 ) test in the case of categorical variables.
Data were analyzed with a binary logistic regression model, where the outcome (0 -alive; 1 -dead) was set as the dependent variable. The predictors of interest included age; sex; functional status; the number of days from disease onset to hospital admission (when the onset of symptoms was unknown, date of diagnosis was considered), and NIV features for each patient during hospital stay: number of days on NIV; average values of EPAP and FiO 2 ; maximum values of EPAP and FiO 2 ; the number of days with the maximum EPAP. To reflect the evolution of NIV features, two variables were created: the number of days with an increase in EPAP and the number of days with an increase in FiO 2 . The area under the curve (AUC), sensitivity, specificity, and precision were extracted as performance metrics of this model.
A decision-tree classification model was implemented to further explore the combination of variables that best predicts the outcome of interest. To achieve this, the same variables defined as independent variables in the logistic regression model were entered as features of interest. With the decision-tree classification model, we aim to divide the dataset of alive and non-alive patients into homogeneous groups, based on successive decision boundaries established according to relevant features that minimize the probability of a random element being incorrectly classified (the Gini impurity).
Descriptive statistics and logistic regression analysis were performed using the free and open-source computer software JASP (Jeffreys's Amazing Statistics Program) (Version 0.16). The decision-tree classification model was implemented with scikit-learn (Version 1.0.1) [28]. Statistical significance was defined at the p<.05 level.

Patient characteristics at hospital admission and group differences
One hundred thirty-nine patients treated with NIV between October 1, 2020, and March 31, 2021, were identified, of which 103 met the inclusion and exclusion criteria and were eligible for analysis. Of these, 39 patients (37.9%) died during their hospital stay due to COVID-19 ( Figure 1).

NIV features and evolution during the hospital stay
The median time on NIV was six days (IQR=4.0-9.0 days) without a significant difference between patients who died or survived. Patients who died had significantly higher average values of EPAP, maximum EPAP, number of days on maximum EPAP, average values of FiO 2 , and maximum FiO 2 . The number of days with increases in EPAP was also significantly higher in those who died as well as the number of days with increases in FiO 2 . These and other results are detailed in Table 2.  Increased age, a higher number of days with increases on FiO 2 , a lower number of days on NIV, a higher number of days with EPAP at maximum values, and a lower number of days from disease onset to hospital admission were associated with increased odds of death ( Table 3).

Decision-tree classification model
The model represented in Figure 3 starts with a root node in which all the observations (n total =103; n alive =64; n dead =39) are present. The first decision boundary was based on the average  disease onset and hospitalization, and the total number of days on noninvasive ventilation (NIV). The word "gini" that is present in the figure refers to the Gini coefficient, which ranges from 0 to 1.

Discussion
A mortality rate of 37.9% was observed in the sample of this study. Patients had a mean age of 66.3 years and 61.2% were males. Arterial hypertension was the most common comorbidity (71.8%) and there was a high prevalence of other cardiometabolic comorbidities. Overall, the mortality and patient characteristics are in accordance with other studies of COVID-19 patients treated with NIV [29].
It should come as no surprise that patients who died were significantly older and had a worse functional status [30,31]. These two factors alone can be easily determined on admission and provide a basis for predicting patient response to NIV. Patients who died had a lower number of days on NIV, which may seem surprising at first sight. Revising our data, 71% of those who died were treated for a period of fewer than five days and 50% of patients of them were older than 70 years. Moreover, patients who died had significant differences in several variables that reflect the need for greater ventilatory support: higher average FiO 2 and EPAP, higher maximum FiO 2 and EPAP as well as a higher number of days with increases in FiO 2 and EPAP, and a higher number of days with EPAP on maximum values. This may reflect a rapidly progressive disease course in a large group of patients. Survivors perhaps had a slower progression of the disease, were able to control inflammation, and reach a plateau before their condition started to improve. In COVID-19, the recovery process is usually slow, potentially justifying why survivors had more days under NIV. This explanation also elucidates why patients who died had significantly less time from disease onset to hospital admission, that is, it may reflect a more aggressive inflammatory response, marking a worse clinical trajectory, as has been suggested [32].
A logistic regression model, which was statistically significant with an area under the curve of 0.994, a sensitivity of 0.974, a specificity of 0.969, and a precision of 0.950, was obtained. Based on the variables used in the logistic regression model, a decision tree was developed, which allows the medical team to quickly stratify a patient into a group of similar patients and extrapolate mortality rates. For instance, it is striking that mortality of 93.8% is expected in a group of patients with an average FiO 2 greater than 72.6% and an average EPAP greater than 9.9. We can further stratify the risk by continuing down the decision tree, but such a high mortality rate will necessarily promote a clinical discussion about the patient's best interest and the most adequate therapeutic approach.
Clinicians have progressively more data and tools available to predict a patient's prognosis. For example, Liu et al. have found several parameters in their study that also help predict NIV failure and were able to develop an online calculator to assist decision-making [24]. Works like this have been mainly limited to predicting NIV failure based on analytical parameters and comorbidities. While these kinds of predictors are undoubtedly important, our model was intentionally built upon variables that reflect the evolution throughout the hospital stay, making this work innovative, useful to clinical practice, and helpful in filling a gap in the literature. Overall, works of this kind may become crucial in settings of scarce medical resources, be it from a chronic lack of resources or from unexpected demand. Accurate assessment of a patient's prognosis allows for better employment of resources, treating those who may benefit from the continuation of advanced therapeutic measures, and giving more thought and time to life-comforting measures which are often regrettably omitted. It must be stated, however, that the results provided should not override clinical sense or wisdom.

Limitations
This study has a retrospective observational design, which does not allow to rule out residual confounding, and has an inferior level of evidence. Data were collected from medical records, which are prone to human error and are not always complete or updated. Most patients initiated NIV when HFNO failed to maintain a partial pressure of oxygen (PaO 2 )>60 mmHg with a FiO 2 >70%. Progression to IMV was based on clinical judgment, case by case, and was not always consistent throughout the pandemic, which may have interfered with sample selection. All patients received the standard of care with dexamethasone, with none receiving other agents such as remdesivir, tocilizumab, or antibiotics. These practices can differ from other centers, making it hard to extrapolate our findings. Similarly, centers with access to distinct models of ventilators that perform differently may experience non-identical results. We may also hypothesize that a sample of fully vaccinated individuals could have shown different mortality rates and patterns. Finally, the number of patients in our center made it impossible to have a separate cohort for cross-validation of the model, a technique to lessen the risk of overfitting, which ideally is done with samples from other centers.

Conclusions
This retrospective observational study, despite its limitations, presents a model to predict NIV failure, defined as COVID-19-related in-hospital death, in patients who did not progress to IMV, with an area under the curve of 0.994 and a precision of 0.950, based on easily applicable variables that also reflect evolution during hospital stay (i.e. sex; age; functional status; days from disease onset to hospital admission; days on NIV; average FiO 2 ; days with FiO 2 increases; average EPAP; days with maximum EPAP), that was further explored with the implementation of a decision-tree classification model, allowing for the best combination of variables to predict mortality. These findings add to other studies on NIV use during COVID-19 and may help clinicians define the best therapeutical approach to each patient, prioritizing life-comforting measures when adequate and optimizing resources, which is crucial within limited or overloaded healthcare systems. Further research is needed on this subject of treatment failure, not only to understand if these results are reproducible but also, in a broader sense, helping to fill this gap in modern medicine guidelines.

Additional Information Disclosures
Human subjects: Consent was obtained or waived by all participants in this study. Ethical Committee for Health of the Hospital of Braga, Portugal issued approval 61_2021. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.