Using Machine Learning Technique to Predict the Most Reliable Diagnostic Finding for Foreign Body Aspiration in Children: Symptoms, Chest X-ray, or Auscultation?

Foreign body aspiration (FBA) is one of the most critical and life-threatening pediatric emergency situations. Prompt diagnosis in these cases is very important as they are associated with high mortality among children. When diagnosing FBA, symptoms of the patient, auscultation findings, and chest X-ray findings are usually evaluated. In this study, we conducted a retrospective analysis of all the cases involving suspicion of FBA in children under the age of 18 years who were hospitalized in the Department of Pediatric Surgery at Denizli Pamukkale University Hospital, Turkey from January 2005 to September 2020. Instead of traditional statistical methods, we used machine learning techniques such as random forest and logistic regression to determine which finding was diagnostically the most reliable. The variables included in the analysis that were considered to be significant were as follows: symptoms, auscultation findings, chest X-ray findings, patient gender, age, location of the foreign body, and the time of admission. For the purpose of this study, we developed four different models. Model 1 included gender, age, time of admission, location, and symptoms as variables; the correct classification rate of the model was found to be 82.3%. Model 2 included auscultation findings in addition to Model 1, and the correct classification rate of the model was 84.8%. Model 3 included chest X-ray findings in addition to Model 1, and the correct classification rate of the model was 87.4%. Model 4, on the other hand, included both auscultation findings and chest X-ray findings in addition to Model 1, and the correct classification rate of the model was 87.6%. Based on our findings, a definitive diagnosis of FBA using only symptoms, auscultation findings, or chest X-ray findings in isolation does not seem possible. Additionally, using only symptoms and chest X-ray findings is also insufficient to make a diagnosis.


Introduction
Foreign body aspiration (FBA) is a life-threatening emergency and requires prompt diagnosis and treatment. It is frequently seen in children younger than three years of age with a peak incidence between the first and second years of life [1][2][3].
FBA should be suspected in any pediatric patient with respiratory symptoms since it is the fourth leading cause of death in the preschool age group as well as among adolescents [4]. Patients may not always manifest the classic triad of wheezing, coughing, and choking [5]. The presentation of the symptoms of FBA depends on the degree of airway obstruction, the location and the type of the object, as well as the age of the child, and the time elapsed since the event [6]. While clinical history and physical examination may offer some clues, FBA can only be conclusively diagnosed with bronchoscopy. Rigid bronchoscopy under general anesthesia still provides a safe and effective treatment [7].
When a child presents to the emergency department due to FBA, the patient's history is taken at the outset and the auscultation findings are evaluated. After that, the patient's chest X-ray is evaluated and, finally, the patient is quickly diagnosed based on all the obtained data. On the other hand, when the data is entered into the machine learning technique, it generates a result that helps us to make a preliminary diagnosis. It shows which findings may be more important in a set of several pertinent findings. Therefore, in this study, we aimed to identify which finding is more statistically important in the diagnosis of FBA by using the machine learning technique.

Materials And Methods
Study design and data collection 1 2 This was a retrospective, single-center, cross-sectional, descriptive study involving children aged 0-18 years who had undergone rigid bronchoscopy with suspected FBA at the Pamukkale University Hospital Pediatric Emergency Department between January 2005 and September 2020. Pamukkale University Hospital is a tertiary referral hospital with 857 beds and a considerable number of patient admissions annually.
Demographic data (age, gender), time of admission (days), presence of airway symptoms, the duration from the suspicion of FBA to hospital admission (symptomatic/asymptomatic), auscultation findings (equal/not equal for both lungs), chest X-ray findings (normal/abnormal), bronchoscopy results (foreign body location, the structure of the foreign body), and duration of hospitalization (days) were collected from the records in the hospital information management system. Since there was no flexible bronchoscope for children in our hospital, all children underwent rigid bronchoscopy under general anesthesia by a pediatric surgeon.
Four different models were created to evaluate the efficacy of symptoms, auscultation, and chest X-ray findings in the diagnosis of FBA (Table 1).  Machine learning techniques such as logistic regression, naive Bayes, support vector machine, bagging, and random forest were used based on the performance criteria of accuracy, F-measure, the area under the receiver operating characteristic (ROC) curve (AUC) (ROC area), and precision-recall curve (PRC) area (PRC area) to evaluate the prediction performances of the models.

Models
In our results, we used the machine learning technique, which is a model that can speed up the decisionmaking process, instead of traditional statistics. Machine learning techniques show great promise in the field of diagnosis in healthcare.

Statistical analysis
SPSS Statistics v. 11.5 (IBM Corp., Armonk, NY) program was used in the analysis of the data. Mean ±standard deviation (SD) and median (min-max) were used for quantitative variables, and the number of patients (percentage) was used for qualitative variables. The Mann-Whitney U test was used to analyze continuous variables between groups according to the distribution of the normality of the variables. Chisquare and Fisher's exact tests were used to evaluate the relationship between two qualitative variables.
Variables were analyzed at a 95% confidence interval and p<0.05 was considered statistically significant. Classification methods of logistic regression and random forest were used in the WEKA program. The data set was evaluated using the 10-fold cross-validation test option. Accuracy, F-measure, PRC area, and ROC area were used as machine learning performance criteria.

Results
Between January 2005 and September 2020, rigid bronchoscopy was performed in 236 children with suspicion of FBA. Five children who lacked follow-up data were excluded from the study. Finally, 231 children were included in the study. The mean age of the subjects was 28.2 ±27.0 months and 153 (66.2%) of the children were male.
The mean admission time was 3.9 ±11.8 days. A total of 77 (33.3%) children had no symptoms of airway obstruction. Of the children included in the study, 62.3% had abnormal auscultation findings and 51.9% had abnormal chest X-ray findings. The foreign body was located in the left lung in 55.6% of the children, in the right lung in 35.2%, and in the trachea in 9.2%. A summary of the demographic data is presented in Table 2.  .2%), crackers (n=11; 5.6%), and watermelon (n=2; 1.0%). Inedible foreign bodies were found in 27 (13.7%) patients. The most commonly removed inedible objects were seeds (n=12; 6.1%), plastic pieces (n=7, 3.6%), and turban needles (n=5; 2.5%). One of the chest radiographs demonstrating the turban needle is depicted in Figure 1. All the patients who aspirated turban scarf needles were adolescent girls with a mean age of 12.4 ±2.0 years.

FIGURE 1: Chest radiography of one of the patients before bronchoscopy -turban pin is in the left main bronchus
The foreign body was found more frequently in children with abnormal auscultation and chest X-ray findings (p<0.001 and p=0.013, respectively). However, age, gender, time of admission, and length of hospitalization did not differ according to the presence of the foreign body ( Table 3). 2022    There was no statistically significant difference between symptomatic and asymptomatic children in terms of the location of the foreign body (p=0.318, Table 4). Of note, 63.5% of the patients who aspirated edible foreign bodies and 55.6% of the patients who aspirated inedible foreign bodies were symptomatic, which revealed a non-significant association (p=0.427, Table 4).

Symptomatic
Asymptomatic P-value  In addition, the association of the structure of the object with auscultation findings and chest X-ray findings was found to be significantly different (p=0.001 and p=0.012, respectively). While 55.6% of the patients with inedible foreign bodies had abnormal auscultation findings, 68.8% of the patients with edible foreign bodies had abnormal findings. While 32.4% of the patients with inedible foreign bodies had normal chest X-rays, 57.6% of the patients with edible foreign bodies had normal X-rays.
As shown in Figure 2, the importance of the variables and the value that the variables add to the data set for the presence of a foreign body were examined by using gain ratio and information gain attribute evaluation tests. The order of significance of the variables included in the analysis, which are considered to be statistically and clinically significant in the study, is as follows: symptoms, auscultation findings, chest Xray findings, age, sex, foreign body location, and time of admission. Feature selection is a method of filtering out the important features as all the features present in the dataset are not equally important. There are some features that have no effect on the output, and hence we can skip them since our motive is to narrow down the data before feeding it to the training model. Hence, feature selection is performed before training machine learning models. We used the variables presented in Figure 2 in this study.

FIGURE 2: Significance of variables regarding the presence of the foreign body
Various machine learning techniques such as logistic regression, naive Bayes, support vector machine, bagging, and random forest were used to evaluate the prediction performances of the models. The results of the logistic regression and random forest methods, which are the methods that give the best results in each model, are given in Table 5. When the performance criteria of accuracy, F-measure, ROC area, and PRC area are evaluated together, the logistic regression method gives the best results as seen in the table. According to this method, when the variables of gender, age, time of admission, location, and symptoms were evaluated together, the correct classification rate of the model was found to be 82.3%. In other words, when we make an estimation about the presence/absence of a foreign body with this model, the accuracy rate of this estimation is 82.3%. Specifically, if the estimation of the presence/absence of the foreign body is made with this model for 100 patients who were admitted to the clinic, the estimation regarding approximately 83 patients will be correct. While the correct classification rate of model 2 with the logistic regression method was found to be 84.8%, that of model 3 with the logistic regression method was found to be 87.4%, and that of model 4 with the logistic regression method was found to be 87.6%.  When the four created models were evaluated according to the logistic regression method, it was observed that the accuracy rate increased from model 1 to model 4 (82.3% for model 1 and 87.6% for model 4; Table 5).

Models
When the models are examined, the rate of detecting the presence of the foreign body in auscultation findings in addition to the symptoms increases by 2.5%. When a chest X-ray is evaluated in addition to the symptoms, the rate of foreign body detection increases by 5.1%. When auscultation and chest X-rays are evaluated in addition to the symptoms, the rate of foreign body detection increases by 5.3%.

Discussion
It is very difficult to diagnose FBAs in pediatric patients that occur without an eyewitness present. Moreover, rigid bronchoscopy is not a simple surgical procedure. For this reason, FBA in children is an issue that must be always held in mind and kept on the agenda. When we retrospectively screened the patients who underwent rigid bronchoscopy at our clinic in order to facilitate the diagnosis, we determined that the patient being symptomatic is the most valuable indicator of FBA.
FBAs usually occur in infants (mean age of 28.2 months in our series) with a male-to-female ratio of 2:1 as per the literature [8], which aligns with our findings as well. The most common types of aspirated foreign bodies are organic substances [9]. Peanut is reported to be the most common type of foreign body, which is consistent with our results. Turban scarf needle aspiration is common in adolescent girls aged 10-16 years in our country [10,11]. In Muslim societies such as ours, adolescent girls hold the needle between their lips while wearing a scarf and can aspirate the needle while talking or laughing. This has been recently referred to as "scarf pin-related Hijab syndrome" in the literature [11]. In our series, turban needles were removed from five adolescent girls.
When the data of symptomatic and asymptomatic patients were compared, we found that they differed only with regard to the structure of the foreign body. A higher rate of edible foreign bodies was seen in symptomatic patients. A similar association was also seen regarding chest X-rays and auscultation findings of the patients. It was thought that an edible foreign body may cause more symptoms since it has a softer surface, is digestible, swells when wet, and can obstruct the bronchus.
A study by Zhu et al. reported that FBAs were misdiagnosed at a rate of 35.6% in healthcare institutions other than tertiary care centers [12]. They showed that the most common misdiagnosis was bronchiolitis (51.3%) [12]. Keeping this in mind, we aimed to achieve more accurate results by scanning previous patient records and trying different models to prevent the physician who first saw the patient from making a wrong diagnosis. The most important diagnostic clue in suspected FBA is the history of witnessed aspiration in children; however, it may not be always present in FBA cases [13][14][15].
In the literature, it has been shown that chest X-ray is not as reliable as CT in diagnosing foreign bodies in adult patients [16]. However, we normally avoid performing CT in children because of the intense amount of ionizing radiation and the need for sedative agents to avoid motion-related artifacts. We believe that the new scoring systems will help diagnose foreign bodies more accurately [17].
Our study has a few limitations. We conducted our study at a tertiary care hospital. Hence, our findings may not be applicable to patients treated at non-tertiary centers. Another limitation is that we collected our data from a single healthcare center. And finally, we recognize that we cannot fully guarantee the accuracy of the information in the available health records, and it could limit our findings as well.

Conclusions
In most cases, the event of FBAs in children occurs unwitnessed. Hence, the family members usually do not remember most of the symptoms at the time of presentation or cannot fully describe them when they arrive at the hospital. Hence, clinicians should always suspect FBA when a child presents with respiratory symptoms. A chest X-ray alone without auscultation is insufficient to confirm FBA in pediatric patients, irrespective of whether respiratory symptoms are present or not. Therefore, as done routinely in most of the clinics, all patients with suspected FBA should be questioned about symptoms at the outset; auscultation should be performed as part of the physical examination, and a chest X-ray should be requested.

Additional Information Disclosures
Human subjects: Consent was obtained or waived by all participants in this study. Pamukkale University Non-Interventional Clinical Research Ethics Committee issued approval E-60116787-020-4960. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.