"Never doubt that a small group of thoughtful, committed citizens can change the world. Indeed, it is the only thing that ever has."

Margaret Mead

Original article
peer-reviewed

The Agreement Between Reverse Transcriptase-Polymerase Chain Reaction (RT-PCR) and Rapid Antigen Test (RAT) in Diagnosing COVID-19



Abstract

Background

False-negative results derived from RT-PCR tests for diagnosing coronavirus disease (COVID-19) have raised questions about whether to consider them the gold standard for the detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Using an imperfect gold standard to assess other diagnostic tests would never let the other tests show better diagnostic performance. The best strategy in such cases is to do an agreement analysis, and this study aims to estimate the agreement between real-time reverse transcriptase-polymerase chain reaction (RT-PCR) and rapid antigen test (RAT) for COVID-19 detection.

Methods

A retrospective study was done using paired data of individuals tested for COVID-19, both by RT-PCR and RAT, obtained from the virology laboratory of Government Bundelkhand Medical College, Sagar, Madhya Pradesh, India. A sample size of 93 was calculated, and the data were abstracted in a data abstraction sheet. Variables included were results of RT-PCR and RAT, age, gender, presence of symptoms, test kit used, and the time duration between sampling for RT-PCR and RAT. Apart from descriptive statistics, keeping in mind the binary outcome of RT-PCR and RAT, Cohen’s kappa was calculated for agreement analysis. A p-value of <0.05 was considered significant.

Results

The data on 100 participants suspected to be infected with COVID-19 (58 male and 42 female) with a mean age of 39.8 (±19.0) years were analysed. The number of discordant pairs was eight. Cohen’s kappa showed substantial agreement between RT-PCR and RAT, κ=0.646, (95% CI 0.420 to 0.871), p<0.001.

Conclusion

Considering the ease of conducting RAT with quick results and substantial agreement with RT-PCR, RAT could be a better choice in detecting SARS-CoV-2 and, hence, COVID-19 disease on a large scale.

Introduction

Rapid and accurate diagnostic tests are crucial for controlling any communicable disease; the same goes for the coronavirus disease (COVID-19) pandemic. Quantitative reverse transcriptase-polymerase chain reaction (RT-PCR) is recommended by the World Health Organization (WHO) as the reference test for the laboratory diagnosis of COVID-19 [1]. Researchers have developed other forms of diagnostic tools, such as antigen-detection diagnostic tests. Generally, the ease-of-use and rapid turnaround time of antigen-detecting rapid diagnostic tests (RDTs) offer to decrease the delays in diagnosis by shifting to decentralised testing of patients with early symptoms [2]. The Indian Council of Medical Research (ICMR) published an advisory regarding the use of the rapid antigen test (RAT) on June 14, 2020, and since then it has been used extensively [3].

No test is expected to produce 100% accurate results. Ideally, a new test is compared with an existing "gold standard" and parameters like sensitivity, specificity, and predictive values of the new test are calculated. There are reports of false-negative RT-PCR results in patients with COVID-19 during the pandemic course [1]. An early study [4] showed that the sensitivity of RT-PCR varies from 68% to 100%, and the specificity is 98.9%, but several authors [5,6] have pointed out the poor performance of this technique, particularly in terms of sensitivity. According to Liu et al., the positivity rate of RT-PCR could be as low as 38% [5] (i.e., not better than chance). The accuracy of the result depends on the site and the quality of sampling as well. Li et al. in a recent study [7] also found a high rate of false-negative RT-PCR results, which were tested on specimens collected from 610 hospitalised patients from Wuhan, China. Thus, using RT-PCR as a reference standard would constitute using an imperfect gold standard. This would never let the other test show a better diagnostic performance. The additional patients identified would be regarded as false positives. With the use of chest computed tomography (CT) scans as a diagnostic method, the same mistake has also been made [8,9].In such a scenario, instead of calculating the sensitivity, specificity, and predictive values of the test (which would require one of the tests to be the gold standard), the best strategy would be to measure the degree of agreement (using the Kappa coefficient) between the two tests [10], i.e., neither of the two tests is considered to be the reference, and therefore, any discrepancies could be linked to either of the tests. Studies determining agreement between these tests are lacking in the literature. This study aims to estimate the agreement between RT-PCR and RAT for detecting severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), i.e., COVID-19 infection.

Materials & Methods

Study design, population, and setting

We conducted a retrospective record-based study between July 2021 and August 2021 after getting clearance from the Institutional Ethics Committee, Government Bundelkhand Medical College, Sagar, Madhya Pradesh, India (IECBMC/2021/24). Data was collected from the virology laboratory of the Government Bundelkhand Medical College from August 2020 to December 2020. It is prudent to mention here that vaccinations against COVID-19 in India started in January 2021. Hence, subjects in this study were unvaccinated for COVID-19. This tertiary care centre caters to a population from both urban and rural areas, and being a referral centre for COVID-19, it has facilities for both RT-PCR and rapid antigen tests. Patients, during their course of management, undergo both tests as per protocol. The study population consisted of people suspected of being infected by COVID-19 (who had symptoms) or contacts of confirmed cases (who could have been asymptomatic) identified through contact tracing.

Inclusion and exclusion criteria

We included only those subjects who had undergone both RT-PCR and RAT tests. The maximum acceptable duration between both tests was set to three days. We believe that three days is short enough to not have affected the severity of the condition and hence the diagnosis. The subjects and investigators were blinded to the result of the first test in cases where the second test was RAT. In those instances where RAT was conducted initially, it was not possible to blind the participant. Those subjects with data showing faulty sample collection (from sites other than the prescribed site) were excluded. Standard procedures were followed for conducting RT-PCR and RAT as recommended by the ICMR.

Kits used for RT-PCR and RAT

The kit used for RAT was the STANDARD Q COVID-19 Ag detection kit (SD Biosensor Inc., South Korea), and for RT-PCR, VIRALDTECT-II (Genes2Me Pvt. Ltd., India). All RT-PCR samples were oropharyngeal and all RAT samples were nasopharyngeal, as recommended by the manufacturer.

Sample size and sampling

The minimum required sample size was calculated using the formula for determining Cohen’s kappa, where we took the proportion of positive rating by rater one (i.e., RT-PCR) as 68% (sensitivity) [4], the proportion of positive rating by rater two (i.e., RAT) as 50% (sensitivity) [3], and assumed κ=85% at 95% CI with 10% precision. It was calculated to be 93. We took 100 subjects. Universal sampling was done, and subjects were included or excluded as per the criteria set above.

Study variables and data analysis

Data were abstracted in a data abstraction sheet designed with the help of Epi Info software (version 7.2.4.0, CDC, Atlanta) for Windows. We abstracted data on results of RT-PCR and RAT (positive and negative), age, sex, the test kit used, the time duration between samples taken for RT-PCR and RAT, and the presence of symptoms (cough, fever, sore throat, and generalised body ache) from the records of subjects for further analysis. Participants were classified as "symptomatic" if any of the symptoms were present. We used IBM Statistical Package for Social Sciences (SPSS) software (IBM Corp., released 2019. IBM SPSS Statistics for Windows, Version 26.0. Armonk, NY: IBM Corp.) for data analysis. Qualitative variables were summarised using percentages, and quantitative variables were summarised using mean (SD) and median (IQR). Keeping in mind the binary outcome of RT-PCR and RAT, Cohen’s Kappa Coefficient (κ) and agreement percentage (accuracy) were calculated. Kappa was classified as <0.00: poor, 0.00-0.20: slight agreement, 0.21-0.40: fair agreement, 0.41-0.60: moderate agreement, 0.61-0.80: substantial agreement, and 0.81-1.00: almost perfect agreement [11]. P-values <0.05 were considered significant throughout. All digital records were password protected, and only the chief investigator had access to them. The reporting was done as per the guidelines for reporting reliability and agreement studies (GRRAS) checklist [12].

Results

We shortlisted 124 subjects, out of which 24 were excluded owing to incomplete information or other lacunae. Data on 100 subjects were finalised for analysis, which met all the inclusion and exclusion criteria. Fifty-eight (58%) were males and 42 (42%) were females, with ages ranging from two to 88 years. The mean age was 39.8 (SD ±19.0) years; the median age was 39 years (IQR 25.5-55 years).

Out of 100 subjects, 13 tested positive and 87 tested negative on both RT-PCR and RAT, with eight discordant pairs. Cohen’s kappa showed substantial agreement, κ=0.646, (95% CI 0.420 to 0.871), p<0.001 (Table 1).

RAT result RT-PCR Total Kappa (95% CI) P value
Positive Negative
RAT Positive 9 4 13 0.646 (0.420 - 0.871)   <0.001
RAT Negative 4 83 87
Total 13 87 100

The Kappa varied with the intervals between taking samples for tests. Figure 1 shows this variation when the sample was collected on the same day and with a one-day difference, with accuracies of 95.1% and 88.9%, respectively. Both were statistically significant, with p-values <0.000 and <0.001 respectively.

When stratified based on the presence or absence of symptoms, 31 participants were symptomatic, having at least one of the symptoms, and 69 were asymptomatic. The corresponding kappa values were κ=0.59 (95% CI 0.28 to 0.91), p<0.001 and κ=0.64 (95% CI 0.27 to 1.0), p<0.001, showing moderate and substantial agreement among symptomatic and asymptomatic participants, respectively.

On RT-PCR, out of 13 participants who tested positive and 87 who tested negative, respectively, nine (69.2%) and 22 (25.3%) were symptomatic. Likewise, on the RAT, out of 13 participants who tested positive and 87 who tested negative, the number of symptomatic participants was eight (61.5%) and 23 (26.4%), respectively.

Discussion

This study investigated the agreement between these two tests for SARS-CoV-2 detection with the help of Cohen’s Kappa [11]. Since the commencement of RAT testing in India, few reliable kits have been available. At that time, an independent two-site study of the sole available or stand-alone antigen detection assay in India, STANDARD Q COVID-19 Ag detection kit, was performed to assess its sensitivity, specificity, and practicality of application as a point-of-care test for early detection of SARS-CoV-2. However, the agreement between the two tests was not tested or mentioned. There are very few studies that have evaluated the agreement between these two tests. The majority of the papers available in the literature have evaluated diagnostic accuracy mainly using sensitivity and specificity [13-16]. With a Cohen’s kappa, κ=0.646, (95% CI 0.420 to 0.871), p<0.001, our study showed a substantial agreement between the two tests.

Jääskeläinen et al. [17] evaluated the performance of three rapid diagnostic tests (RDTs), viz., Sofia, STANDARD Q and Panbio™ and found overall agreements of 84.57%, 84.85%, and 86.32%, respectively, with a kappa value of 0.636, 0.633, and 0.660, respectively. This is similar to the results of our study on the STANDARD Q COVID-19 Ag detection kit. The study by Peña M et al. [18] on 854 asymptomatic individuals from the Chilean region calculated an accuracy of 97.04% with a kappa value of 0.78 (95% CI 0.70-0.86), which showed better performance than that of our study. Felipe Pérez‐García et al. [19] did a retrospective comparative evaluation between two antigen rapid diagnostic tests (Ag-RDTs), viz., Panbio and SD-Biosensor, for detection of SARS-CoV-2 and found agreement in accuracy and κ values of 80.9% and 0.596 and 82.6% and 0.646, respectively. The results were similar to our study. They also concluded that RDTs were excellent for the diagnosis of high viral load samples and those early in the disease.

Ristić M et al. [20] evaluated the clinical performance of the STANDARD Q COVID-19 Ag Test (SD Biosensor, Gyeonggi-do, South Korea) by analysing prospectively collected data on 120 symptomatic patients and concluded a strong agreement between the STANDARD Q COVID-19 Ag Test and RT-PCR with a kappa of 0.852 and a pooled accuracy of 92%, whereas our study reported a similar agreement percentage and a κ=0.59 (95% CI 0.28 to 0.91), p<0.001 among symptomatic patients. This difference could be explained by the fact that they included suspects who were in their first five days of illness, whereas our sample included a mix of suspects who had samples taken on various days after symptoms began.

Among the studies done in India, the strength of agreement between these two tests ranged from moderate to almost perfect. A prospective study [21] done among 756 patients from a district hospital in North India shows a substantial agreement like ours with a κ value of 0.6482 (95% CI: 0.5801 to 0.7163). However, Pandey et al. found moderate agreement between these two tests [22]. The reported κ value in their retrospective study was 0.57. Whereas, a κ value of 0.86 was reported by Gupta et al. in their study, which showed almost perfect agreement between RAT and RT-PCR [23].

In contrast, findings by Amer RM et al. in their cross-sectional study [24] on 83 COVID suspects showed much less agreement with a kappa of 0.3 (95% CI 0.16-0.59) and an accuracy of 75.9%, recommending against using RAT alone for COVID-19 diagnosis. With a smaller sample size in the Amer RM study, this sharp contrast between results needs to be interpreted carefully.

Limitations

Owing to the pandemic, logistical hurdles, and the retrospective nature of the study, we did not have control over the quality of samples for both tests. However, we considered only those participants for whom the sample was taken from the recommended site as mentioned by the kit manufacturer. We also could not consider the order in which the tests were conducted. However, we did take into account the time difference (in days) between collecting samples for the respective tests. The time interval between symptom onset and sample collection could not be recorded, even though the number of subjects included in the analysis is more than the minimum required sample size calculated with the given precision. However, evidence from a larger sample study would have been more conclusive.

Conclusions

Evaluating the ease of conducting tests, the quick results, the lack of requiring a laboratory setup, the affordability, and the substantial agreement (κ=0.646, 95% CI 0.420 to 0.871, 92% agreement accuracy) with RT-PCR, we conclude that RAT may be a better choice for detecting SARS-CoV-2 and hence COVID-19 disease on a large scale. However, larger prospective studies would be needed to substantiate our findings.


References

  1. da Silva SJ, Silva CT, Guarines KM, Mendes RP, Pardee K, Kohl A, Pena L: Clinical and laboratory diagnosis of SARS-CoV-2, the virus causing COVID-19. ACS Infect Dis. 2020, 6:2319-36. 10.1021/acsinfecdis.0c00274
  2. Antigen-detection in the diagnosis of SARS-CoV-2 infection using rapid immunoassays. (2022). Accessed: April 13 2022: https://apps.who.int/iris/bitstream/handle/10665/334253/WHO-2019-nCoV-Antigen_Detection-2020.1-eng.pdf.
  3. ICMR- advisory on use of rapid antigen detection test for COVID-19. (2022). Accessed: July 7 2022: https://www.icmr.gov.in/pdf/covid/strategy/Advisory_for_rapid_antigen_test14062020.pdf.
  4. Dinnes J, Deeks JJ, Adriano A, et al.: Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev. 2020, 8:CD013705. 10.1002/14651858.CD013705
  5. Liu R, Han H, Liu F, et al.: Positive rate of RT-PCR detection of SARS-CoV-2 infection in 4880 cases from one hospital in Wuhan, China, from Jan to Feb 2020. Clin Chim Acta. 2020, 505:172-5. 10.1016/j.cca.2020.03.009
  6. Xie J, Ding C, Li J, et al.: Characteristics of patients with coronavirus disease (COVID-19) confirmed using an IgM-IgG antibody test. J Med Virol. 2020, 92:2004-10. 10.1002/jmv.25930
  7. Li Y, Yao L, Li J, Chen L, Song Y, Cai Z, Yang C: Stability issues of RT-PCR testing of SARS-CoV-2 for hospitalized patients clinically diagnosed with COVID-19. J Med Virol. 2020, 92:903-8. 10.1002/jmv.25786
  8. Ai T, Yang Z, Hou H, et al.: Correlation of chest CT and RT-PCR testing for coronavirus disease 2019 (COVID-19) in China: a report of 1014 cases. Radiology. 2020, 296:E32-40. 10.1148/radiol.2020200642
  9. Majidi H, Niksolat F: Chest CT in patients suspected of COVID-19 infection: A reliable alternative for RT-PCR. Am J Emerg Med. 2020, 38:2730-2. 10.1016/j.ajem.2020.04.016
  10. Dramé M, Tabue Teguo M, Proye E, Hequet F, Hentzien M, Kanagaratnam L, Godaert L: Should RT-PCR be considered a gold standard in the diagnosis of COVID-19?. J Med Virol. 2020, 92:2312-3. 10.1002/jmv.25996
  11. Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33:159-74.
  12. Kottner J, Audigé L, Brorson S, et al.: Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol. 2011, 64:96-106. 10.1016/j.jclinepi.2010.03.002
  13. Berger A, Nsoga MT, Perez-Rodriguez FJ, et al.: Diagnostic accuracy of two commercial SARS-CoV-2 antigen-detecting rapid tests at the point of care in community-based testing centers. PLoS One. 2021, 16:e0248921. 10.1371/journal.pone.0248921
  14. Oh SM, Jeong H, Chang E, et al.: Clinical application of the standard Q COVID-19 Ag test for the detection of SARS-CoV-2 infection. J Korean Med Sci. 2021, 36:e101. 10.3346/jkms.2021.36.e101
  15. Nalumansi A, Lutalo T, Kayiwa J, et al.: Field evaluation of the performance of a SARS-CoV-2 antigen rapid diagnostic test in Uganda using nasopharyngeal samples. Int J Infect Dis. 2021, 104:282-6. 10.1016/j.ijid.2020.10.073
  16. Jakobsen KK, Jensen JS, Todsen T, et al.: Accuracy and cost description of rapid antigen test compared with reverse transcriptase-polymerase chain reaction for SARS-CoV-2 detection. Dan Med J. 2021, 68:
  17. Jääskeläinen AE, Ahava MJ, Jokela P, et al.: Evaluation of three rapid lateral flow antigen detection tests for the diagnosis of SARS-CoV-2 infection. J Clin Virol. 2021, 137:104785. 10.1016/j.jcv.2021.104785
  18. Peña M, Ampuero M, Garcés C, et al.: Performance of SARS-CoV-2 rapid antigen test compared with real-time RT-PCR in asymptomatic individuals. Int J Infect Dis. 2021, 107:201-4. 10.1016/j.ijid.2021.04.087
  19. Pérez-García F, Romanyk J, Moya Gutiérrez H, et al.: Comparative evaluation of Panbio and SD Biosensor antigen rapid diagnostic tests for COVID-19 diagnosis. J Med Virol. 2021, 93:5650-4. 10.1002/jmv.27089
  20. Ristić M, Nikolić N, Čabarkapa V, Turkulov V, Petrović V: Validation of the standard Q COVID-19 antigen test in Vojvodina, Serbia. PLoS One. 2021, 16:e0247606. 10.1371/journal.pone.0247606
  21. Choudhary S, Ishrat A: Validation of rapid SARS-COV-2 antigen detection test as a screening tool for detection of Covid-19 infection at district hospital in northern India. Asian J Med Sci. 2022, 13:7-10. 10.3126/ajms.v13i4.40765
  22. Pandey AK, Mohanty A, Hada V, Rath RS, Kumar S, Kishore S, Kant R: Comparison of the rapid antigen testing method with RT-qPCR for the diagnosis of COVID-19. Cureus. 2021, 13:e17405. 10.7759/cureus.17405
  23. Gupta A, Khurana S, Das R, et al.: Rapid chromatographic immunoassay-based evaluation of COVID-19: a cross-sectional, diagnostic test accuracy study and its implications for COVID-19 management in India. Indian J Med Res. 2021, 153:126-31. 10.4103/ijmr.IJMR_3305_20
  24. Amer RM, Samir M, Gaber OA, et al.: Diagnostic performance of rapid antigen test for COVID-19 and the effect of viral load, sampling time, subject's clinical and laboratory parameters on test accuracy. J Infect Public Health. 2021, 14:1446-53. 10.1016/j.jiph.2021.06.002

Original article
peer-reviewed

The Agreement Between Reverse Transcriptase-Polymerase Chain Reaction (RT-PCR) and Rapid Antigen Test (RAT) in Diagnosing COVID-19


Author Information

Rupesh Sahu

Preventive Medicine, Government Chhindwara Institute of Medical Sciences, Chhindwara, IND

Amarnath Gupta

Preventive Medicine, Government Bundelkhand Medical College, Sagar, IND

Sumit Rawat

Microbiology, Government Bundelkhand Medical College, Sagar, IND

Abhijit Das Corresponding Author

Preventive Medicine, Government Bundelkhand Medical College, Sagar, IND


Ethics Statement and Conflict of Interest Disclosures

Human subjects: Consent was obtained or waived by all participants in this study. Institutional Ethics Committee, Bundelkhand Medical College Sagar MP issued approval IECBMC/2021/24. The Institutional Ethics Committee, Bundelkhand Medical College, has approved the research work. Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue. Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following: Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work. Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work. Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Acknowledgements

We acknowledge the support of Dr. RS Verma (Dean) and Dr. Rohit Trivedi (Professor, Department of Community Medicine) of Government Bundelkhand Medical College, Sagar. We also would like to acknowledge the contribution of the staff of the Microbiology Department at the Government Bundelkhand Medical College, Sagar (MP) for helping us conduct RT-PCR and RAT tests and for maintaining records.



Original article
peer-reviewed

The Agreement Between Reverse Transcriptase-Polymerase Chain Reaction (RT-PCR) and Rapid Antigen Test (RAT) in Diagnosing COVID-19


Figures etc.

SIQ
6.7
RATED BY 5 READERS
CONTRIBUTE RATING

Scholarly Impact Quotient™ (SIQ™) is our unique post-publication peer review rating process. Learn more here.