INTRODUCTION
The COVID-19 pandemic has challenged public health systems worldwide, from patient care to epidemic management, but also to even guarantee the quality of SARS-CoV-2–related diagnostic products. 1 Hundreds of in vitro SARS-CoV-2 RT-qPCR diagnostic kits are available on the market, some of them have received emergency use authorization (EUA) from the U.S. Food & Drug Administration (FDA) or other international agencies like Chinese CDC (C-CDC), whereas many others only report clinical validations made by manufacturers. The U.S. CDC (CDC) designed the FDA EUA 2019-nCoV CDC kit (IDT), based on N1 and N2 gene targets to detect SARS-CoV-2 and RNase P as an RNA extraction quality control that is considered a gold standard worldwide for SARS-CoV-2 RT-PCR diagnosis. 2–5
Among the commercial kits available in South America for SARS-CoV-2 diagnosis, several are made in China. For instance, COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are both multiplex SARS-CoV-2 RT-qPCR kit for N and ORF1ab viral gene targets and RNase P as an RNA extraction quality control. Although both of them are authorized for clinical use in China, only the Sansure Biotech kit holds the FDA EUA. 6 We also have previously reported that other made in China SARS-CoV-2 RT-PCR kits lacking FDA EUA but holding C-CDC EUA had a great clinical performance and analytical sensitivity. 7
We herein present a comparison of the analytical and clinical performance of COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) for SARS-CoV-2 RT-qPCR diagnosis from nasopharyngeal samples, using the CDC protocol as a gold standard.
MATERIALS AND METHODS
Study design.
Ninety seven clinical specimens (nasopharyngeal swabs collected on 0.5 mL TE pH 8 buffer) were included on this study. Also, negative controls (TE pH 8 buffer) were included as control for carryover contamination, one for each set of RNA extractions.
RNA extraction and RT-qPCR for SARS-CoV-2 diagnosis using 2019-nCoV CDC kit.
All the samples included on the study were tested following an adapted version of the CDC protocol: (1) using AccuPrep Viral RNA extraction kit (Bioneer, Daejeon, South Korea) as an alternate RNA extraction method, and (2) using CFX96 BioRad instrument. 8–12
RT-qPCR for SARS-CoV-2 diagnosis using COVID-19 Nucleic Acid Test Kit (eDiagnosis).
Same RNA extractions from all the samples included in the study were tested using COVID-19 Nucleic Acid Test Kit (eDiagnosis), following the manufacturer’s manual. To avoid RNA degradation, samples were processed for 2019-nCoV CDC kit and COVID-19 Nucleic Acid Test Kit (eDiagnosis) within the same day, or frozen at −80 C to be processed the next day.
RT-qPCR for SARS-CoV-2 diagnosis using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech).
Seventy four from the total 97 RNA extractions included in the study were tested using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech). Although this kit includes a lysis buffer for an alternative RNA isolation without using RNA extraction kits, we did not follow that protocol. Also, although the manufacturer’s manual recommends a final volume reaction of 50 µL (30 µl of PCR Master Mix + 20 µL of RNA extraction), we used a final volume reaction of 15 µL (9 µL of PCR Master Mix + 6 µL of RNA extraction), keeping the proportion of reagents recommended by the manufacturer. To avoid RNA degradation, samples were processed for 2019-nCoV CDC kit and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) within the same day, or frozen at −80 C to be processed the next day.
Analytical sensitivity.
Limit of detection (LoD) was performed using the commercially available 2019-nCoV N positive control (IDT); provided at 200.000 genome equivalents/mL, it was used for calibration curves to obtain the viral loads of the samples. Viral loads can be expressed as copies/µL of RNA extraction or copies/mL of sample; the conversion factor is 200, as 0.2 mL of sample is used for RNA extraction and 40 µL is used as final elution volume of RNA extraction.
Statistics.
95% confident intervals were calculated using Jamovi statistical software.
Ethics statement.
All samples have been submitted for routine patient care and diagnostics. Ethics approval was not sought because the study involves laboratory validation of test methods and the secondary use of anonymous pathological specimens that falls under the category “exempted” by Comité de Etica para Investigación en Seres Humanos from “Universidad de Las Américas.”
RESULTS
Clinical performance and estimation of LoD of COVID-19 Nucleic Acid Tests Kit (eDiagnosis) using 2019-nCoV CDC kit as a gold standard.
Ninety seven samples were tested for SARS-CoV-2 following both COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV CDC kit protocols as described on the methods. For the 2019-nCoV CDC EUA kit, 63 samples tested positive and 34 samples tested negative (Table 1 and Supplemental Tables 1 and 2). Thirty-two of 34 samples tested negative for 2019-nCoV CDC kit were also SARS-CoV-2 negative for COVID-19 Nucleic Acid Test Kit (eDiagnosis), so the specificity obtained in our study was 94.1% (91.0–97.2, CI 95%). The two “false-positive” samples had Ct values > 38 (Supplemental Tables 1 and 2) that are associated to low viral loads. As the LoD of COVID-19 Nucleic Acid Test Kit (eDiagnosis) is 2.5 copies/µL according to the manufacturers, these two “false-positive” samples would be actually true-positive SARS-CoV-2 samples that are below the LoD of the CDC protocol. 8–12 So the real specificity for COVID-19 Nucleic Acid Test Kit (eDiagnosis) is 100%.
Clinical performance of COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) using the CDC protocol as a gold standard (% values: sensitivity)
| RT-PCR kit | Positive samples | False-negative samples | Total SARS-CoV-2–positive samples |
|---|---|---|---|
| COVID-19 Nucleic Acid Test Kit (eDiagnosis) | 43 (100%) | 0 | 43 |
| 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) | 41 (95.3%) | 2 | 43 |
Only SARS-CoV-2–positive samples included on the study are detailed.
For the 63 SARS-CoV-2–positive samples for 2019-nCoV CDC EUA kit, all were also positive for COVID-19 Nucleic Acid Test Kit (eDiagnosis), resulting in a sensitivity of 100% (Table 1 and Supplemental Tables 1 and 2).
As the LoD is defined as the lowest viral load in which all replicates are detected (100% sensitivity), our data indicate that the LoD for COVID-19 Nucleic Acid Test Kit (eDiagnosis) would be around 2.5 copies/µL of RNA extraction (500 viral RNA copies/mL of sample), as no failure to detect SARS-CoV-2–positive samples above this threshold was found (Supplemental Table 1).
Clinical performance and estimation of LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) using 2019-nCoV CDC kit as a gold standard.
Seventy-four samples were tested for SARS-CoV-2 using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) and 2019-nCoV CDC kit protocols as described on the methods. For the 2019-nCoV CDC EUA kit, 43 samples tested positive and 31 samples tested negative (Table 1 and Supplemental Table 1). Twenty-eight of 31 samples tested negative for 2019-nCoV CDC kit were also SARS-CoV-2 negative using 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech), so the specificity obtained in our study was 90.3% (87.3–93.7, CI 95%). The three “false-positive” samples had Ct values > 37 (Supplemental Table 1) that are associated to low viral loads. As the LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) is 1 copie/µL according to the manufacturers, these three “false-positive” samples would be actually true-positive SARS-CoV-2 samples that are below the LoD of the CDC protocol. 7–11 So the real specificity for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) is 100%.
For the 43 SARS-CoV-2–positive samples for 2019-nCoV CDC EUA kit, 41 samples were also positive for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech), resulting in a sensitivity of 95.3% (93.1–97.5, CI 95%; Table 1 and Supplemental Table 1).
As the LoD is defined as the lowest viral load in which all replicates are detected (100% sensitivity), our data indicate that the LoD for 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) would be above five copies/µL of RNA extraction (1,000 viral RNA copies/mL of sample), as only one failure (40/41; 97.6% sensitivity) to detect SARS-CoV-2–positive samples above this threshold was found (Supplemental Table 1).
DISCUSSION
Although the main limitation of our study is the sample size (97 specimens), our results support that both COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) had a great clinical performance compared with 2019-nCoV CDC EUA, with sensitivity values up to 100% and 95.3%, and specificity values of 100%.
As we have described on the Results, we could estimate the LoD of COVID-19 Nucleic Acid Tests Kit (eDiagnosis) on approximately 500 viral RNA copies/mL of sample which is on agreement with the LoD detailed at the manufacturer’s manual. Moreover, we could estimate the LoD of 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) to be around on 1,000 viral RNA copies/mL of sample, slightly over the LoD of 200 copies/mL detailed at the manufacturer’s manual. However, the LoDs obtained for both kits are quite acceptable for a reliable SARS-CoV-2 diagnosis considering the viral load frequency population distributions. 13,14
On Table 2, analytical parameters and other characteristics for COVID-19 Nucleic Acid Test Kit (eDiagnosis) and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) are summarized. As we have detailed on the Introduction, only the Sansure Biotech kit has both FDA EUA and C-CDC EUA, whereas the eDiagnosis only holds C-CDC EUA. Moreover, we have already published another report showing a great clinical performance from another C-CDC EUA kit, from Da an Gene company. 7 In addition, we have also already published other clinical evaluation reports for Spanish, Canadian, and South Korean RT-PCR kits with really variable results in terms of sensitivity and LoD. 7,8,15–17 Although further studies would be necessary involving many more SARS-CoV-2 RT-PCR diagnostic kits, we already found an interesting trend: only those kits with their country of origin clinical use authorization had an acceptable clinical performance and analytical sensitivity.
Comparison of 2019-nCoV CDC EUA (IDT), COVID-19 Nucleic Acid Test Kit (eDiagnosis), and 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech)
| SARS-CoV-2 RT-PCR kit (company/country) | Viral gene targets | LoD observed (promised by the manufacturer) | EUA |
|---|---|---|---|
| 2019-nCoV CDC EUA (IDT) | N1, N2 | 1,000 viral copies/mL | FDA |
| COVID-19 Nucleic Acid Test Kit (eDiagnosis) | N, ORF1ab | 500 viral copies/mL (500 viral copies/mL) | C-CDC |
| 2019-nCoV Nucleic Acid Diagnostic Kit (Sansure Biotech) | N, ORF1ab | 1,000 viral copies/mL (200 viral copies/mL) | FDA C-CDC |
C-CDC = Chinese CDC; EUA = Emergency Use Authorization; FDA = Federal Drug Administration; LoD = limit of detection.
Considering the worldwide high demand of reagents for SARS-CoV RT-qPCR diagnosis, independent clinical performance and analytical sensitivity evaluations are necessary to guarantee the quality of the supplies in the market for every country in the world, particularly for developing countries usually lacking of reliable regulatory agencies. We claim for the role of universities at developing countries like Ecuador, not only to assess quality evaluations like the ones described on this work but also to support an extremely needed massive SARS-CoV-2 diagnosis.
ACKNOWLEDGMENTS
We thank Tannya Lozada from “Dirección General de Investigación de la Universidad de Las Américas,” and also the authorities from Universidad de Las Américas, for their logistic support to make SARS-CoV-2 diagnosis possible. The American Society of Tropical Medicine and Hygiene has waived the Open Access fee for this article due to the ongoing COVID-19 pandemic.
REFERENCES
- 1.↑
Ortiz-Prado E et al. 2020. Clinical, molecular, and epidemiological characterization of the SARS-CoV-2 virus and the coronavirus disease 2019 (COVID-19), a comprehensive literature review. Diagn Microbiol Infect Dis 98: 115094.
- 2.↑
Lu X et al. 2020. US CDC real-Time reverse Transcription PCR panel for detection of severe acute respiratory syndrome coronavirus 2. Emerging Infect Dis 26: 1654–1665.
- 3.
CDC , 2020. Interim Guidelines for Collecting, Handling, and Testing Clinical Specimens from Persons for Coronavirus Disease 2019 (COVID-19). Atlanta, GA: Center for Diseases Control and Prevention. Available at: https://www.cdc.gov/coronavirus/2019-ncov/lab/guidelines-clinical-specimens.html.
- 4.
Rhoads DD , Cherian SS , Roman K , Stempak LM , Schmotzer CL , Sadri N . Comparison of abbott ID now, diasorin simplexa, and CDC FDA EUA methods for the detection of SARS-CoV-2 from nasopharyngeal and nasal swabs from individuals diagnosed with COVID-19. J Clin Microbiol 58: e00760-20.
- 5.↑
Nallaa AK et al. 2020. Comparative performance of SARS-CoV-2 detection Assays using seven different primer/probe sets and one assay kit. J Clin Microbiol 58: e00557-20.
- 7.↑
Freire-Paspuel B , Bruno A , Orlando A , Garcia-Bereguiain MA , 2021. Analytical and clinical evaluation of two RT-qPCR SARS-CoV-2 diagnostic tests with emergency use authorization in Ecuador. Am J Trop Med Hyg XX: XXX. Available at: https://doi.org/10.4269/ajtmh.20-1439.
- 8.↑
Freire-Paspuel B , Vega-Mariño P , Velez A , Castillo P , Cruz M , Garcia-Bereguiain MA , 2020. Evaluation of nCoV-QS (MiCo BioMed) for RT-qPCR detection of SARS-CoV-2 from nasopharyngeal samples using CDC FDA EUA qPCR kit as a gold standard: an example of the need of validation studies. J Clin Virol 128: 104454.
- 9.
Freire-Paspuel B , Vega-Mariño P , Velez A , Castillo P , Gomez-Santos EE , Cruz M , Garcia-Bereguiain MA , 2020. Cotton-tipped plastic swabs for SARS-CoV-2 RT-qPCR diagnosis to prevent supply shortages. Front Cell Infect Microbiol 10: 356.
- 10.
Freire-Paspuel B , Vega-Mariño P , Velez A , Castillo P , Cruz M , Garcia-Bereguiain MA , 2020. Sample pooling of RNA extracts to speed up SARS-CoV-2 diagnosis using CDC FDA EUA RT-qPCR kit. Virus Res 290: 198173.
- 11.↑
Freire-Paspuel B , Vega-Mariño P , Velez A , Castillo P , Masaquiza C , Cedeño-Vega R , Lozada T , Cruz M , Garcia-Bereguiain MA , 2020. One health" inspired SARS-CoV-2 surveillance: the galapagos Islands experience. One Health 11: 100185.
- 12.↑
Freire-Paspuel B , Garcia-Bereguiain MA , 2020. Analytical sensitivity and clinical performance of a triplex RT-qPCR assay using CDC N1, N2 and RP targets for SARS-CoV-2 diagnosis. Int J Infect Dis 102: 14–16.
- 13.↑
Lavezzo E et al. 2020. Suppression of a SARS-CoV-2 outbreak in the Italian municipality of Vo’. Nature 584: 425–429.
- 14.↑
Kleiboeker S , Cowden S , Grantham J , Nutt J , Tyler A , Berg A , Altrich M , 2020. SARS-CoV-2 viral load Assessment in respiratory samples. J Clin Virol 129: 104439.
- 15.↑
Freire-Paspuel B , Garcia-Bereguiain MA , 2020. Poor sensitivity of AccuPower SARS-CoV-2 real Time RT-PCR kit (Bioneer, South Korea). Virol J 17: 178.
- 16.
Freire-Paspuel B , Vega-Mariño P , Velez A , Cruz M , Perez F , Garcia-Bereguiain MA , 2020. Analytical and clinical comparison of Viasure (CerTest Biotec) and 2019-nCoV CDC (IDT) RT-qPCR kits for SARS-CoV2 diagnosis. Virology 553: 154–156.
- 17.↑
Freire-Paspuel B , Garcia-Bereguiain MA , 2021. Low clinical performance of “Isopollo COVID19 detection kit” (Monitor, South Korea) for RT-LAMP based SARS-CoV-2 diagnosis: a call for action against low quality products for developing countries. Int J Infect Dis 104: 303–305.