Evaluation of SARS-CoV-2 Rapid Antigen Tests in Use on the Isle of Youth, Cuba

INTRODUCTION The use of various diagnostic techniques is increasingly common in pandemic scenarios. It is important to update evaluations of their metric properties in diﬀerent times and settings. OBJECTIVE Evaluate metric properties of a SARS-CoV-2 rapid antigen test relative to a reference standard. METHODS We carried out a prospective evaluation study of the SARS-CoV-2 rapid antigen test as compared to the RT-PCR test, which is considered the reference standard. Our sample was comprised of 778 individuals, and we calculated sensitivity, speciﬁcity, predictive values, prevalence and validity indices. RESULTS Of the total 778 samples, 70 were true positives, 658 were true negatives, and 27 were false negatives when compared to RT-PCR test results. We obtained a sensitivity of 75.3% (95% CI = 65.96–84.50); a speciﬁcity of 96.1% (95% CI = 94.53–97.59); 72.2% for positive predictive value, and 96.6% for negative predictive value. The estimated prevalence was 11.9% and the validity index was 93.6%. CONCLUSIONS The index values validate use of the SARS-CoV-2 rapid antigen test until prevalence falls below 2.5%, since as SARS-CoV-2 infection prevalence decreases, so does the predictive value of the PCR result. The use of the SARS-CoV-2 rapid antigen test on the Isle of Youth, Cuba, was decisive in the pandemic’s clinical– epidemiological management.


INTRODUCTION
The COVID-19 health emergency, now recognized as a global pandemic, had its turning point on February 23, 2020, when the city of Wuhan-where the first cases were reported-was put into quarantine. [1] The virus continues to spread at an accelerated pace, due to efficient transmission not only by symptomatic, but also by asymptomatic and presymptomatic persons as well. [1,2] New and more contagious variants appeared after this study was concluded, and the proportion of symptomatic patients has also increased.
The world's scientific community has raced to find solutions to the problems posed by the SARS-CoV-2 virus in diagnosis, treatment and prevention. [3] Thus, over the course of the pandemic, efforts in various fields have yielded more or less effective therapies, including monoclonal antibodies, as well as preventive vaccines based on various platforms. Numerous and diverse diagnostic tests have also appeared on the market, each aimed at identifying infected patients as soon as possible.
Control strategies for diagnostic testing have been based on molecular detection of viral RNA in respiratory samples; and in most available commercial assays, by reverse-transcriptase polymerase chain reaction (RT-PCR). RT-PCR is used as a reference technique due to its sensitivity and specificity for detecting respiratory viruses. [2] However, rapid serological marker tests also support diagnosis, based on detection of antigens and antibodies: the former are qualitative tests that only express one result (positive or negative), while the latter yield a quantitative measurement of circulating immunoglobulins (IgG), the amount depending on the individual patient's infectious phase. [4] South Korea, calling on extensive experience in other epidemic outbreaks like SARS1 in 2015, is one of the countries that carried out the largest number of tests per population, in the process demonstrating that one of the best strategies for controlling the epidemic was massive testing. This strategy has been implemented by a number of countries in their fight against the COVID-19 pandemic. [5] And "test, test and test" has been WHO's recommendation to countries worldwide in handling the COVID-19 pandemic. [6] The values that manufacturers of diagnostic tests include in their products come from patients in reference hospitals under ideal conditions which should not be extrapolated to other populations or real-world situations. [6] Therefore, tests should be chosen based on reliability and validity, measured in terms of their sensitivity, specificity, and predictive values (positive and negative) obtained from open populations. [7,8] WHO has included in its Emergency Use Listing of COVID-19 diagnostic tools some general considerations about rapid antigen tests, as well as several assays;[9] however, objective quantitative evaluations of their metric properties under field conditions are still needed. This is our primary goal in evaluating the rapid COVID-19 antigen test used on the Isle of Youth, Cuba, an area whose geographical location and clinical-epidemiological man-

IMPORTANCE
This study provides recent information on fundamental attributes of a rapid antigen test for SARS-CoV-2 infection in an epidemiological scenario with particular geographic and epidemiological characteristics.

Original Research
Peer Reviewed agement strategy had an impact and marked a milestone in the management and containment of the disease.
The Isle of Youth is a Special Municipality south of Cuba's main island and is bordered to the north by the Gulf of Batabanó, to the east by waters shared by Matanzas province, and to the south and west by the Caribbean Sea. These unique characteristics mean that the island can only be reached by air or sea. At the beginning of the pandemic, navigation was restricted to cargo transport with strict control at points of entry, and travelers flying in were under close surveillance upon arrival and were isolated for several days in designated quarantine centers. The same procedures were applied to anyone arriving by sea, which limited movement to areas of highest risk and minimized access to the territory.
Compliance with epidemic control measures was easier in a territory with only 83,479 population and a population density of 37.9 inhabitants per km 2 . Although a localized epidemic in this context would translate into high incidence and lethality rates, containment was less complicated due to both the low number of inhabitants and the restriction of access routes into the territory; so, everything that occurs on the island would be controllable, theoretically, once the transmission chain was interrupted.
The Isle of Youth's demographic characteristics-together with timely diagnosis of SARS-CoV-2 infection using the rapid antigen test-allowed epidemiological control actions to be carried out practically in real time, which positively impacted the COVID-19 epidemic's containment, with much more favorable results than those exhibited by Cuba as a whole at that time.
The objective of this study was to evaluate metric properties of a SARS-CoV-2 rapid antigen test relative to a reference standard under field conditions.

Study design and participants
We carried out a prospective study to evaluate the SARS-CoV-2 rapid antigen test, comparing it with the RT-PCR reference standard.
Procedures All persons presenting with COVID-19-suggestive symptoms detected during January-April 2021 (who gave their consent to participate) were included in the study (778 in total). They were identified in doctor's visits aimed at detecting cases or as a result of epidemiological controls-a strategy in which passive, active and specialized surveillance was combined during the epidemic. All patients underwent both a rapid antigen test and RT-PCR testing. This period corresponded to a local transmission phase in the area during the second wave of the epidemic in Cuba.
Statistical analysis examined point and interval estimates at 95% of the tests' operational characteristics with respect to the reference standard (sensitivity, specificity, predictive values, and validity indices; the latter defined as the total percentage of coin-cidences between tests).

Study test descriptions
Reference test (gold standard) RIDAGENE SARS-CoV-2 (R-Biopharm, Germany) was used in this study; it is a real-time multiplex RT-PCR for direct quantitative detection of SARS-CoV-2 RNA from oropharyngeal or nasopharyngeal swabs from individuals with symptomatic respiratory infections.

Rapid test
The SARS-CoV-2 rapid antigen test used (SD BIO-SENSOR, INC., South Korea) is a rapid immunochromatographic test for qualitative detection of specific SARS-CoV-2 antigens present in the nasopharynx. The manufacturer's clinical evaluation of the test's sensitivity and specificity were 96.52% and 99.68%, respectively.
Sampling Two nasopharyngeal exudate samples were collected from each patient whose clinical manifestations were suggestive of COVID-19. Appropriate protective measures were used during collection, and sampling was carried out by trained personnel. One sample was used to detect the SARS-CoV-2 antigen (processed in situ in laboratories designated for this purpose on the Isle of Youth, following manufacturer instructions), and one was used for RT-PCR, kept at -20 °C, and sent to Cuba's National Reference Laboratory at the Pedro Kourí Tropical Medicine Institute (IPK) in Havana for processing.
Ethics This study was approved by the municipal ethics commission and written informed consent was obtained from all participants.

RESULTS
We obtained rapid antigen tests and RT-PCR tests for SARS-CoV-2 for all 778 individuals. Of the total 778 sample-pairs, 70 (9.0%) were positive and 658 (84.6%) negative by both tests (Table 1).
Sensitivity was high (75.3%; 95% CI = 66.0-84.6) but lower than specificity (96.1%; 95% CI = 94.5-97.6) ( Table 2). According to the positive predictive value, we can estimate a 72.2% probability of viral infection if the test is positive. If, on the other hand, the test is negative, there is a probability of 0.97% of not being actually infected. Estimated prevalence was 11.9% and the validity index was 93.6%. Concordance between the two tests, given by the validity index, was remarkably high, as are negative predictive value and specificity (Table 2).

DISCUSSION
This study was carried out in an unfavorable epidemiological context, during the second wave of the COVID-19 epidemic on the Isle of Youth, when the incidence rate reached 54.5 cases per 10,000 population-the highest in the country.
Many factors favored eventual epidemic control, but rapid antigen testing facilitated testing practically in real time-the utility of which was documented 6 months later when the territory reduced its cumulative incidence rate to 7.7 cases per 10,000 population; an average of 10 cases per month, all of which were sporadic and most of which were detected at the territory's points of entry. These results made the Isle of Youth an interesting example among municipalities concerning pandemic control actions and results.
The rapid antigen test's sensitivity and specificity were lower than those reported by the manufacturer, likely due to differences between the ideal conditions in which manufacturers test and validate their products and conditions prevalent in the field. However, our results are similar to those of other evaluations, including those of a study carried out in Mallorca, Spain, which included patients with symptoms suggestive of infection in remission reported by family physicians, or patients with previous contact with infected individuals whose infections were confirmed by RT-PCR, in which the Panbio (Abbott Rapid Diagnostic Jena GmbH, Jena, Germany) test's overall sensitivity and specificity were 71.4% and 91.8%, respectively. [10] A study of symptomatic patients in the Netherlands based on the Roche/SD Biosensor rapid antigen test showed higher sensitivity and specificity than this investigation, at 84.9% and 99.5%, respectively. [11] BinaxNOW's rapid antigen test [12] had very high specificity in both adults and children (at 100%) and high sensitivity in adults with recent symptoms (96.5%).
Rapid antigen testing under tents in a plaza in an urban environment, specifically San Francisco's Mission District (California, USA)-a setting of ongoing community transmission-reported 100% sensitivity and 97% specificity in adults who were ≤7 days from symptom onset. [13] Another investigation using this test conducted in Oshkosh (Washington, USA) demonstrated a sensitivity of 78.6% and a specificity of 99.8% in 1188 symptomatic patients after ≤7 days of clinical evolution. [14] In Germany, the performance of a rapid antigen test (RAT) Viva-Diag SARS-CoV-2 Ag Rapid Test Device (VivaCheck Biotech [Hanghzou] Co., Ltd., Hangzhou, China) in everyday clinical practice was assessed in all hospitalized patients at the Helios University Hospital Wuppertal, as well as their accompanying relatives at the Children's Hospital, resulting in a sensitivity and specificity of 27.5% and 99.6%, respectively. Sensitivity varied by group and was higher in symptomatic patients (52.9%) than in asymptomatic patients (20.6%), while specificity, at 99.6%, was the same in both groups. [15] This cohort's remarkably low sen-sitivity contrasts with that reported by WHO for the same rapid test (at 75.1%), which was similar to that in our study. [16] A study of patient samples from three hospitals in Pinar del Río Province and IPK in Cuba using the rapid antigen test SD BIOSENSOR ROCHE DiagnosticGmbH found a sensitivity of 80% for symptomatic and 61% of asymptomatic patients, and a specificity of 92% for both groups. [17] Although all studies report high specificity values in the tests' ability to detect antigens, the tests' overall performance varies and appears to be highly dependent on brand and context. In general, rapid antigen tests are much more specific than sensitive, and exhibit better metric performance in asymptomatic cases, as documented by authors in various countries (Austria, France, Brazil, Italy, Chile and India), whose research consistently shows specificity values of 97%-100%. [18][19][20][21][22][23] The vast majority of studies discussed in this paper meet or at least approach WHO performance requirements for rapid SARS-CoV-2 antigen tests (sensitivity ≥80% and specificity ≥97%). [24,25] The results of our study support the rapid antigen test's diagnostic use, at least until SARS-CoV-2 prevalence dips below 2.5%. [11] The use of rapid antigen testing is further recommended in settings where molecular testing is limited or unavailable, or where it is only available with long turnaround times.
[9] This last circumstance was the case on the Isle of Youth, when it was necessary to send samples to IPK, and results could take up to 72 hours.
WHO has updated its information regarding PCR diagnostic testing in the context of the COVID-19 pandemic, and notes "as the prevalence of SARS-CoV-2 infection decreases, so does the positive predictive power of the PCR result." In this study, we estimated 11.9% prevalence in people suspected of infection; which is not considered 'low' but could explain the low positive predictive value. However, infection prevalence in the population during the study period is below the prevalence of expected cases, and if the test is performed on people with no suspicion of disease who had no infected contacts, any positive result has a high probability of being a false positive, [26] so rapid diagnostic tests are not recommended in populations in which expected disease prevalence is low. [27,28] According to a report from the European Center for Disease Prevention and Control (ECDC), rapid testing is recommended to evaluate individuals regardless of symptoms in settings where the proportion of positive tests is suspected to be ≥10%. [27] Several studies consulted [10,29,30]  No diagnostic test is perfect. Despite the good metric properties of antigen-detecting techniques, diagnoses must be corroborated by molecular methods, monitoring compliance and other factors during preanalytical stages to minimize the risk of false negatives.

Study limitations
The main limitation of this study was failure to explore test performance in different scenarios: asymptomatic and symptomatic, including analyses according to cycle threshold (Ct). The study was limited to symptomatic cases and did not stratify the sample according to different clinical evolution times or epidemiological categories, such as direct and indirect contact. Therefore, it was not possible to make any inferences as to the test's applicability in other contexts, like screening populations in circumstances where transmission is expected, in border control, or in workplaces, all of which have different vulnerabilities.
Further research is merited to explore test performance in asymptomatic and symptomatic individuals according to Ct values and epidemiological and clinical strata.

CONCLUSIONS
The use of various diagnostic techniques has become increasingly common in the pandemic scenario. This study found index values that validate SD BIOSENSOR's rapid antigen test's use for diagnostic purposes for prevalence values ≥2.5%, with acceptable sensitivity and positive predictive values, high specificity and negative predictive values, and high validity indices similar to those found in other studies carried out in conditions of high SARS-CoV-2 infection prevalence. Moreover, the use of this test on the Isle of Youth was decisive in the clinical-epidemiological management of the epidemic.