The Pitfalls of Cut-off Values in Real-time, Quantitative PCR: Should Target Density or Amplification Cycle Threshold be used for Interpreting qPCR Results?

Courtney P. Olwagen^1,2*, Sarah L. Downs^1,2, Marta C. Nunes^1,2 and Shabir A. Madhi^1,2
*Correspondence: Courtney P. Olwagen, South Africa Medical Research Council Vaccines and Infectious Diseases Analytics Research Unit, University of the Witwatersrand, Faculty of Health Science, Johannesburg, South Africa, Email:

Introduction

Cycle of quantification (Cq) cut-off values are routinely applied to qPCR to ensure results reproducibility [1] and to control for amplification and fluorescence artifacts [2]. Normally, amplification for a specific target where the Cq is lower than the cut-off value will be interpreted as positive, while amplification with Cq above the cut-off value will be interpreted as equivocal or negative. Concerns have been raised that the use of Cq values that are either too high or low can result in the reporting of false-positive or false negative results, respectively [3]. Further, an arbitrary Cq value does not account for variations in the efficiency of different assay-sets. Assays can be optimised for sensitivity, specificity, precision, accuracy, and limit of detection (LOD); however, differences in the calibration or standard curves remain, resulting in variation in the calculated concentration when applying a single Cq cut-off across multiple assay-sets.

In a hypothetical reaction-set, containing assay-sets A and B, we presume both assay-sets are well optimised, with efficiency of 90-110%, the correlation coefficient >0.98, the LOD equivalent to ≤10-100 copies, inter-and intraassay variations <0.1 standard deviation (SD), and an accuracy ratio within ± 0.1. Assay-A has a calibration curve equation of y = -3.5x + 36.356 and is more efficient at detecting its respective target compared to Assay-B with the calibration curve y = -3.1x + 39.345. If a Cq cut-off value of 35 is applied to both assays, then positive samples with a log10 concentration of <1 unit and <2 units will be excluded from the analysis for Assay-A and Assay-B, respectively. A sample is thereby 100 times more likely to be reported as positive when amplified with Assay-A compared to Assay-B just because Assay-A is more efficient at amplifying its target. These differences have been observed in a study, where up to a 1000-fold difference was reported between different assays detecting SARS-CoV2 [4]. Further, where an algorithm is applied to interpret results within a typing scheme (Table 1), the use of an arbitrary Cq cut-off may incorrectly type a sample as Type 1, due to the higher efficiency of Assay-A where the copy number of the target is close to the limit of detection.

Uniform Cq cut-off values applied across multiple assay-sets hinder our ability to interpret findings from molecular qPCR reaction-sets holistically, and biases toward more efficient assay-sets. While this may be inconsequential when comparing a single target, for example, seasonal influenza within the same setting over time, it compromises our ability to compare qPCR results across different settings where different assay-sets or reagents have been used to amplify the same target. Further, this complicates the comparison of findings for several targets amplified in the same cohort. For example, molecular qPCR methods are currently being optimised to detect over 100 serotypes of Streptococcus pneumoniae to monitor the impact of routine pneumococcal conjugate vaccine (PCV) immunization on serotype distribution [5,6]. Using a single Cq value as a cut-off to interpret these results will not account for variations in the efficiency of the multiple assay-sets targeting these serotypes and would introduce some bias when comparing the prevalence of one serotype to another in the same cohort.

Alternatively, the use of concentration density calculated from the calibration curve would correctly account for variation in the efficiency of assay-sets, as the performance of each assay-set would be normalised to allow for interpretation and comparison of results from different assay-sets. For instance, we have previously published a detection and typing method that included an algorithm applied to three assay-sets to distinguish S. pneumoniae individual serotypes 6A, 6B, 6C, and 6D within serogroup 6. In our method, the algorithm mistyped or failed to detect a small proportion of serotype 6A (3%) and 6B (6%) that were previously detected using standard culturebased methods, as the arbitrary Cq cut-off value did not adjust for assay-set efficiencies. If we re-analyse the same dataset using the concentration density of 1 log10 colony-forming units/ml as a cut-off and apply the algorithms, the method is now able to correctly type 33% (3/9) of 6A previously mistyped as 6B. Further, an additional 50% (2/4) of the samples typed as Serotype 6A and 33% (1/3) as Serotype 6B that were missed when using a Cq cut-off, were correctly typed by using a density-based cut off. This equates to a 7.1% increase in sensitivity for serotype 6A (87.2 and 94.3% for Cq and Density cutoffs, respectively), a <1% increase for 6B (93.9 and 94.7%, respectively), and no change in specificity of our assay when compared with the referent culturebased Quellung serotyping method.

There are, however, some limitations to using a density-based cut-off:

• The cut-off density should not be lower than the LOD for the assay.

• The guidelines for Minimal Information for Publication on Quantitative Real-Time PCR Experiments (MIQE) should be applied together with the linear equations used to calculate density

• Where possible standard calibration curves should be included in each qPCR experiment, or alternatively a Levy Jennings Plot should be constructed to ensure calibrators included on different plates remain in the same range (i.e., they do not vary by more than 2 SD of the average).

Conclusion

In conclusion, Cq values are specific to a particular assay-set and cannot be compared directly between different settings, assay-sets or within a reaction-set due to variations in efficiency. To circumvent this limitation, density should be calculated from the calibration curves and used as cut-off to allow for comparison, as the performance of each qPCR reaction will be normalised.

References

1. Bustin, Stephen A., Vladimir Benes, Jeremy A. Garson and Jan Hellemans, et al. “The MIQE guidelines: Minimum information for publication of quantitative real-time PCR experiments.” Clin Chem (2009): 611-622..

Google Scholar, Crossref, Indexed at

2. Caraguel, Charles G.B., Henrik Stryhn, Nellie Gagné and Ian R. Dohoo, et al. “Selection of a cutoff value for real-time polymerase chain reaction results to fit a diagnostic purpose: Analytical and epidemiologic approaches.” J Vet Diagn 23 (2011): 2-15.

Google Scholar, Crossref, Indexed at

3. FDA. “SARS-CoV-2 Reference Panel Comparative Data”.
4. Wyllie, Anne L, Alienke J. Wijmenga-Monsuur, Marlies A. Van Houten and Astrid A.T.M. Bosch, et al. “Molecular surveillance of nasopharyngeal carriage of Streptococcus pneumoniae in children vaccinated with conjugated polysaccharide pneumococcal vaccines.” Sci Rep 6 (2016.): p.1-9.

Google Scholar, Crossref

5. Dhoubhadel, Bhim Gopal, Michio Yasunami, Lay-Myint Yoshida and Hien Anh Nguyen Thi, et al. “A novel high-throughput method for molecular serotyping and serotype-specific quantification of Streptococcus pneumoniae using a nanofluidic real-time PCR system.” J Med Microbiol 63 (2014): 528-539.

Google Scholar, Crossref, Indexed at

6. Downs, Sarah Leah, Shabir Ahmed Madhi, Lara Van Der Merwe and Marta Coelho Nunes, et al. “High-throughput nanofluidic real-time PCR to discriminate Pneumococcal Conjugate Vaccine (PCV)-associated serogroups 6, 18, and 22 to serotypes using modified oligonucleotides.” Sci Rep11 (2021): 1-11.

Google Scholar, Crossref, Indexed at