Abstract
External Quality Assessment (EQA) is recognized by international standards (ISO), and EQA performance is used by accreditation bodies as a tangible measure of the quality of a laboratory’s performance. EQA allows an independent appraisal of the laboratory’s results and a comparison of these to validated results and performance criteria. The aim of EQA is to be educational and to improve quality of care where a laboratory is failing to meet the required standard. EQA is also an external verification of quality of service and gives confidence to the laboratory director and host institution that the laboratory’s performance is satisfactory.
Cell-free DNA-based noninvasive prenatal testing (cfDNA NIPT) for aneuploidy is an effective prenatal screening test for the common trisomies (chromosomes 13, 18, and 21). A survey of current practice, including reporting for cfDNA-based NIPT and noninvasive prenatal diagnosis, was undertaken in order to provide an overview of the needs and approaches employed by laboratories across the globe and to develop a plan for the pilot EQA(s) to meet the needs of participants. This survey enabled the EQA providers to implement and provide a cfDNA test pilot EQA program to participants. Initially, artificial samples were sent out but due to the limitations of this material, subsequent EQA rounds have used patient material. This chapter discusses the outcome of the survey and the initial EQA distributions.
Keywords
External Quality Assessment, Proficiency testing, Cytogenomic External Quality Assessment Scheme, European Molecular Quality Network, UK National External Quality Assessment Service, Cell-free DNA-based noninvasive prenatal testing, Fetal sexing, Noninvasive prenatal diagnosis
Introduction
External Quality Assessment (EQA), which is also referred to as proficiency testing (PT) in some countries, is a means of assessing clinical analytical and interpretation performance with interlaboratory comparison and against international standards (ISO). EQA allows an independent appraisal of the laboratory’s results and interpretation compared to the validated results and to the performance criteria. The aim of EQA is to be educational and to improve quality of care where a laboratory is failing to meet the required standard. EQA is also an external verification of quality of service and gives confidence to the laboratory director and host institution that the laboratory’s performance is satisfactory. However, if the performance is poor, it will alert the laboratory to investigate and perform a root cause analysis to identify and subsequently rectify the problem; participation in an EQA may also enable a laboratory to identify areas of concern before an error leading to patient harm is made. EQA can be qualitative (presence of absence of a variant) or quantitative (uses arithmetic parameters to set limits of acceptability) but should reflect the nature of the testing. In genetics and specifically cell-free DNA-based prenatal testing and prenatal diagnosis, the test results are qualitative. The EQAs are designed to check and sometimes challenge laboratory screens and tests for a particular genetic disease/disorder or gene/target combination. Ideally the EQA covers the whole preanalytical, analytical, and interpretation of results process through the distribution of the relevant clinical samples, although for very limited or rare genetic samples, the EQA provider may only offer an EQA for part of the process, for example, analytical and interpretative.
EQA is recognized by ISO and consequently, accreditation bodies use EQA performance as a tangible measure of the quality of a laboratory’s performance. The ISO15189 standard requires laboratory participation in EQA(s) for all of the diagnostic services offered and this participation should be continuous for all aspects of the service.
There are many benefits to EQA. Firstly, it is educational and may highlight deficiencies and/or improvements needed to the diagnostic service. EQA is also useful for benchmarking a laboratories’ own performance. Over the years Cytogenomic External Quality Assessment Scheme (CEQAS), European Molecular Quality Network (EMQN), UK National External Quality Assessment Service (UK NEQAS) for Molecular Genetics, and other EQA providers have seen improvement in the quality of analysis and reporting in genetics laboratories over time through regular EQA participation . EQA also identifies gaps in the Quality Management system or in installation of updated software packages thus enabling laboratories to improve their service provision. In addition, the delivery of EQA can highlight where there is a lack of Professional Guidelines and EQA providers are proactive in collaborating with the professional bodies to publish new guidelines thereby assisting laboratories further in reporting genetic results . Guidelines are established based upon the opinion and experience of experts working within the field of the subject, for example, cell-free DNA (cfDNA)-based noninvasive prenatal testing (NIPT) and noninvasive prenatal diagnosis (NIPD) . Consensus opinion is essential as it allows different points of views to be expressed and evidence to be heard in a mutually beneficial way to establish best practice. Finally, EQA providers occasionally identify flaws in commercial kits and software, which may either result in the withdrawal of the kit or an improvement of the kit/software, and as a consequence improves the accuracy of the patient test result (unpublished data).
Developing New EQA for CELL-FREE DNA-Based NIPT
Having demonstrated the benefits of establishing EQA in this field and ascertaining the need from the diagnostic community, EQA providers have been developing assessments to address the unique challenges of cfDNA NIPT EQA.
Problems Associated With Developing Materials for Plasma-Based EQAs
The major challenge for any cfDNA NIPT EQA is to provide sufficient material that is of a suitable quality and has the relevant genetic aberration; an ideal EQA would provide every EQA participant with a batch of material which is identical to the type of sample they normally receive for routine testing (i.e., the same material type provided in the same quantity, collected, stored, and transported in the optimal manner). To allow interlaboratory comparisons the material provided to all participants should ideally come from a single or small number of batches validated by two independent laboratories. The EQA provider also needs to ensure that the samples have not deteriorated in the interval between validation and use by the participant. Finally, the EQA provider must consider the cost of the materials and transportation requirements which might affect those costs. For plasma-based EQA there are difficulties in all of these aspects and the provider therefore needs to carefully consider the advantages and disadvantages of both real clinical samples and artificial reference materials.
Using patient-based clinical material has the main advantage that a result can be obtained by any relevant methodologies and is therefore applicable to all possible participants; patient-based material represents the gold standard being truly representative of what a laboratory is likely to encounter day to day. However, for cfDNA NIPT where laboratories usually require approximately 4 mL of plasma to complete testing it is not possible to obtain the amount of donated plasma required for all participants from a single pregnant patient. Hence there will be a need to use multiple samples in the EQA, limiting the reliability of interlaboratory comparisons. Additionally, any validation requirements prior to distribution further limit the volume of sample available for participants to test. As cell-free DNA in plasma is very susceptible to degradation during handling, transit time/conditions and freeze/thawing are all issues which will require careful consideration to ensure samples arrive in the testing laboratory in an optimal condition. In order to ensure stability of real samples it may be necessary to consider low temperature distribution increasing the costs and logistical difficulties of exporting the materials. Finally, limited availability of materials from pregnancies with the rarer trisomies, for example, Edwards/Patau syndrome is an additional limiting factor for including these into the EQA.
Artificial reference materials have the main advantage that large quantities can be manufactured and validated prior to distribution. When designing an artificial material, it might also be possible to include an artificial freeze-dried plasma component, providing a more stable sample. In this case handling and shipping problems/costs may be reduced. As stated previously having a single batch of material removes variables, leading to more reliable interlaboratory comparisons. Another benefit of artificial materials is the ability to engineer specific characteristics of interest into the sample, for example, in a Chinese cfDNA NIPT EQA the effect of low fetal fraction was investigated by limiting the amount of sheared “fetal” DNA in the plasma . However, artificial materials are not identical to real samples and any laboratory obtaining a poor performance will always question if their performance was due to differing characteristics in the artificial sample. For example, firstly mechanical shearing of DNA produces fragments which differ than those found in maternal plasma with smaller DNA fragments (< 100 bp) . Secondly, the fragments produced by mechanical shearing may be problematic for paired-end NGS approaches. Thirdly, at the current time, commercially available artificial reference materials are not matched for the parent/child relationship, that is, the “maternal” and “fetal” components are not from related individuals, and therefore SNP analyses are not possible. For some of these problems there is potential to overcome these deficiencies, for example, by providing patient materials or producing artificial materials that include matched mother/“fetal” DNA to allow SNP analysis. One future approach might be to include both real and artificial materials in an EQA to access the benefits of each type of material.
Background to the Joint EQA
“NIPT” is the term used for a cell-free DNA (cfDNA) based prenatal genetic screening test while NIPD is the term used for a rapid cfDNA-based prenatal genetic diagnostic test. NIPT and NIPD techniques are both based on the analysis of circulating cfDNA within maternal plasma. The cfDNA is derived from the placenta in early gestation and at/after 10 weeks’ gestation between 3% and 20% of the cell-free DNA in maternal plasma is of “fetal” origin . The fetal fraction generally increases with advancing gestational age but is also affected by maternal weight and multiple pregnancies .
Cell-free DNA-based NIPT for aneuploidy has been shown to be an effective prenatal screening test for the common trisomies (chromosomes 13, 18, and 21) and initially was introduced through the commercial sector and private healthcare providers . Some of these additionally screen for sex chromosome aneuploidies and for other chromosomal rearrangements, for example, microdeletion syndromes. Although sensitivity and specificity is high for trisomy 21, the test performance for other aneuploidies (involving chromosomes 13 and 18, X & Y) is more variable . cfDNA-based NIPT is a screening test and so false positives and negatives can occur. Consequently, a high-risk cfDNA NIPT result should always be followed up with a diagnostic invasive test, for example, chorionic villus sampling or amniocentesis. In addition, current cfDNA NIPT does not identify the range of chromosome abnormalities detected by microarray analysis of placental or fetal cells obtained following invasive testing . In contrast, cfDNA-based NIPD is a diagnostic test used for sex determination, and a range of monogenic disorders and does not require any additional invasive testing.
cfDNA NIPT results may be discordant with diagnostic prenatal tests due to a variety of parameters including: a statistical/technical false positive/negative, low fetal fraction, confined placental mosaicism (vanishing), twin pregnancies, maternal mosaicism and in rare circumstances even maternal malignancy . This is discussed in more detail in Chapter 5 . Most performance data reported to date only relates to singleton pregnancies; however, there is increasing data becoming available for twin pregnancies . NIPD is used for identifying male fetuses, Rhesus D status, and other disorders where the presence of chromosome-specific sequences (e.g., SRY sequences or a paternal autosomal recessive or dominant mutation) in the mothers’ blood can only be of fetal origin. False positives are rare but may occur, for example, if a previous pregnancy was male and there is still cfDNA circulating in the maternal blood. The challenges of delivering EQA for NIPD are discussed in Jenkins et al. .
Current EQA
Fetal Sexing From CELL-FREE DNA in Maternal Plasma (EMQN and UK NEQAS for Molecular Genetics)
A pilot EQA for noninvasive prenatal diagnosis (NIPD) for fetal sex determination was introduced by EMQN and UK NEQAS for Molecular Genetics in 2013 and ran for 3 years with up to 46 laboratories participating in the EQA, each receiving 3 samples to test. Although genotyping error rates were always low (e.g., 0.7%, 1/138 genotypes in 2015) it was noted that there was a considerable improvement in the quality of submitted reports over the duration of this EQA. It is noteworthy that when this EQA was designed, a majority of laboratories were performing testing based on the detection of Y chromosome markers (e.g., using real-time PCR) and confirming the presence of “fetal” DNA in the sample (e.g., RASSF1A assay). Since that time there has been an expansion in the number of laboratories analyzing free “fetal” DNA using Next-Generation Sequencing (NGS) or using a single-nucleotide polymorphism (SNP) based approach. In designing this EQA it was found, despite extensive testing, that no synthetic material performed adequately for inclusion in the EQA. Consequently, to ensure all participants received material of similar quality, pooled plasma samples were used. Although this approach worked well at the beginning, an increasing number of participants were unable to use the pooled plasma samples as their methods relied on comparing maternal/fetal SNP data. Additionally, there was an increasing demand for a cfDNA NIPT EQA for trisomy testing and the decision was taken to temporarily suspend the EQA on fetal sexing to allow the development of the 2016 cfDNA NIPT pilot EQA for trisomy testing. However, participants subsequently recognized the need for a continuing specific fetal sex determination EQA for laboratories performing sexing for X-linked genetic disorders therefore the EQA was reinstated during 2017.
Pilot EQA for CELL-FREE DNA-Based NIPT for Trisomy 13/18/21 and Sex Chromosome Aneuploidies (China)
An EQA for cfDNA-based NIPT for trisomy 13/18/21 and sex chromosome aneuploidy was introduced in The People’s Republic of China in 2014 . In this pilot EQA artificial materials (13 in total) were created using either cell lines or placental DNA from pregnancies as the source of aneuploidy. Fragmented DNA from the cell lines/placental DNA was sonicated and mixed with DNA from healthy female donors. In this EQA only 55% laboratories correctly identified all of the samples, although there were no false positive results reported. The problems arose because this EQA included challenging samples which had simulated fetal fractions below the 4% quality threshold value recommended for many assays ; there were a number of laboratories who reported false negative results after failing to identify the low fetal fraction. The authors did note that there were a number of limitations of the materials used: mechanical shearing of DNA necessitated end repair for use in some assays; the material could not be used by laboratories using SNP-based assays as the female donor (simulated maternal plasma) and the simulated “fetal” DNA were not from related individuals, the mechanically sheared DNA samples (unlike real materials) were deficient in fragments < 100 bp and the plasma from nonpregnant females yielded shorter fragments than those generally seen in pregnant females.
Joint Pilot NIPT EQA
In 2016, a pilot EQA for cfDNA-based NIPT for common aneuploidy was introduced as collaboration between three EQA providers: CEQAS, EMQN, and UK NEQAS for Molecular Genetics. The plan was to learn from the experience of existing EQAs, to survey laboratories to understand current practice and to facilitate the writing of Best Practice Guidelines to improve clinical practice and to allow the development of EQA marking criteria.
NIPT/NIPD Service Provider Survey
In order to provide an overview of the needs and approaches employed by laboratories across the globe and to develop a plan for the pilot EQA(s) to meet the needs of the participants, a survey of current practice, including reporting for cfDNA-based NIPT and NIPD, was undertaken.
In February 2016, approximately 3000 laboratories registered with CEQAS, EMQN, and UK NEQAS for Molecular Genetics were invited to complete an online survey focused on services provided for cfDNA NIPT and NIPD. Many of those do not perform cfDNA-based NIPT and therefore were less likely to respond.
One hundred and twenty-one laboratories responded (5% return rate). Four of these laboratories offered a service for NIPD but were not included as one only offered NIPD for achondroplasia, and two only for fetal Rhesus D genotyping and one was currently validating the NIPD service. Seventy-five (62%) of the responding laboratories performed cfDNA NIPT for aneuploidy at the time of the survey, and 43% of the responding laboratories submitted sample reports. A further 25 laboratories were in the process of validating a cfDNA NIPT assay and 17 were offering cfDNA NIPT on a research basis.
The volume of blood laboratories stated to require for undertaking cfDNA NIPT/NIPD testing varied considerably: from 4 mL to a total of 30 mL of blood. The majority of laboratories (43%) required 10 mL of blood and a further 40% of centers required < 10 mL. The volume of plasma required was more consistent with most centers (86%) requiring 5 mL or less. Centers used either Streck tubes (76%) or EDTA tubes (24%) for collecting the blood sample and the vast majority of centers received these blood samples via a courier.
Of the laboratories delivering a cfDNA NIPT service, 19 centers performed up to 50 tests per month ( Fig. 1 ) and a small number (eight) of centers > 1000 per month. The methods used varied: the majority (74%) were based on NGS-based counting methods, 11% used the SNP-based sequencing approach, and 15% used other methods including Sanger sequencing, array based, MLPA, and TaqMan. Of the centers that used an NGS platform, 70% followed a whole genome approach rather than a targeted one. Six different NGS platforms were used and included Illumina NextSeq, HiSeq, and MiSeq and ThermoFisher Ion Proton and Ion Proton Personal Genome Machine (PGM). The aim of this type of testing is to deliver quick accurate results therefore it is important to monitor turnaround times. This TAT varied from 3 to 21 days with the majority reporting within 14 days ( Fig. 2 ).
