Consumer DNA testing remains a booming industry, but new research released Monday suggests that it has a glaring flaw in its predictive power. The findings indicate that a common method of testing used by many companies is awful at correctly identifying rare genetic mutations that could raise the likelihood of health problems like cancer. As a result, many users could be at risk of a false health scare.
Researchers at the University of Exeter in the UK looked into a testing method that relies on single nucleotide polymorphisms, or SNPs. SNPs are an individual variation of a single nucleotide (the basic building block of DNA and RNA) along the genome. Scientists have developed chips that can test for lots of SNPs at once, which companies commonly use as markers to pin down a person’s recent ancestry as well as their likelihood of having certain traits or developing certain diseases.
These tiny changes are the most common type of genetic variation a person can have, with over 300 million SNPs discovered to date. But some SNPs are far rarer than others. And according to the authors of this new study, published Monday in the BMJ, these chips aren’t so hot at identifying the rarer SNPs.
The researchers analyzed data from the UK BioBank, a research project that’s collected genetic information from over a half million residents in hoping of better understanding public health. Some 50,000 participants in the project have both gotten SNP testing and their entire genome sequenced, allowing the researchers to have a baseline to compare the former’s accuracy against. They also compared data from the Personal Genome Project, originally founded at Harvard University, from 21 volunteers who had SNP testing and their genome sequenced through a commercial company.
When it came to spotting frequently appearing genetic variants, the chips did a great job, with over 99% accuracy when matched to the sequencing data. But the rarer the SNP they were supposed to look for, the worse they performed at confirming it was really there. For variants that are thought to appear in fewer than one out of every 100,000 people (0.001%), the average false-positive rate for these tests was 84%, meaning that it correctly identified the variant existing only 16% of the time. And in 20 out of the 21 volunteers from the Personal Genome Project, the researchers found at least one false-positive result.
“While SNP chips are excellent at detecting common gene variants, they were not designed to test rare variants and it was well known in the genetics research community that they were not the right tool for this purpose,” senior study author Caroline Wright, a geneticist and professor in genomic medicine at the University of Exeter, told Gizmodo via email. According to the researchers, though, theirs is the first study to try estimating this lack of accuracy against the gold standard of whole genome sequencing.
The team’s research doesn’t single out a specific company. But they said they have seen a growing trend of companies using these SNP chips to test for disease-causing rare mutations, including those associated with breast cancer. And, unsurprisingly, there have been case reports of consumers who mistakenly believed they were destined to develop cancer or other serious rare disorders based on their results.
“We did not look at any particular company, but this will be an issue most companies that offer direct-to-consumer DNA testing will need to consider, as most of them use SNP chip technology,” co-author Leigh Jackson, a lecturer in genomic medicine at Exeter, said in an email.
There are safeguards that can be used against potential false positives involving rare variants, such as double-checking the results with a more accurate test. But it’s easy for customers to end up getting fooled if they upload their data to certain third-party sites.
“Although some consumer genomics companies perform sequencing to validate important results before releasing them to consumers, most consumers also download their ‘raw’ SNP chip data for secondary analysis, and this raw data still contain these incorrect results,” Jackson noted.
Other research has shown SNPs in general aren’t that great at predicting people’s future health, even when many possibly predictive markers are combined into a single risk score. One reason is because most health problems are influenced a lot more by than our genetics. If SNP-based tests are also bad at predicting the sort of diseases often caused by rare genetic variants, that should further sink their value as fortune tellers for our health, at least when used by themselves.
“Our research shows that, where rare genetic variants are detected, the results are more likely to be wrong than right, and no clinical action should be taken without the results being verified by other means,” Jackson said.