Diagnostic tests: the good, the bad, and the ugly

Unlike Hollywood films, the good, the bad, and the ugly in diagnostic testing can have real-life ramifications on patients. Photo: s_bukley/Bigstock Veterinarians have an enormous selection of tests available to help us diagnose and manage disease in our patients. As in many other aspects of our lives these days, however, so many choices and so much available information can overload our limited human brains. It seems like more ready access to more information should make our decisions more evidence-based and effective, but that has not proved true. The inability to distinguish data from opinion and reliable information or sources from misinformation and propaganda is causing great harm in our society generally right now. The necessary critical thinking skills and techniques for recognizing and minimizing the impact of bias on how we seek and use information don’t occur naturally; they must be learned. Similarly, having a plethora of tests we can run can be as much a curse as a blessing. Instead of improving our understanding and decision-making, the information overload from so many testing options can lead to less effective diagnosis and treatment. There are also external forces encouraging the use of unreliable tests or testing strategies we must be aware of in order to resist them. The fact we assume more information is always better only makes the problem more difficult to solve. I have written extensively on the impact of cognitive biases and other intrinsic features of human reasoning on medical decision-making,1 and also on the subject of how the misuse of diagnostic tests can impede effective diagnosis and treatment and harm our patients.2,3 Learning to recognize and mitigate the flaws in our reasoning, in the use of diagnostic tests as well as in other aspects of life, is a complex and ongoing process. I have included a general bibliography at the end of this article with a variety of resources that have informed my efforts to identify and control bias and error in my own selection and use of information, and which I hope will be useful to you. For the purposes of this quick overview, I will discuss some strategies for selecting and using diagnostic tests and for avoiding unreliable tests or testing practices. Why test? It seems obvious, but the reason we perform diagnostic tests is ultimately to have information that helps us prevent or reduce suffering or death in our patients. There is little value in information, including a definitive diagnosis of a disease, that doesn’t offer us anything to do that benefits the animals we treat. Sometimes, an “answer” of some kind might be psychologically satisfying for us or our clients, but generally we should strive to use diagnostic tests to seek actionable information, not merely the comfort of believing we understand a situation we can’t remediate. Outside of a research setting, clinical laboratory information is not intrinsically valuable, and more information is not always better in terms of establishing an accurate and useful understanding of our patients’ condition. In fact, as I will discuss, information that is collected haphazardly or “just in case” can mislead us and actually harm our patients! In order to make effective use of the testing options available to us, we must have a clear understanding of the tests themselves, including their strengths and limitations. We must also have a rational diagnostic strategy founded in an understanding of the most likely diagnoses and the actions we might take in response to specific test results. Diagnostic testing is not an oracle, which we consult blindly in the hope of replacing confusion with wisdom. It is a targeted process of collecting and evaluating specific information in the context of our existing understanding, and altering that understanding incrementally to better approximate reality and inform our actions. The good The good news is the tests we have to choose from are always getting better and more numerous. New diagnostic tools, from targeted blood biomarkers to PCR to metabolomics, are being developed faster than ever. While this can be problematic if our understanding of how to use these tools doesn’t keep up, overall this proliferation of diagnostic tools is a boon to us and our patients. As a clinician, part of my job is to be informed about new diagnostic testing options and to evaluate which specific tests might be useful or appropriate for which of my patients. There are a number of characteristics we can use to evaluate the potential reliability and value of new tests. Here is a brief list of features to look for when evaluating new testing options: Applicability. Good tests have often been evaluated in patients with natural disease and in populations that resemble those you might use the test for in practice. A new blood test for occult hemangiosarcoma, for example, that was developed and tested using Labrador retrievers and German shepherd dogs in the general practice setting would be a lot more likely to be useful to me than one based entirely on testing with in vitro cell lines or in a mouse model. Limited bias. Ideally, new tests should be validated independently, not just by the company or research group that invented them. Even with the best of intentions and appropriate methods, it is difficult for researchers not to design and conduct studies in ways that confirm existing beliefs. A single study by a group with a vested interest in the outcome should always be viewed skeptically. A test validated by multiple studies and different research groups is likely pretty reliable. Practicality. Clinicians in practice need tests we can actually use. If a test requires a sample that is very difficult to collect, particularly if sampling is painful or risky for the patient, that test will be a lot less useful regardless of its other characteristics. Tests that are very expensive, have prolonged waiting times for results, or are difficult to interpret clearly, are also unlikely to be useful in practice. Many diagnostic tools of value to specialists or researchers in universities are not appropriate for application in general practice. However, many new tests have emerged during my career, which are convenient and affordable and can be used rapidly in the clinic to provide valuable information. Accuracy. This is one of the most important – and most misunderstood – aspects of diagnostic tests. Accuracy is a feature not only of the tests themselves, but of how they are used. Figure 1 illustrates some of the tools clinicians can use to assess the accuracy and meaning of test results. Sensitivity and specificity. Most veterinarians are familiar with sensitivity and specificity, values that compare the performance of a diagnostic test to some gold standard. Sensitivity is the proportion of patients with the disease that correctly test positive, so the higher the sensitivity the less likely the test is to fail to detect an individual with the disease. Specificity is the proportion of patients without the disease that correctly test negative, so the higher the sensitivity the less likely the test is to falsely detect disease in a healthy patient. Sensitivity and specificity are a rough guide to the accuracy of diagnostic tests. Generally, the higher these values the better, though the importance of each depends on the context. A highly sensitive test is good when missing a diagnosis would be catastrophic while mistakenly identifying a disease that is not present would have minimal impact. Higher specificity is especially valuable when a false positive diagnosis is likely to lead to unnecessary additional testing and treatment or significant anxiety and expense, so long as the consequences of a false negative are not too high. Likelihood ratios. More useful to the clinician than sensitivity and specificity are positive and negative likelihood ratios. These are calculated proportions that indicate how strongly a test results alters the probability of a patient being tested actually has the disease the clinician is looking for. A positive LR > 10 or a negative LR<0.1 strongly alters the probability of true disease. If the likelihood ratio is unlikely to be large enough to change our estimation of the probability a patient has the disease we are testing for, then running the test probably serves no purpose. Positive and negative predictive value. While many clinicians understand high sensitivity and specificity reflect in some way the accuracy of diagnostic tests, it is less widely understood these indicators don’t directly answer the question of greatest importance for the clinician: how likely is it that a patient with a positive or negative test actually has, or does not have, the disease we are testing for? The best answer to this question requires not only information about the accuracy of the test but also about the prevalence of the disease in the population of which the patient is a member. In general, the more common a disease is (i.e., the higher the prevalence of the disease), the more likely a patient with a positive test actually has the disease. Similarly, the less common the disease, the more likely a patient with a negative test is truly free of it. The positive predictive value (PPV) is the probability a patient with a positive test actually has the disease, and the negative predictive value (NPV) is the probability a patient with a negative test does not have the disease. Of course, this requires knowing the prevalence of the disease we are looking for, which often we don’t know with any certainty. However, understanding the general principles of PPV and NPV and the importance of prevalence is still useful. We should be skeptical of positive test results when a disease is uncommon and require additional confirmatory evidence before making a confident diagnosis. We can be more confident in positive results when testing for common conditions for which we have other corroborating evidence. This is nothing more than a formal method for applying a general reasoning strategy already familiar to veterinarians. We are trained to consider common disorders before those that are rare (remember the old cliché about zebras and horses?). Establishing a clinical index of suspicion, in which the relative likelihood of the possible diagnoses is assessed based on general knowledge of which conditions are more of less common in a given locale or patient population, is an informal way of estimating prevalence. A lot of very useful, reliable tests have come along in the 20-odd years I’ve been in practice. I make frequent use of rapid in-house tests, for example. In-house laboratory equipment has also gotten better and cheaper, allowing faster results for clinicians and anxious clients. There are also many new tests in development, such as bloods tests to screen for occult neoplasia and biomarkers to assess the impact of aging and anti-aging therapies on health. If these meet the criteria above for utility and accuracy, they will be of great value to veterinarians in promoting animal health. The bad The most common problems with diagnostic tests are really problems with how they are used. There is a myth in veterinary medicine, actively promoted not only by companies selling diagnostic tests, but by many well-meaning experts, that more testing is better medicine. Guidelines often recommend, for example, extensive lab testing before surgery even in young, healthy animals, or regular testing of healthy geriatric patients twice a year or more. However, the evidence from human medicine is clear that targeted testing based on a well thought-out and prioritized differential diagnosis list is far more likely to benefit patients than broad screening of health individuals or indiscriminate use of multiple tests in sick patients.2,3 The now classic examples of this are prostate-specific antigen (PSA) testing in men and mammography in women.4 Both of these tests can detect cancer before there are clinical symptoms. It seems intuitively obvious that this is a good thing, since earlier detection should lead to earlier treatment and better outcomes. Unfortunately, this often turned out not to be the case. Not only do these tests sometimes detect disease that isn’t present (false positive results and misdiagnosis), they can detect disease that actually is present, but will never cause harm (overdiagnosis). Many men actually have indolent prostate cancers that will never progress or cause clinical symptoms. Some studies suggest that as many as 25 percent of the men diagnosed with prostate cancer based on PSA testing would never have known they had the disease otherwise. Yet, the psychological harm of the diagnosis, and the physical harms of further testing and treatment, reduce both the quality and length of life in these men.4,5 Similarly, early detection of breast cancer is only useful if treatment is going to improve survival and quality of life in patients. Depending on the age at which screening in women without mammary masses is done, up to 30 percent of women diagnosed by mammography experience more harm from testing and treatment than they would have experienced from the indolent lesions detected.4,6 Both PSA testing and mammography recommendations have been dramatically scaled back since these harms were recognized. Unfortunately, we rarely have the necessary epidemiologic evidence to assess the benefits and harms of such testing in veterinary patients. Screening (testing asymptomatic individuals) can absolutely benefit patients under the right circumstances: accurate tests, high prevalence, and treatment which is more effective when delivered earlier, and which does less harm than the untreated disease. The problem is with the assumption that it always benefits and never harms patients, which is clearly false. Apart from screening, other common testing practices that are likely to be ineffective or even harmful include testing for very unlikely diseases (a low PPV and high risk of misdiagnosis), testing for diseases for which there is no treatment or other useful action to take (raises cost and owner anxiety without benefitting patient), testing for diseases when treatment is not beneficial or has more risk than benefit (e.g. asymptomatic bacteriuria7), and overinterpreting test results (treating the test results rather than the patient). Such testing approaches result from a variety of causes. Inexperience and lack of confidence in one’s diagnostic ability can result in excessive or inappropriate use of diagnostic tests, and this is likely quite common in inexperienced clinicians. Strong recommendations for screening or excessive testing are also common in the veterinary press and scientific literature. These are usually motivated by well-meaning individuals who are basing their recommendation on opinion and experience, unfortunately, often without supporting objective evidence. However, we have to be honest about the fact diagnostic testing is also a source of revenue, for commercial laboratories and for veterinary practices. Regardless of our best intentions, we are not immune to the types of bias and incentives that influence decision-making in human medicine. Physicians clearly order more tests when they have a financial incentive to do so, and this likely plays a role in diagnostic practices in veterinary medicine as well.8 While I have emphasized overtesting, since this is a common but unrecognized problem, of course, insufficient use of diagnostic tests is also an ineffective strategy. As we get more experienced and more confident in our diagnostic abilities, we tend to rely more on our observations and reasoning than on objective test results. This has the benefit of reduce overuse of tests, but it can lead us to an excessive confidence in our clinical experience and raise the risk of underdiagnosis and misdiagnosis. A rational testing strategy balances the need for judicious selection of tests based on prevalence and clinical index of suspicion with the risks of overreliance on unaided observation and reasoning, prone as this is to many cognitive biases and errors.1 The ugly Most of the problems with how veterinarians use diagnostic tests are rooted in good intentions and honest, if flawed, reasoning. Unfortunately, there is another category of diagnostic testing that infects the field of veterinary medicine, which is less deserving of sympathy. There are many tests available, and marketed heavily to vets and clients, which lack any reasonable evidence showing they are accurate or lead to better patient outcomes. In the most egregious cases, these tests clearly do not work, and it mystifies me there is no meaningful regulatory limitation to marketing such tests. Less egregious, but still problematic, are tests introduced with only limited validation of basic functionality (often conducted by those with a financial interest in the test) and with no clear evidence their use actually benefits patients. The second category often includes new tests that might well be useful, but get marketed prior to appropriate investigation of their clinical value. Commercial laboratories function in a competitive market environment. It is common for them to devise a new test in-house intended to capture market share from a competitor. Such tests are often validated to the extent the accurately measure whatever biomarker they are meant to assess. At this point, they are sold, or even run for free, and without being requested on samples submitted for other purposes. The well-known marketing strategy of giving something away for free to engender trust and reliance before then charging for it is not an appropriate way to promote diagnostic tests until there is strong evidence they actually benefit patients. More disturbing is the proliferation of tests that clearly have no value and can only mislead pet owners and clinicians. Many alternative medicine methods use such tests. Applied kinesiology, dowsing, bioelectric impedance, so-called “energy medicine” used to sense disorders and guide therapy, and many others are typically limited to practitioners of these marginalized approaches.9 However, many unproven or worthless tests or testing strategies are marketed as if they have scientific legitimacy, and they can easily mislead vets and owners. I have written about some examples of such tests before. Saliva tests for food allergies have repeated failed to demonstrate accuracy or clinical value, yet they are widely available and actively promoted. Similar salivary tests supposedly identifying inflammation and oxidative stress are unvalidated and promoted with unproven claims. Tests claiming to evaluate pet health through fecal microbiome testing are a long way from meaningful validation, yet they are actively used to guide diet and other interventions. Hair analysis is also a common approach, which is typically unproven at best and clearly worthless at worst, depending on the intended use. The use of such unreliable tests poses a real threat to the wellbeing of veterinary patients. The risk of misdiagnosis and overdiagnosis is high, leading to inappropriate treatment. The results of such tests often contradict more evidence-based assessments, and this can lead clients to reject appropriate diagnosis and therapy in favor of untested or pseudoscientific treatments. And in an environment where cost often dramatically limits the testing and treatment we can provide for our patients, resources spent on tests that provide misinformation are tragically misused. Rational use of diagnostic tests The strategies discussed above for evaluating diagnostic tests should help veterinarians recognize and avoid such practices. Published validation data collected in an appropriate patient population under controlled conditions, ideally with independent corroboration from researchers without a personal or financial investment in the tests being evaluated, should be the minimum requirement rather than the seldom-achieved gold standard for supporting widespread use of diagnostic tests. Critical, skeptical evaluation of claims and evidence are every bit as important in considering use of specific diagnostic tests as in choosing which therapies to offer. Figure 1. Tools for evaluating tests and interpreting test results. (From McKenzie, BA. Rational use of diagnostic and screening tests. J Sm Anim Pract. Epub before print. 1–6. DOI: 10.1111/jsap.13393) For those interested in more thorough discussion of how to select and use diagnostic and screening tests effectively, or just interested in making their practices more science-based and avoiding common pitfalls of medical reasoning, I recommend the articles and books in the bibliography following the references. Brennen McKenzie, MA, MSc, VMD, cVMA, discovered evidence-based veterinary medicine after attending the University of Pennsylvania School of Veterinary Medicine and working as a small animal general practice veterinarian. He has served as president of the Evidence-Based Veterinary Medicine Association and reaches out to the public through his SkeptVet blog, the Science-Based Medicine blog, and more. He is certified in medical acupuncture for veterinarians. Columnists’ opinions do not necessarily reflect those of VPN Plus+. References McKenzie BA. Veterinary clinical decision-making: cognitive biases, external constraints, and strategies for improvement. J Am Vet Med Assoc. 2014;244(3):271-276. doi:10.2460/javma.244.3.271 McKenzie BA. Overdiagnosis. J Am Vet Med Assoc. 2016;249(8). doi:10.2460/javma.249.8.884 McKenzie BA. Rational use of diagnostic and screening tests. J Small Anim Pract. June 2021:jsap.13393. doi:10.1111/jsap.13393 Welch HG, Schwartz LM, Woloshin S. Overdiagnosed: Making People Sick in Pursuit of Health. 2011. Beacon Press; Boston, MA. Tabayoyong W, Abouassaly R. Prostate cancer screening and the associated controversy. Surg Clin North Am. 2015;95(5):1023-1039. doi:10.1016/j.suc.2015.05.001 Berry DA. Breast cancer screening: Controversy of impact. Breast. 2013;22 Suppl 2(0 2):S73-6. doi:10.1016/j.breast.2013.07.013 McKenzie B. Asymptomatic Bacteriuria--To Treat or not to Treat? Vet Pract News. 2019:38. Bishop TF, Federman AD, Ross JS. Laboratory test ordering at physician offices with and without on-site laboratories. J Gen Intern Med. 2010;25(10):1057-1063. doi:10.1007/s11606-010-1409-7 McKenzie B. Placebos for Pets: The Truth About Alternative Medicine for Animals. 2019. Ockham Publishing. Bibliography Cockcroft, P. D., Holmes, M. A. eds. (2003). Handbook of Evidence-Based Veterinary Medicine. Oxford, UK: Blackwell Publishing Ltd. http://doi.wiley.com/10.1002/9780470690833 (June 11, 2015). Gilovich, T. (1993). How We Know What Isn't' So: The Fallibility of Human Reason in Everyday Life. New York: The Free Press. Kahneman, D. (2011). Thinking, Fast and Slow. New York: Farrar, Straus and Giroux. Kida, T. (2006). Don’t Believe Everything You Think: The 6 Basic Mistakes We Make in Thinking. New York: Prometheus Books. McKenzie, BA. Veterinary clinical decision-making: cognitive biases, external constraints, and strategies for improvement.Journal Amer Vet Med Assoc. 2014;244(3):271-276. McKenzie BA. Overdiagnosis. J Am Vet Med Assoc. 2016;249(8). doi:10.2460/javma.249.8.884 McKenzie BA. Rational use of diagnostic and screening tests. J Small Anim Pract. June 2021:jsap.13393. doi:10.1111/jsap.13393 Schmidt, PL. (Ed.). Evidence-Based Veterinary Medicine. Veterinary Clinics of North America: Small Animal Practice. 2007 May: 37(3).

Cookie	Duration	Description
cookielawinfo-checkbox-analytics	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional	11 months	The cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance	11 months	This cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
JSESSIONID	session	The JSESSIONID cookie is used by New Relic to store a session identifier so that New Relic can monitor session counts for an application.
viewed_cookie_policy	11 months	The cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.

Cookie	Duration	Description
vuid	2 years	Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos to the website.
_ga	2 years	The _ga cookie, installed by Google Analytics, calculates visitor, session and campaign data and also keeps track of site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognize unique visitors.
_gat	1 minute	This cookie is installed by Google Universal Analytics to restrain request rate and thus limit the collection of data on high traffic sites.
_gat_UA-	1 minute	A variation of the _gat cookie set by Google Analytics and Google Tag Manager to allow website owners to track visitor behaviour and measure site performance. The pattern element in the name contains the unique identity number of the account or website it relates to.
_gid	1 day	Installed by Google Analytics, _gid cookie stores information on how visitors use a website, while also creating an analytics report of the website's performance. Some of the data that are collected include the number of visitors, their source, and the pages they visit anonymously.
__gads	1 year 24 days	The __gads cookie, set by Google, is stored under DoubleClick domain and tracks the number of times users see an advert, measures the success of the campaign and calculates its revenue. This cookie can only be read from the domain they are set on and will not track any data while browsing through other sites.

Cookie	Duration	Description
ADV_u_id	3 months 8 days	Unique customer identifier used to track unique ad views and interactions with some ads.
anj	3 months	AppNexus sets the anj cookie that contains data stating whether a cookie ID is synced with partners.
fr	3 months	Facebook sets this cookie to show relevant advertisements to users by tracking user behaviour across the web, on sites that have Facebook pixel or Facebook social plugin.
IDE	1 year 24 days	Google DoubleClick IDE cookies are used to store information about how the user uses the website to present them with relevant ads and according to the user profile.
loc	never	AddThis sets this geolocation cookie to help understand the location of users who share the information.
OAGEO	session	OpenX sets this cookie to avoid the repeated display of the same ad.
OAID	1 year	Cookie set to record whether the user has opted out of the collection of information by the AdsWizz Service Cookies.
test_cookie	15 minutes	The test_cookie is set by doubleclick.net and is used to determine if the user's browser supports cookies.
uuid2	3 months	The uuid2 cookie is set by AppNexus and records information that helps in differentiating between devices and browsers. This information is used to pick out ads delivered by the platform and assess the ad performance and its attribute payment.
_fbp	3 months	This cookie is set by Facebook to display advertisements when either on Facebook or on a digital platform powered by Facebook advertising, after visiting the website.

Diagnostic tests: the good, the bad, and the ugly

Quality testing is more than whether or not you’re feeling lucky

Membership Required