Sharpen That Needle? No, I Don’t Think So
A study out in Archives of Neurology this week has generated a lot of interest. For example, the New York Times had a front-page story with the subhead “100% Accuracy in Test for Alzheimer’s” or something similar. The “100% Accurate” part is what caught my eye, and others too. I doubt that a “10% Accurate” headline would have made it to the front page or that the story would have been covered on the major networks. But is the headline true? My first thought was that no, of course not, otherwise the headline would have been “First-Ever 100%-Accurate Medical Test Developed.” 100% is a number we don’t see too much in medicine, particularly in the ambiguous world of neurological diagnosis. A test like that would be an enormously huge deal.
Does the study live up to the hype? Sadly, no. And it certainly doesn’t justify the an editorial entitled “Sharpen That Needle” which appeared in the same issue, written by two doctors who should know better. There is a 100% result found within the study results, but it isn’t the accuracy of the test exactly. It’s the sensitivity. And not the overall sensitivity, but the sensitive among patients with mild cognitive impairment who went on to develop Alzheimer’s. Now in the next paragraph I’ll explain sensitivity, so those of you who don’t want or need a (very) basic statistics lesson can skip over it.
The “sensitivity” of a test tells us how good it is at finding true cases of what it’s looking for. Of all the people who have a certain disease, say, how many will test positive on our proposed test? If 5% of people with the disease test negative (false negatives) then our test is 95% sensitive. The other basic quality of a test is “specificity.” It tells us how many of the people who come up positive on our test actually have the disease we’re testing for (true positives). So let’s say that 10% of the time our test is positive, it’s wrong. Then we have a 90% specificity. You can’t judge a test based on just one of these elements. As an extreme example, let’s say we check if people are breathing to see whether they have Alzheimer’s. Now since we only test living people, all of the people with Alzheimer’s will have a positive test result: 100% Sensitivity, hooray! But if only 3% of the people we test have Alzheimer’s, then we have a 3% Specificity. Boo! Now the test proposed in the article did a lot better than that, but you get the idea.
If you guessed that the problem with the test is in the specificity, you’d be right. About 36% of the normals in the study also tested positive. So our specificity is down around 64%. This is a problem. It’s a problem because if you use this as a screening test in your office, about 1/3 of the people you inform that they should prepare for Alzheimer’s won’t actually get it. Not good. The other problem is that the “accuracy” of the test is better described by the positive predictive value, which depends on the sensitivity/specificity but also the prevalence of the disease. About 20% of people 75-85 have Alzheimer’s. If we use that as the prevalence, I get a 38% PPV for the test. (This is where my statistics starts to get hazy, so please send corrections if needed). That means that if you test positive on this test, and you live past 75, there’s about a 40% chance you’ll get Alzheimer’s. There are just too many false positives for this test to be a good screen.
And the biggest problem with this test for most people won’t even be its accuracy. The big problem will be that it requires a spinal tap. Let me tell you firsthand that lots of people are deathly afraid of taps. They’re really not so bad, but if you think it’s hard to get people in for colonoscopy, forget about spinal taps.
Not to mention the biggest problem of all: if you do get Alzheimer’s there’s not a lot we can do for you anyway. Most doctors would or should question the ethics of testing for an untreatable illness. Now AD isn’t exactly untreatable, but there’s no evidence that the meager treatments we do have change the course of the illness in the long run, or that starting them earlier makes much difference.
If this did become a standard screening test, I’d certainly stand to benefit: as a neurologist I would start a dementia clinic and tap everybody and send lots of big bills to Medicare. And if I thought it would do my patients any good, I’d do just that. In fact, I’d team up with a GI doc and offer a combined colonoscopy/lumbar puncture. It’d be perfect: you’re already lying on your side, sedated, and you won’t feel or remember a thing. Turning 50? Come on in!
But until we have something to offer our Alzheimer’s patients, I’ll postpone sharpening that needle, thanks.