Ages ago, I found myself debating the value of anecdotes with a friend.

He argued that anecdotes should never be used because they can just as easily represent outliers as norms; I argued that they were extremely valuable as vehicles — people remember stories better than they remember reams of data.

I now realize that we were arguing at cross-purposes. He was arguing that anecdotal evidence should not be used to confirm or deny research hypotheses (a position on which we actually agree); I was arguing that anecdotes have a place in explaining the findings of research to people who don’t necessarily know a great deal about statistics and levels of measurement and all that jazz. I have no idea how he feels about that. I’ll have to ask the next time I see him.

It is true that individual anecdotes can’t tell us much about how the world actually works: if we only hear one story, we can’t glean from that single story whether that story is typical or atypical. Therefore, we can’t base statistical analyses on small samples of individual anecdotes, and we can’t make sound statements about causality or even, really, about correlations based on small samples of individual anecdotes.

When we try to ascribe causality based on anecdotes, we run into problems: for example, a book detailing how the use of Applied Behavioral Analysis (ABA) led to one child’s “recovery” from autistic spectrum disorder (ASD) does not actually mean that ABA can produce the same results for all, or even most, kids (or adults) with ASD. In fact, most of the time, it doesn’t (this isn’t to say that ABA isn’t a valuable tool; just that it’s not usually a miracle cure) — but reading one or two books recounting the story of one or two kids who “recovered” can lead to the impression that ASD is “curable” in most or all cases and “should” be cured using ABA(1).

If, as a parent or helping professional, you read only that book, or those two books, and you decide that they represent a typical view of the world, you’ll have based your entire understanding — your entire statistical analysis, informal though it may be — on an anomaly, but you won’t necessarily know it.

A research paper, meanwhile, that looks at a sample of a couple of thousand folks with ASD and examines outcomes of their experiences with ABA will undoubtedly produce a different picture of the efficacy of ABA and the “curability” of ASD. From such a paper, we’ll be able to get a better sense of the relationship between ASD and ABA: does ABA lead to improved outcomes(2) for people with ASD? Such a study will, ideally, also answer a few other important questions:

• How often do improved outcomes occur?
• How much improvement are we talking about?
• How are we even defining “improved outcomes?”

If a study is sufficiently well-designed and well-controlled, we might even be able to draw some inferences about causality(3).

Unfortunately, the results of the research in question will most likely be published only in academic journals and textbooks, only a few of which will actually be accessible to the general public, only a few of whom will actually have the knowledge to interpret the results(4).

Thus, the likelihood that the average parent of a young child with ASD will be able to lay hands directly on sound research is much smaller than the likelihood that the same parent will be able to read a popular, anecdotal book.

This is where the power of the anecdote comes in handy: as researchers and as helping professionals, we have access to data that can help us to convey information to a broader audience — and we can select anecdotes that do reflect statistical realities. We also, incidentally, are often people who are interested in stories: research, to some extent, is about figuring out the “who (even if the who in question is, for example, a chemical messenger or an invisible physical force), what, when, where, why, and how” of things. In short, results can be translated into anecdotes.

Those anecdotes, in turn, can help people feel connected to the subject at hand, which is immensely important. A bunch of dry statistics about ABA and ASD won’t really influence how most people feel about whether ABA is an important and useful intervention or not; a true story, on the other hand, will — but with that power comes responsibility. When they’re used to reflect the realities revealed by careful research, anecdotes should be told in a way that reflects those realities; a way that reflects a typical (that is, an average) experience(5).

We’ve probably all seen television ads for weight-loss products with “RESULTS NOT TYPICAL” emblazoned across the bottom of the screen. Unfortunately, biographical and autobiographical books recounting anecdotes about recovery from neurological and psychological conditions aren’t required to carry those labels.

We probably can’t (and, for various reasons, probably shouldn’t) force them to — but we can find ways to tell harness the power of anecdotes to present results that are typical, so potential health-care consumers have a better shot at making informed choices.

Notes

1. I don’t agree with either of these assertions, by the way; nor do I agree that the ability to function just like a neurotypical person in the neurotypical world is necessarily a desirable or reasonable goal. For some of us, it might be. For others, it’s not.
2. “Improved outcomes” as operationally defined by the study — I guess operational definitions are a subject for another post.
3. though this is notoriously hard to do in the field of psychology as a whole due to difficulties both with ethics (we’re not generally allowed to dissect people after applying a drug intervention to see what happened to their brains, nor can we keep them isolated in stimulus-free environments, etc.) and the complexity of human subjects (even if we could keep humans isolated in stimulus-free environments, that alone could become a confound, etc).
4. In short, this is a skill that’s usually taught at the university level, and then primarily to students in disciplines with strong research components. In the US, that amounts to a fairly small subset of the general populace.
5. I think it’s okay to mention the outliers as well — those atypical results that look so great, or so awful — but we must do so with the knowledge that, on the whole, most of us expect our individual case to be an exception, and moreover, to be a good exception: one with results better rather than worse, than is typical. In short, we need to present outliers with several grains of salt, and we need to balance the better-than-typical outcomes by also presenting the worse-than-typical ones.