Data seems so straight forward. I yearn some days to be part of a field that makes more of its decisions using data. It is probably a greener grass on the other side dilemma. There is a lot of power in telling stories. And there is no denying instinct or gut feelings. What happens, though, when we take a story and turn it into what appears to be data?
This Twitter post went by today and caught my eye.
I was curious and clicked through. One of the first attention-getting sentences in this piece says:
That sounds to me like a data point.
When you click through to the source though, you find that this is not exactly what the author of this law journal article wrote. She said (based on my searching the document and finding three references to “two-thirds”):
I’m picking nits, I suppose. But the author in the Washington Post says that “well over two thirds” (which is more than the law journal estimate) of critical cases (a word the law journal doesn’t use) appear. Not estimated to appear, just the word appear. It creates the perception that this is a measured fact.
The Washington Post author is a non-profit executive who is writing a persuasive piece that supports the non-profit’s mission. I totally understand that. And I’m not even sure this is an excellent example of what I have observed, which is the return to anecdote or statements when facts either don’t exist or aren’t clearly defined.
There was a great one that came out of either the 2015 or 2014 ABA Techshow, that was tweeted and retweeted and for which there didn’t seem to be any underlying support. I can’t find it any longer but remember the slow dawning of disappointment as I realized that, despite it being an interesting story, it seemed to be unsubstantiated.
It continues to surprise me that, for all the things courts measure, we can’t get a good handle on how many people walk into court without a legal professional at their side. When we work in an environment where the data isn’t exact, it is probably reasonable to be clear about that.
For what it’s worth, I have been much more skeptical since reading Blur: How to Know What’s True…. I’d encourage anyone who deals with information to read it.