Balancing Data and Perceptions

Balancing Data and Perceptions by David Whelan I don’t generally have to see something to believe it but, in my work life at least, I prefer to rely on measurements than people’s stories, given the option. When I was in a help desk role, and later as a reference librarian, I frequently found that what a person told me was not quite what they meant. Sometimes they said things that weren’t correct because we weren’t communicating clearly, and they would say they had done X but in reality, they hadn’t. Sometimes they were asking for one thing but they really wanted me to give them something else. This is particularly true when dealing with people who have expertise in one area (lawyers, for example) and think that the expertise extends into other areas. I’ve recently had two experiences where I have data to show the rationale for doing one thing, and competing anecdotes from external experts telling me why I shouldn’t. It’s difficult to balance those two.

Or perhaps not!  I’m sure there are many people – and I often initially react this way – who would say that the data trumps the anecdotes.  It’s not always that easy, though.  Data is often only part of the picture, whether it’s usage statistics or sales figures.

Telling Stories

Anyone interacting with library management will have experienced this.  Circulation statistics are down (or up), electronic database usage is up (or down) or reflects unexpected trends, foot traffic is down (or up).  Data can measure many of the activities that we experience in libraries.  It doesn’t measure the value of what we do, just the activity itself.

At the same time, you may have spoken with a librarian or manager or trustee who can tell you a story about how busy the library is (when foot traffic is trending down) or how important certain databases are (when electronic usage shows that other databases are more heavily used).  They tell the story honestly, sincerely, and yet the details do not jibe with what seems to be measured.  Like any story, there is often a motive behind it.  Sometimes it is political or to justify something that is not supported by data (and, perhaps, is not measurable in that way).  These stories represent someone’s wish to explain the space around the data, the intangibles that they think are important but not (or incapable of being) measured.

What About the Data?

Stories that conflict with data are problematic, though.  Well-meaning anecdote can mean investment in ineffectual services or expensive resources.  If customers or staff are saying that X is one of the most important resources and fight against its cancellation, but usage statistics show it’s never used, what is the right answer?  Are there times when the data can be wrong?

I don’t think so.  I find data about library usage to be far more accurate a representation of what is happening in a library than by the anecdotal reports of what is happening.  It doesn’t mean that the data trumps everything else, but it becomes a baseline for discussions.  It should certainly act as a barrier to doing something that’s obviously contrary to what the data suggests.

My Kingdom for a Database

Here’s an example.  You’ve got an electronic database that gets very low usage but is relatively high cost.  You’ve also licensed a second database with similar content, similar low usage, but much lower cost.  Cancelling the first database is an easy call.  But if the outraged response calls for its reinstatement, should you?

First, you might analyze why you have an outraged response.  It may not have anything to do with the database.  Or rather, it may have far more to do with you goring someone’s ox and impacting their autonomy or comfort level, rather than their access to information.  You may need to add a better communication strategy to your future cancellation decisions, but otherwise stick to your guns.  Practice will acquaint you with how to anticipate reactions that are disconnected to the database content and represent some other emotion or basis.

Sometimes the response is based on something relatively complex.  My own experience with law libraries provides a good environment in which to see this.  There is a particular process of legal research that has been followed since year dot.  I’m supportive of it as much as any:  use a secondary legal resource in the relevant area, follow its leads to primary law, and then update.  For some, that is less a process than a foundation for life.

It contrasts with the decades-old complaint that new lawyers “don’t know how to research”.  Sometimes that complaint arises because the process outlined above isn’t followed.  These brash upstarts jump right to the second step and “overlook” print resources.  They head directly to online databases and type in a search query, looking for matching cases.

In fact, the data I see on law library usage supports exactly this pattern:  heavy electronic primary usage, light electronic secondary usage.  The conflict between what is expected of the legal researcher, what they appear to be doing, and what the data shows they are doing, is causing a lot of problems for law librarians.  For me, it begs the question about why we haven’t aligned our expectations for how lawyers should be researching with what we know about how lawyers are actually researching.  The cost of legal research is so high – and getting higher – that licensing expensive content based on anecdote and not on data can be a significant strategic error.  It is particularly difficult to sift this decision out when your own preferences may be conflicting with the data on which you must rely.

In the end, if your data supports your decision and the overall benefit (cost, etc.) outweighs even perceived negatives, batten down the hatches for awhile.  Once you’ve weathered the storm, re-evaluate your process for the next time and see how you could handle the cancellation and communication better.

But I MUST Be On the Home Page!

Another example includes the department or individual who equates being represented on the home page with operational success.  Failure to have that link will mean virtual invisibility, and a tumbling into an abyss of ignominy.  Web analytics are pretty clear, though.  Not only can you track usage, as with many library services, you can follow users around your site and see how they enter and exit.

Again, there is a communications issue here.  The first step is probably to understand yourself and then help others understand that your home page is probably not the first page th

at your visitors are seeing.  Search engines are sending visitors directly into the middle of the site and having a link on the home page may or may not have any impact whatsoever.

More importantly, though, is to try to communicate the difference with what the internal department wants and what the organization’s customers are showing they want through their Web site usage.  The sad reality for many of us – not just libraries, but lots of departments in many organizations – is that we aren’t all going to be the most important content on a Web site.  Some things are more important, and more core, to the organization’s mission and customers than other things.

Deal with it.  If the home page is intended to be a quick entry point for highly sought after content and a source for high visibility content, and your department doesn’t represent either of those, then don’t try to horn your way onto the home page.  You are better off trying to raise your profile by aligning yourself more closely to what your customers seek, or honing the content you have on the Web site for those customers who do value it.  For those of you from a business or commercial environment, it may seem crazy but non-profits and other non-commercial environments may have initiatives and other activities that need to be on the Web site but are never going to be a significant destination for visitors.  A commercial site would drop that content and redeploy the resources to other areas.  That redeployment can mean that low use content is always being eliminated.  Non-profits and others may have lots of content in the low use “long tail”, though, and it probably shouldn’t all be represented on the home page.

If you’ve got usage data that indicates what your high use (and, I’d say, high value) content is, that should trump most internal demands for visibility.  Your customers should be the more important measure, rather than the internal esteem of any given department or individual.

Balancing the Data

In the end, the data is a strong foundation.  But there is always a grey area around it and it’s there where the anecdotes and perceptions take hold.  Sure, you’ve got a “top 10” list of the most sought-after content.  But in reality, you’re going to dig further down your list to highlight some content or resources – they might hold promise, they might be more popular based on time of year, they might need more promotion to get higher usage data – and balance your decisions with the political and other realities of your organization’s needs.  The benefit of data is not in providing a bright line for decisions but by providing a barrier to doubling back into bad decisions that can be a drag on your organization.

Document Actions
David Whelan

I improve information access and lead information teams. My books on finding information and managing it and practicing law using cloud computing reflect my interest in information management, technology, law practice, and legal research. I've been a library director in Canada and the US, as well as directing the American Bar Association's Legal Technology Resource Center. I speak and write frequently on information, technology, law library, and law practice issues.