Seek and Hide Irritant

The move to Canada has meant a near constant immersion in new information: new resources, Web sites, facts about people. The fantastic thing about librarians (a.k.a information professionals) is that, if you are tool-oriented, your tools are often portable from one environment to another. For the most part, that has been true here. I spent a bit of time last week trying to hone some search tweaks that would enable me to search a variety of primary law sources. You can imagine how I felt when one of them blocked my search!

Now, I don’t take it personally.  The site maintainers have blocked all robot activity, to stop Google and other engines from indexing the content that the maintainers have created.  How?  By using the noindex, nofollow, and noarchive attributes (here’s an official Google blog posting about the exclusion protocol).  But since the goal of this and similar sites is public access to information, you would think that they would LIKE to have major engines finding their content.

No.  And one day I’ll find out why – when I’m no longer the new kid on the street – because it’s a weird twist.  I guess I expected that when there was a large and otherwise-public data store available (you can go to the site and search it directly), the content providers would be happy to have remote searches executed on the site.

So for now I am limited to having this particular site as a custom Firefox search (using the Opensearch plugin) and cannot use tools like Rollyo to aggregate the site into a larger search.