Another reason why search initiatives rarely live up to their potential is that search solutions don't deliver relevant results. Now this statement sounds a bit over the top, so let's start with the data. A 2007 study conducted by Accenture made it clear just how poorly search tools deliver relevant results:
50% of respondents indicated that the information delivered via their search solution had no value whatsoever. Can you think of any other piece of technology infrastructure that functions as intended 50% of the time? Can you imagine if your email system delivered email half the time? So what's happening here? Why are the results so poor?
Relevance is tricky business because it's often subjective. what's relevant to one person might not be so the next, even if they are using the same keywords when they search. To this day, search vendors struggle to perfect ranking algorithms that will magically calculate what someone is looking for based on a two or three keyword query. Search vendors recognize that this isn't possible and often suggest that companies add metadata to each document to assist the search engine and the user in refining the search to a subset of the data, increasing the likely hood of locating the relevant content. In fact, this is the basis of Google's PageRank algorithm, which works reasonably well on the Internet. SharePoint has a marvelous new construct called Content Types that provide the ability to add rich metadata to document content as well as other types of data. SharePoint Search can be much improved through the use of content types, but more on that in a later post.
Relevance is comprised of two characteristics. The first is called precision and it's what I'm referring to above. The ability of the search engine to return relevant data to your search and weed out irrelevant content. Recall, on the other hand, represents the search engines ability to return all of the relevant content that's available. In my view, the Accenture study highlights a problem with recall.
One of the mistakes I see companies making repeatedly is to take a traditional approach to Enterprise Search. Approaching search as infrastructure, IT takes a top down, one size fits all approach. The problem here is that different departments have vastly different information needs. The data that the legal department uses on a daily basis is very different than that of finance, for example. Most search initiatives only focus on document content. The data contained in the ERP, CRM, or custom applications are not included in the scope of the project, or "it'll be included in Phase II of the project", which never happens. If the data that one uses on a daily basis, can't be accessed via search how will the search initiative be relevant to anyone?
Search technology now has the ability to index structured content. In the case of SharePoint Search, SharePoint's Business Data Catalog (BDC) provides this functionality. Nine times out of ten, companies deploying search aren't using the BDC. Companies have invested millions in their ERP, CRM, and custom applications. Unlocking this data and making it directly accessible through search will usher in a new era of productivity.
More on this in the next post.