Changing the delivery of IT

Tony Bishop

Subscribe to Tony Bishop: eMailAlertsEmail Alerts
Get Tony Bishop: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Article

In Search of the Human Element

Silent observation is key to the success of social search

George Orwell didn't specifically mention enterprise search in his visionary book 1984, but he made a statement that still resonates today. When it comes to relevant search query results, "it's not about the statistics." Sanity comes from the human element.

Until recently, this concept was mostly ignored by enterprise search solutions. Instead, search was based on text-matching algorithms and models that methodically sifted through link structures or categorization schemes. After analysis and compilation, a surge of search results (all quantifiably correct) were released. While a statistician might be impressed by the outcome, the user who initiated the search simply drowned in a sea of homogenous data.

The question became, "Is it possible to produce a set of meaningful search results that will help people, rather than inundate them?"

In response, enterprise search technologies introduced tools that allowed "experts" to tune the search algorithm for their specific content, and introduced meta-tagging best practices to structure content and produce more meaningful results. More recently, explicit user actions such as click-throughs, ratings, and feedback were introduced to solve the issue of search relevancy. Today, advanced techniques including "social search" have evolved to take into account how humans search for, find, and consume information and products in the physical world.

Comparing Various Approaches to Solve Search
Enterprise search has long been the de facto standard for corporate knowledge databases and websites. Because this approach is typically based on text-matching algorithms, it does a thorough job in finding all documents that match a particular query term. Some highly evolved search engines use specialized classification systems or pattern recognition techniques that rely on statistical inference. As a result, enterprise search engines have experienced huge increases in performance, comprehensiveness, and automation. However, they still lack the single most important ingredient that produces relevant search results: subjectivity. Guided by a few ambiguous words typed into a text field, a search engine still has no reliable way of accurately interpreting the actual intent of an individual user.

To cope with subjectivity, and the lack of understanding around user intent, various approaches including expert-based tuning and user-explicit tuning have been developed to try and provide more useful results to the end user.

Different expert tuning approaches can include tuning based on algorithms, tuning the actual content, and tuning based on queries. The primary issues with these expert-driven approaches are that they can be severely labor intensive, biased, and lack insight as to the true intent of the user. Ultimately they proved to be too rigid and laborious over time and so a more scalable model was envisioned, one that would take into account user feedback.

More Stories By Mike Svatek

Mike Svatek is currently vice president of marketing for Baynote and has been taking Internet technologies to market for more than a decade. Prior to Baynote, he was at Interwoven where he set the product direction and marketing strategy for initiatives related to search, portal, and web content management, and drove business development activities with partners in support Interwoven's global customer base. Mike holds an MBA from the Haas School of Business, UC Berkeley, and a BBA from the McCombs School of Business at the University of Texas, Austin.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.