Custom Site-Search Engine Using the Bing API

For Mix07 in London this week I created a sample site search to illustrate creating your own site search engine using the Live Search API. I've posted the sample code here, along with some notes on implementation (including a couple features I didn't have time to finish).

Here's a quick list of the features covered:

  • Spelling corrections
  • Web search results
  • Image search results
  • OpenSearch 1.1 (Putting your search in the browser searchbox)
  • Live Search Macros

Setting up your environment

Before we get to the code you'll need to provision yourself an Application ID to gain access to the web services, and then you'll also need to add a web reference to your visual studio project that references our WSDL: http://soap.search.msn.com/webservices.asmx?wsdl. Visual Studio will read the WSDL and use that to automatically generate several classes for rich objects that you can use instead of directly interacting with the SOAP API. Note, using these classes does not create any dependency on Visual Studio or your project AKA they only make your life easier.

Once you've added the reference, you can include the library in your project with use com.msn.search.soap

Define your custom search query

The first step is to use the live search query language to define the scope of your search. If you're building a site search there really is only one keyword you'll need, the "site:" operator. It scopes the search to just the domain/subdomain you specific, and you can join multiple together using the OR operator. Here's the query that we'll use in this application:


(site:msdn2.microsoft.com OR site:asp.net OR site:codeproject.com OR site:blogs.msdn.com)

Tip: including a lot of sites can dramatically slow down the query, so chose your sites carefully and consider omitting domains to simplify things (e.g. do this "site:asp.net" instead of this: "site:ajax.asp.net OR www.asp.net")

Executing the query

With each request, you can query any or all of the supported data sources: web, image, spelling, phonebook (directory) and news. The query must be the same for all sources, however, there are a few parameters you can adjust within each data source you request data from. This allows you to build a rich SERP with a single request to Live Search, and a single mark against your quota for the day.

   1:  // used to index the query request and the query results
   2:  protected int SR_SPELLING = 0;
   3:  protected int SR_IMAGE = 1;
   4:  protected int SR_WEB = 2;
   5:   
   6:  protected SearchResponse runQuery(string query)
   7:  {
   8:      MSNSearchService s = new MSNSearchService();
   9:      SearchRequest searchRequest = new SearchRequest();
  10:   
  11:      SourceRequest[] sr = new SourceRequest[3];
  12:   
  13:      // SPELLING SUGGESTION QUERY
  14:      sr[SR_SPELLING] = new SourceRequest();
  15:      sr[SR_SPELLING].Source = SourceType.Spelling;
  16:      sr[SR_SPELLING].ResultFields = ResultFieldMask.Title;
  17:   
  18:      // IMAGE SEARCH QUERY
  19:      sr[SR_IMAGE] = new SourceRequest();
  20:      sr[SR_IMAGE].Source = SourceType.Image;
  21:      sr[SR_IMAGE].ResultFields = ResultFieldMask.Description
  22:          | ResultFieldMask.Image | ResultFieldMask.Title;
  23:      sr[SR_IMAGE].Count = 5;
  24:   
  25:      // WEB SEARCH QUERY
  26:      sr[SR_WEB] = new SourceRequest();
  27:      sr[SR_WEB].Source = SourceType.Web;
  28:      sr[SR_WEB].ResultFields = ResultFieldMask.Description
  29:          | ResultFieldMask.DisplayUrl | ResultFieldMask.Title
  30:          | ResultFieldMask.Url;
  31:      sr[SR_WEB].Count = 10;
  32:   
  33:      searchRequest.Query = "futbol";
  34:      searchRequest.Requests = sr;
  35:      // You can provision your application id here: http://search.live.com/developer
  36:      searchRequest.AppID = "Your_ApplicationID_Goes_Here";
  37:      searchRequest.CultureInfo = "en-US";
  38:   
  39:      // this call executes the search
  40:      SearchResponse searchResponse = s.Search(searchRequest);
  41:   
  42:      return searchResponse;
  43:  }

 

Essentially you create a collection of SourceRequest objects, each one corresponding to a different data source within the Live Search Web service. You then create a SearchRequest object, give it the SourceRequests you configured, plus some information about the actual search itself. Finally, we grab the SearchResponse object, which will contain the search results for each data source from the Web service.

Parsing the query results

Because of the rich objects created by visual studio from the WSDL, parsing the search results is relatively trivial. I have removed the code that renders the data and just left the code required to get the values you probably care about out of SearchResponse object, but I encourage you to explore the object and see what other information is in there.

Let's get the results from the spelling suggestions first because it is the easiest. Note that you will only see a result here if Live Search thinks your query term may have been misspelled. Otherwise, it won't have any results.

   1:  // spelling suggestion
   2:  void renderSpellingSuggestion(SearchResponse searchResponse)
   3:  {
   4:      string spellingSuggestion = "";
   5:   
   6:      // Note that SR_SPELLING is simply an int defined to be 0 in this example, based on RunQuery() 
   7:      //    method above.    
   8:      if (!String.IsNullOrEmpty(searchResponse.Responses[SR_SPELLING].Results[0].Title))
   9:          spellingSuggestion = searchResponse.Responses[SR_SPELLING].Results[0].Title;
  10:  }

 

Next we'll parse the web results:

   1:  // web results
   2:  void renderWebResults(SearchResponse searchResponse)
   3:  {
   4:      // Note that SR_WEB is simply an int defined to be 2 in this example, based on RunQuery() 
   5:      //    method above.
   6:      foreach (Result sourceResult in searchResponse.Responses[SR_WEB].Results)
   7:      {
   8:          string title, description, url;
   9:  
  10:          title = (!String.IsNullOrEmpty(sourceResult.Title)) ? sourceResult.Title : "";
  11:          description = (!String.IsNullOrEmpty(sourceResult.Description)) ? sourceResult.Description : "";
  12:          url = (!String.IsNullOrEmpty(sourceResult.Url)) ? sourceResult.Url : "";
  13:      }
  14:  }

 

And finally the image results, this is extra long because there are lots of interesting fields:

   1:  // image results
   2:  void renderImageResults(SearchResponse searchResponse)
   3:  {
   4:      // Note that SR_IMAGE is simply an int defined to be 1 in this example, based on RunQuery() 
   5:      //    method above.
   6:      foreach (Result sourceResult in searchResponse.Responses[SR_IMAGE].Results)
   7:      {
   8:          string title, description, imageUrl, imagePageUrl, thumbnailUrl, thumbnailPageUrl;
   9:          int imageHeight, imageWidth, imageFileSize;
  10:          int thumbnailHeight, thumbnailWidth, thumbnailFileSize;
  11:  
  12:          title = (!String.IsNullOrEmpty(sourceResult.Title)) ? sourceResult.Title : "";
  13:          description = (!String.IsNullOrEmpty(sourceResult.Description)) ? sourceResult.Description : "";
  14:  
  15:          imagePageUrl = (!String.IsNullOrEmpty(sourceResult.Url)) ? sourceResult.Url : "";
  16:          imageUrl = (!String.IsNullOrEmpty(sourceResult.Image.ImageURL)) ? sourceResult.Image.ImageURL : "";
  17:          imageHeight = sourceResult.Image.ImageHeight;
  18:          imageWidth = sourceResult.Image.ImageWidth;
  19:          imageFileSize = sourceResult.Image.ImageFileSize;
  20:  
  21:          thumbnailPageUrl = (!String.IsNullOrEmpty(sourceResult.Url)) ? sourceResult.Url : "";
  22:          thumbnailUrl = (!String.IsNullOrEmpty(sourceResult.Image.ThumbnailURL)) ? sourceResult.Image.ThumbnailURL : "";
  23:          thumbnailHeight = sourceResult.Image.ThumbnailHeight;
  24:          thumbnailWidth = sourceResult.Image.ThumbnailWidth;
  25:          thumbnailFileSize = sourceResult.Image.ThumbnailFileSize;
  26:      }
  27:  }

 

Advanced Features

For more information on the advanced features like adding your search engine to the browser's search box, and deeper integration with Live.com, check out the power point presentation.

Files and resources:

comments powered by Disqus