sunspot_solr ruby gem with solr autosuggest/autocomplete, JQuery UI autocomplete, and apache HTTP server

00 Rails autocomplete

Architecture: Rails 3.2, Solr 1.4 with sunspot_solr gem, Apache HTTP server as reverse proxy to 
Rails app, Jquery UI

Note: I don't care about the distinction between autocomplete and autosuggest. I am implementing what some 
people call autosuggest, and I'm calling it autocomplete.


Given the above existing architecture, I want to add an autocomplete field into my app for a single 
attribute on a single model. It needs to be fast and secure (duh). The search target field could have 
white space and the search input should allow it, e.g., if I search with "iker's gui" it should return 
"Hitchhiker's Guide to the Galaxy". I know Solr has the capabilities and I'm already using the JQuery UI 
autocomplete feature for statically defined data sources.


I got my direction and some code from a sunspot_autocomplete plugin that seems to be rooted here 
https://github.com/haitham/sunspot_autocomplete and a sunspot_autocomplete gem found here 
https://github.com/xponrails/sunspot_autocomplete. None of them implemented things exactly how I prefer. 
I also browsed the ajax-solr javascript library at https://github.com/evolvingweb/ajax-solr (which the 
gem pulls in). Ultimately, the implementation did not require much code, so I just hand rolled it. And, 
I think I spent about the same amount of time but got a better understanding of the underlying components 
than if I'd used an unsupported abstraction.

0. Wherever you see "model" in the code below, replace it with your model name.

1. Access to Solr instance (almost) directly from client
One thing to bear in mind is you don't really want the client to talk directly to your Solr server, since 
that would leave the Solr instance vulnerable. I already use apache HTTP server as a reverse proxy to my 
Rails app, so I can do the same for my Solr instance. Check the httpd.conf snippet below. The key is to use 
the <Location> directive in an existing virtual host definition, since, as you know, the ajax call must be 
on the same host.domain. Although, you could use JSONP and a separate virtual host definition - explained
inline as comments in the code below. Note, I cut this out of a working conf and deleted the superfluous 
bits, so don't just assume you can cut-n-paste and it will work.

2. Add the autocomplete type and field to the Solr schema. Add the bits in the schema.xml file below then
Restart Solr.

3. Add the autocomplete type to the sunspot gem. Create the sunspot.rb file with the contents below and 
put in lib/ext. Then load it in config/application.rb, as below.

4. Instrument your model with the autocomplete field, as outlined below in model.rb.

5. Cool, now you are ready to reindex and test that it works. 
You can do "bundle exec rake sunspot:reindex[Model]". Then, visit 
localhost.domain.com/solr/select?wt=json&start=0&fq=type:Model&q=*:*&rows=20&fq=name_ac:"search text"
Note the phrase "search text" in the URL. This is the text you are searching for. Change that for 
something you know will match. Note the double quotes that keep the whitespace intact. That's important 
based on options used in the schema definition.

6. Excellent, now just wire that into the JQuery UI autocomplete on a page somewhere a la 
name_search.html.erb below. In this gist, I just use embedded javascript in a erb template. It assumes all 
the necessary prerequisites are already loaded, namely JQuery and JQuery UI. The important part here is 
how we connect the source data to the autocomplete plugin. We use a callback which invokes the ajax call 
to Solr then formats the result for the plugin. The plugin does allow for a URL instead, but then the 
server is responsible for formatting the result for the plugin, and one of the goals here is performance, 
so we let the client do the formatting as needed.

httpd.conf.part

<VirtualHost localhost.domain.com>
    ServerName localhost.domain.com

    LogLevel debug
    ErrorLog "logs/error_log"
    CustomLog "logs/access_log" combined

    <Location /solr/select>
        ProxyPass http://127.0.0.1:8982/solr/select
        ProxyPassReverse http://127.0.0.1:8982/solr/select
    </Location>

    ProxyRequests Off
    ProxyPass / http://127.0.0.1:3000/
    ProxyPassReverse / http://127.0.0.1:3000/

</VirtualHost>

model.rb

class Model < ActiveRecord::Base

...

  searchable do
    autocomplete :name
  end

  handle_asynchronously :solr_index # works like a charm with delayed_job

...

end

name_search.html.erb

<style>
  .ui-autocomplete-loading { background: white url(image_asset('ui-anim_basic_16x16.gif')) right center no-repeat; }
</style>
<%= javascript_tag do %>
  jQuery(function() {

    $('#search_name').keypress(function() { // hide found text when user starts typing again
        $('#found_name').html('');
    });

    $('#search_name').placeholder(); // if using placeholder plugin, init the field with the text

    $('#search_name').autocomplete({
      minLength: 2,
      source: function(request, response) {
        $.ajax({
          url: '<%= request.scheme + '://' + request.host_with_port %>/solr/select?wt=json&fq=type:Model&start=0&rows=20&q=*:*',
          dataType: "json",
          // pduey: note, you could use dataType : "jsonp" and add an additional parameter 
          // jsonp: json.wrf. see http://wiki.apache.org/solr/SolJSON
          data: {
            fq: 'name_ac:' + '"' + request.term + '"' // wrap in quotes so user doesn't have to
          },
          success: function(data) {
            response($.map(data.response.docs, function( item ) {
              return {
                label: item.name_ac[0],
                // sunspot always returns model id in response formatted as 'Model ID'. keep just the ID
                // in a arbitrary attribute of the autocomplete plugin for later use.
                model_id: item.id.replace(/Model /,"")
              }
            }));
          }
        })
      },
      select: function(event, ui) {
        $('#found_name').html('found ' + ui.item.label + ' with id: ' + ui.item.model_id);
      }
    });
  });
<% end %>
<div style="padding-bottom: 20px;">
  <%= label_tag :search_name, "Looking for a specific name? " %>
  <%= text_field_tag(:search_name, "", placeholder: 'Start Typing a Name', size: 50) %>
  <span id="found_name"></span>
</div>

schema.xml.part

  <types>

...

    <fieldType name="autocomplete" class="solr.TextField" positionIncrementGap="100">
      <!-- The index analyzer adds parts of the field from 2 - 25 chars including whitespace etc. -->
      <analyzer type="index">
        <tokenizer class="solr.NGramTokenizerFactory" minGramSize="2" maxGramSize="25"/>
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
      <analyzer type="query">
        <!-- The query analyzer takes the whole input, whitespace and all -->            
        <tokenizer class="solr.KeywordTokenizerFactory"/>
        <filter class="solr.LowerCaseFilterFactory"/>
      </analyzer>
    </fieldType>
  </types>

  <fields>
...
    <dynamicField name="*_ac" type="autocomplete" indexed="true"  stored="true"/>
  </fields>