sunspot solr in rails4 production

2014-11-24

one way to pave over recent sunspot solr configuration changes

sunspot/solr developers have been busy. Directory layouts and configuration schemas were improved... and now your rails app can no longer search or reindex its searchable.

RSolr::Error::Http (RSolr::Error::Http - 404 Not Found
Error:     Not Found

Your sunspot.yml looks good:

production:
  solr:
    hostname: localhost
    port: 8983
    log_level: WARNING
    path: /solr/production

So you go digging in the usual places and come to the conclusion that you are in upgrade purgatory with some old but precious (now useless) config and data state sitting on your server which need to translated from some old magic format to some newer magic format.

You are happy to realize that your data set is only a few thousand records, which means you can just rebuild the index for your models. So you blow away your solr/ directory and restart your solr server, which causes it to completely rebuilt its little work area, but with all-new files it actually likes.

% rm -rf solr
% RAILS_ENV=production rake sunspot:solr:start 
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/admin-extra.html => ...my_app/solr/conf/admin-extra.html
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/elevate.xml => ...my_app/solr/conf/elevate.xml
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/mapping-ISOLatin1Accent.txt => ...my_app/solr/conf/mapping-ISOLatin1Accent.txt
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/protwords.txt => ...my_app/solr/conf/protwords.txt
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/schema.xml => ...my_app/solr/conf/schema.xml
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/scripts.conf => ...my_app/solr/conf/scripts.conf
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/solrconfig.xml => ...my_app/solr/conf/solrconfig.xml
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/spellings.txt => ...my_app/solr/conf/spellings.txt
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/stopwords.txt => ...my_app/solr/conf/stopwords.txt
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/conf/synonyms.txt => ...my_app/solr/conf/synonyms.txt
Copying .../lib/ruby/gems/2.1.0/gems/sunspot_solr-2.1.1/solr/solr/solr.xml => ...my_app/solr
Successfully started Solr ...
% RAILS_ENV=production rake sunspot:solr:stop 
Successfully stopped Solr ...

Being curious and not really very trusting of this self-healing magic (remember how we got here), you look around and notice that your friendly running sunspot solr has created directories for you:

solr/data/production
solr/test/data
solr/development/data
solr/default/data

Having read some online postings, you know that you need to tweak the default solr environments, otherwise you wind up with 'development', 'test', and ... 'default' when the only thing you really want it 'production'.

You need to edit solr/solr.xml from this:

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="false">
  <cores adminPath="/admin/cores" host="${host:}" hostPort="${jetty.port:}">
    <core name="default"     instanceDir="." dataDir="default/data"/>
    <core name="development" instanceDir="." dataDir="development/data"/>
    <core name="test"        instanceDir="." dataDir="test/data"/>
  </cores>
</solr>

to this:

<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="false">
  <cores adminPath="/admin/cores" host="${host:}" hostPort="${jetty.port:}">
    <core name="production" instanceDir="." dataDir="production/data"/>
  </cores>
</solr>

Then some cleanup

cd solr
rm -rf default test
mv development production
cd data
rmdir production
ln -s ../production/data production

Having repurposed the pristine development area into the production data area you really came for, you should now be ready to start the sunspot solr daemon again, but this time both sunspot and solr will agree on where to park the search index, which means you are ready to reindex.

bundle exec rake sunspot:solr:start
Successfully started Solr ...
webman@zeebar-technology-services:~/sites/audrey % rails c
Loading production environment (Rails 4.1.7)
irb(main):001:0> Article.reindex
D, [2014-11-24T11:34:10.783622 #6079] DEBUG -- :   SOLR Request (462.7ms)  [ path=update parameters={} ]
[2014-11-24 11:34:10 -0800] Start Indexing
D, [2014-11-24T11:34:11.844369 #6079] DEBUG -- :   SOLR Request (1001.3ms)  [ path=update parameters={} ]
D, [2014-11-24T11:34:12.531274 #6079] DEBUG -- :   SOLR Request (685.9ms)  [ path=update parameters={} ]
[2014-11-24 11:34:12 -0800] Completed Indexing. Rows indexed 50. Rows/sec: 29.200150186884464 (Elapsed: 1.71231996 sec.)
=> nil
irb(main):002:0>

Ta-dah!

all articles