Reading list for scaling Solr

Brain dump time. I kinda need this as a memory aid for myself, and I figure it’ll be useful to anyone else who is building a Solr cluster. There’s probably a lot of crossover here for tuning any JVM-based application servicing a large number of requests, but this is my first, so it’s all together.

Some of the background for this list can be read here, and for some further context, this is some of what I read to build something probably more powerful than websolr’s top tier offering (those guys are probably worth investigating before building your own cluster, by the way). There were some pretty “out there” requirements for though (potentially thousands of FQ permutations per search phrase, lots of big, ugly, old data, etc).

Some of the issues I ran into scaling Solr are relatively unique, but the general approach should be the same for everyone:

  1. Learn how to gather information
  2. Gather all the information you can
  3. Analyze / Evaluate
  4. Make incremental changes
  5. Return to Step 2.

Figure out what you’re actually running

If you’re unfamiliar with the world of Java (or a rusty shade of green like me), you might be horrified surprised to discover that there are a few different implementations (let alone versions) of the Java Virtual Machine (JVM) available to you. What’s more, the best documented and supported one, the “Oracle” JVM (still documented almost everywhere as the Sun JVM) is probably not what you’re running if you’re running Ubuntu Server.

There’s also a difference between engineering numbers and product numbers, which may not be immediately apparent from the outset, and often they appear to be used interchangeably.

Understanding the JVM

Understanding Solr

Somewhat passive aggressive Lucid Imagination advice –
An example of some of the hilarious bureaucracy in Solr development –

There’s probably plenty more, but those are the ones I have saved :)

This entry was posted in Technology. Bookmark the permalink. Both comments and trackbacks are currently closed.