Reading list for scaling Solr

Brain dump time. I kinda need this as a memory aid for myself, and I figure it’ll be useful to anyone else who is building a Solr cluster. There’s probably a lot of crossover here for tuning any JVM-based application servicing a large number of requests, but this is my first, so it’s all together.

Some of the background for this list can be read here, and for some further context, this is some of what I read to build something probably more powerful than websolr’s top tier offering (those guys are probably worth investigating before building your own cluster, by the way). There were some pretty “out there” requirements for Boards.ie though (potentially thousands of FQ permutations per search phrase, lots of big, ugly, old data, etc).

Some of the issues I ran into scaling Solr are relatively unique, but the general approach should be the same for everyone:

Learn how to gather information
Gather all the information you can
Analyze / Evaluate
Make incremental changes
Return to Step 2.

Figure out what you’re actually running

If you’re unfamiliar with the world of Java (or a rusty shade of green like me), you might be ~~horrified~~ surprised to discover that there are a few different implementations (let alone versions) of the Java Virtual Machine (JVM) available to you. What’s more, the best documented and supported one, the “Oracle” JVM (still documented almost everywhere as the Sun JVM) is probably not what you’re running if you’re running Ubuntu Server.

There’s also a difference between engineering numbers and product numbers, which may not be immediately apparent from the outset, and often they appear to be used interchangeably.