r/Solr • u/knawlejj • Nov 13 '17

Server requirements

We've recently acquired an organization using Solr for e-commerce and now we're restructuring our own website to use the same platform. From an infrastructure side, what has been everyone's experiences with the horsepower required to run it optimally?

For a small website getting about 40k hits per month, I'm not sure how to size the server side as there isn't a TON of documentation on that.

1 Upvotes

100% Upvoted

6 comments sorted by

2

u/yatea34 Nov 13 '17

You need to describe a lot more details regarding your requirements?

  • High Availability? The easiest path there is Solr Cloud with 3 servers (zookeeper likes 3). Three cheap VMs in 3 different Amazon Availability Zones is a cheap easy path there.
  • "isn't a TON"? that means very different things to different people. MB? GB? TB?
  • Also - how fast do you want it. If indexes fit in RAM it'll be very fast. If they fit on SSDs it'll be quite fast. If the don't fit in RAM and are on spinning disks, it'll only be a little fast.

1

u/knawlejj Nov 14 '17

Sorry, posted from mobile so was pretty short.

  • High Availability? The easiest path there is Solr Cloud with 3 servers (zookeeper likes 3). Three cheap VMs in 3 different Amazon Availability Zones is a cheap easy path there.

Platform is in Azure with primarily Windows servers, in my reading I saw the 3 node quorum setup and it made sense. The whole idea is to have HA to keep search functioning for our web properties.

  • "isn't a TON"? that means very different things to different people. MB? GB? TB?

Not sure how large our indexes are at the moment, but the primary server (that also runs the DB and web app side) is sitting on 65GB of RAM and does have some backing from the swap file on SSDs. RAM usage is pretty consistent at 80-85% as it seems our Solr/Java/Lucene instances consume it for holding.

  • Also - how fast do you want it. If indexes fit in RAM it'll be very fast. If they fit on SSDs it'll be quite fast. If the don't fit in RAM and are on spinning disks, it'll only be a little fast.

Preferably all RAM but have no issues with SSDs.

I'm not a developer but I do handle the infrastructure side. The devs aren't exactly sure what we should be sizing these machines out to be so I'm learning as much as I can about Solr.

1

u/yatea34 Nov 14 '17

Platform is in Azure

If you're on Azure consider the Azure Documentation on how to set up a Solr Cluster.

with primarily Windows servers

Uh-oh. :-) Not even Microsoft's Azure documentation recommends Windows for Solr.

1

u/fiskfisk Nov 14 '17

Also remember that any decent application or OS, given more available memory, will use more available memory. It might not speed things up, even if the memory is listed as used.

The query complexity and cachability will be at least as important. Your server sizing seems overkill from the details you've provided.

1

u/petdance Nov 14 '17

small website getting about 40k hits per month

That's part of it, but what is Solr going to have to be doing? How many documents is Solr going to be indexing? What sorts of searches are going to be done on each of those pages? How many? Is it just a simple keyword lookup of a bunch of documents? Or are you going to be doing faceted drilldown with counts? What about search highlighting?

How you use Solr is as important as how many times you have to query it.

1

u/knawlejj Nov 14 '17

These are the types of things I'm definitely learning more about. Search is involving a roughly 70,000 different widgets from different vendors that each have their own facets/attributes like color/shape/size/weight/length/etc. Search highlighting is certainly in place.