Artirix News


David vs Goliath (or Solr vs FAST)

July 20, 2011

This is a benchmark of the legacy FAST search engine (ESP 5.1) and Solr 1.4.

We load tested a generous sample of queries from the globrix.com website as recorded in the fast query logs.

With a FAST->Solr migratory query translator developed by Artirix we translated these into the equivalent solr queries. This handled the various quirks of the FAST IN syntax, booleans, etc. and translated to compatible queries for Solr.

The hardware running on was 2 Dell 2950s, each with 20GB memory and SAS 15k disks. Solr was run as a master on a separate host and a slave on each 2950s, with haproxy in front of them to load balance. FAST had the admin node on a separate host and a search node on each of the 2950s.


SOLR

threads qps mean query time (ms) median query time (ms)
1 17.9 54.6 23.6
6 83.7 70.3 23.4
10 79.2 124.9 38.2

FAST

threads qps mean query time (ms) median query time (ms)
1 6.1 162.6 170.3
6 23.6 252.9 257.4
10 27.4 363.5 366.0

As we were testing prior to the full geo search capabilities in solr 3, the geo radius queries were translated into bounding boxes. This resulted in a small difference in result count between FAST and solr.

The figures show better than 3x performance under load! The latency is vastly better too – 10 times smaller for solr vs FAST for median timings – very impressive.

We also tested solr under constant updates under load with the slaves mirroring every minute and the performance hit it suffers is about 15-20%. I haven’t played with the searcher warming-up queries, but this should get this hit down, as some of the live queries effectively run as warm-up. The advantage FAST offers is it presumably does this for you