Elastic{ON}16 – Takeaways – Part 2

I attended the “Stories from Support” session at Elastic{ON}. There were a few new troubleshooting tips I learned as well as some recommendations that I was not aware of.

I recently had been looking for a rule of thumb for shard sizes.  I’m glad this was clarified during the session. The only information I was able to find for it online showed about 30GB to be the upper limit for a shard size. During the session, they stated this to be actually at 50GB.  I would also like to emphasize that this is a rule of thumb. Actual performance results will vary depending on a number of factors.

I also learned that a higher number of shards could actually negatively impact your  search times.  You also don’t get any benefit from having more shards than nodes if your index size is small. The old default of 5 shards per index is actually not great, especially if none of your indexes are not very large. It also isn’t that great if your indexes are time based and you are creating indexes daily. I’ve run into this myself with Logstash. If you leave the defaults in place, you will quickly have a high number of shards.

You can view your shards with the following command:

GET _cat/shards?v

Setting the number of shards can only be done at index creation time. Here is the command to set the number of shards:

PUT [newIndexName]
"number_of_shards" : xx

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s