Elastic{ON}16 – Takeaways – Part 1

This was my first time at an ElasticSearch conference.  I had a great time, learned a few new things and even stumbled upon some new software I’m looking forward to trying out.

I’ve been to a few conferences before and typically my experience with the outside vendors hasn’t been anything too exciting.  The software would either cost more than we would be willing to pay or we already developed something internally that solved our need. I was pleasantly surprised when this was not the case at Elastic{ON}.   Two of the products I found the most interest in were Kibi and StreamSets.  Kibi a platform built on top of Kibana. For me, Kibana has had a few gaps. You cannot join two different indexes and you cannot reference external data sets.   With Kibi, you are able to do joins and even reference external data sets.  Since Kibi is built on top of Kibana, the interface is familiar and not too difficult to pick up if you have previous experience with Kibana. I’ve started looking at Kibi, but looks like to fully take advantage of everything, we need to update to ElasticSearch 2.0 which we are not quite ready to do yet.

The other piece of software that really sparked my interest was StreamSets. One of the difficulties in making a jump to some of these technologies has been the lack of any type of user interface.   Moving data to HDFS from SQL with Sqoop was not something the average user wanted to do.  StreamSets interface is visually appealing and a much easier tool to use for any user, especially those who will be newer to the big data world.

I did get a chance to setup and move some data with StreamSets. Setting up a SQL datasource, I found, was a little tricky at first. The drivers for MSSQL are not bundled with StreamSets. You have to add these yourself.  Also, when creating the connection, the password is stored in clear text.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s