ElasticSearch 101

What you should know before you make
your hands dirty.

Dariusz Mydlarz


@dmydlarz | dmydlarz@gmail.com | dmydlarz.com

### Agenda 1. What ElasticSearch is? 1. Architecture 1. API 1. Use cases 1. Pitfalls

What ElasticSearch is?

  1. document database
  2. search engine
  3. data analytics tool

What ElasticSearch is?

  1. built on top of Apache Lucene
  2. open source - Github
  3. maintained by Elastic.co
  4. part of ELK stack
    • ElasticSearch
    • Logstash
    • Kibana

Benefits

  1. full text search
  2. distributed and highly available
  3. RESTful
  4. scalable
  5. ACID (per document)
  6. multi tenant

Scalable

Response times with increasing number of nodes.

Architecture

source: https://stephen.fm/getting-started-with-elasticsearch/

Highy Available

Shards distribution among the nodes.

Query Phase

  1. query to master
  2. distribute to shards
  3. merging shards data
  4. sending response
### APIs * Search APIs * cat APIs * Query DSL * ...
### Search API ```json POST /twitter/_search?routing=kimchy { "query": { "bool" : { "must" : { "query_string" : { "query" : "some query string here" } }, "filter" : { "term" : { "user" : "kimchy" } } } } } ```
### Sample Response ```json { "took": 1, "timed_out": false, "_shards":{ "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 }, "hits":{ "total" : 1, "max_score": 1.3862944, "hits" : [ { "_index" : "twitter", "_type" : "_doc", "_id" : "0", "_score": 1.3862944, "_source" : { "user" : "kimchy", "message": "trying out Elasticsearch", "date" : "2009-11-15T14:12:12", "likes" : 0 } } ] } } ```

Use Cases


Pitfalls

  • easy to delete: curl -XDELETE /index
  • heterogeneous data sets
  • (Near) Real Time Search
  • nested objects
  • can go down easily
    1. brain splits
    2. quorum
    3. JVM memory
    4. etc...
### Key Takeaways 1. Know your tool 1. Monitor it 1. Adapt to changes

Thank You

Reach me out: @dmydlarz, dmydlarz@gmail.com

### Go deeper * [https://www.elastic.co/products/elasticsearch](https://www.elastic.co/products/elasticsearch) * [https://www.elastic.co/guide/index.html](https://www.elastic.co/guide/index.html) * [https://lucene.apache.org/core/](https://lucene.apache.org/core/) * [https://stephen.fm/getting-started-with-elasticsearch/](https://stephen.fm/getting-started-with-elasticsearch/) * [https://www.oreilly.com/ideas/10-elasticsearch-metrics-to-watch](https://www.oreilly.com/ideas/10-elasticsearch-metrics-to-watch) [print to pdf](?print-pdf)