Join us for our upcoming meetup, we'll have two talks for the night starting at 6:30pm –– doors will open at 6pm. We'll have food, beverages, and good conversation! Thank you to Iterable for hosting! 

In this talk we'll discuss adapting Elasticsearch for use in a large multi-tenant enterprise SaaS application. We'll talk about an approach for scaling indices to handle thousands of tenants with shard routing that avoids explosions in cluster state size. We'll also discuss extending Elasticsearch to add encryption-at-rest using Lucene and translog encryption. We'll show Workday's recently-released 'escalar' library for working with Elasticsearch from Scala, including the beginnings of a Scala DSL for Elasticsearch's query language. Finally, we'll see how these three problems connect to three core concerns for managing enterprise data at scale: managing complexity; security; safety and quality.

Thomas Kim is soon to be an engineer at Iterable. He has been working in enterprise software and SaaS for over 15 years. He was formerly a tech lead on Workday Search. Prior to that, he was the CTO of a small BI startup and an early engineer at Salesforce. He loves dogs, snowboarding, and statically typed functional programming. Being a bandwagon Warriors fan makes his wife laugh.

Come learn how Tubular Labs analyzes billions of engagements to provide real-time audience insights with Elasticsearch.

**this is the same talk presented at the Elastic Silicon Valley meetup on April 6th**

Tubular Labs ingests billions of user interactions with online video content (likes, comments, shares, etc.) from all major video platforms, and uses that data to power various types of audience analysis.  We'll present how we leveraged Elasticsearch's powerful query language, augmented with the graph API from X-Pack, to breathe new life into one of our core products that shows what other channels or videos audience members also watch, given a set of seed channels or videos.  By constructing a schema tailored to let Elasticsearch do the heavy lifting, we are able to support a rich query paradigm with dynamic filters (that might target a particular time range or specific demographic), and surface thousands of results within seconds, providing with each result a variety of core metrics like overlap percentage, affinity index and audience size comparisons.  That this data resides in Elasticsearch also enables us to trivially construct visualizations in Kibana, which give us deeper out-of-the-box insight than would other storage options.

Scott Strickland spent almost 10 years engineering at Advertising.com, building out distributed optimization & adserving platforms prior to joining Tubular Labs in March of 2016.  This move represented a fundamental shift in paradigm towards more of a data-centric role, and as such he is an expert in neither data science nor Elasticsearch (though maybe that is a testament to the technology).  Scott is a C++ developer at heart, but finds himself working mostly with Python these days, and is somewhat concerned about losing his touch with template metaprogramming, or that C++17 will finally be released without him noticing.  He’s a fan of and enjoys playing music, baseball, and (on occasion) video games, and is happy to have a beer with you and talk about stuff.