Join us for our next meetup on July 13! 

Agenda:

• 5:30 PM - Pizza and Networking

• 6:00-6:45 PM - Talk#1: Scaling Log Aggregation At Fitbit with Elasticsearch

• 6:45-7:30 PM - Talk#2: Security Analytics with the Elastic Stack 

• 7:30 PM - Open Discussion

Scaling Log Aggregation At Fitbit with Elasticsearch by Breandan Dezendorf 

A walkthrough of scaling a Elasticsearch based log aggregation pipeline from 30,000 logs per second to over 225,000 logs per second in a demanding multi-user environment.  This process involved upgrades to every part of the pipeline and changing out major architectural features along the way. Also discussed will be some of the design considerations and challenges for disaster recovery, long term archiving and practical limitations of running very large cost effective Elasticsearch clusters.

Breandan has been working in UNIX and Linux operations for over 15 years. His specialties include monitoring, alerting, trending, and log aggregation at scale. Recently he has been focused on scaling log aggregation for Fitbit, Inc to over 225,000 logs per second. 

Security Analytics with the Elastic Stack 

Peter's presentation will discuss using the Elastic Stack and X-Pack for Security Analytics. The machine learning capabilities of X-Pack can be used to automate the detection of anomalies in time series data (security logs) that correspond to cyber attack behaviors.  From data exfiltration to malware command and control activity, hear how you can put Elastic Machine Learning to work for you to detect hard-to-find attack activity as anomalies before security incidents progress too far.

Peter Kim is a Solutions Architect at Elastic with over 10 years of experience architecting and developing search applications with Endeca, MarkLogic and core Lucene. Currently, he helps organizations learn how to get real-time insights into their data using Elastic products. On his free time, Peter enjoys discovering new single-origin coffees and exploring NYC with his wife and two sons.