Since the passing of the General Data Protection Regulation (GDPR), many companies needed to increase the security of their data systems. In order to easily migrate teams to new, secure clusters, the HubSpot Data Infrastructure team wrote a managed ingestion pipeline using Kafka that enables them to seamlessly migrate teams' indices from one cluster to another with minimal effort on the teams' behalf. Patrick will discuss the ingestion pipeline, the migration process, and how it leverages Elasticsearch features to make the migration seamless for HubSpot's other development teams.

Patrick has worked at HubSpot for the last four years, and is currently the Technical Lead of the Data Infrastructure: Search and Data Infrastructure: Processing teams at HubSpot. He has also previously lead the Data Infrastructure: Big Data team. Patrick has previously spoken at HBasecon about solving multitenancy problems in HBase.

In this new release we have taken a bold step and opened the X-Pack code for viewing, comment, and bug tracking. In addition, we’ve now included all of the free X-Pack Features in the default distribution. Learn how you can benefit from these changes to accelerate your projects and get started with the Elastic Stack today!

Brad Quarry is a Senior Solution Architect with Elastic. After a 15 years of helping build and deploy parallel computing architectures he is most comfortable putting putting himself squarely between the customer and the problem at hand. When he’s not hip deep in error logs, he enjoys playing with his two boys William (2y) and Sam (6mo), running, and geeking out watching Sci-Fi movies.