Time series data tends to grow over time. And while it might be easier to store and manage this data via a single index, it’s often more efficient and cost-effective to store large volumes of time series data across multiple, time-based indices. Multiple indices enable you to move indices containing older, less frequently queried data to less expensive hardware and delete indices when they’re no longer needed, reducing overhead and storage costs.
If you’re collecting a terabyte of data per day, for example, that’s seven terabytes a week. Kept over several years, this easily grows to petabytes of data. Users need a way to manage this exponential storage growth and have the capability to search for everything. Learn how to meet these goals with the help of Elastic.
• Part 1: Index Lifecycle Management (ILM) - [13th October 2021]
ILM automates how you want to manage your indices over time. You control how indices are handled as they age by attaching a lifecycle policy to the index template used to create them. You can update the policy to modify the lifecycle of both new and existing indices.
• Part 2: Data streams - [20th October 2021]
Data streams abstract away — and simplify — some of the complexity that comes with having to manage numerous time-series indices, making features like index lifecycle management (ILM) a breeze to configure and simple to maintain.
• Part 3: Searchable snapshots- [27th October 2021]
Searchable Snapshots allow us to keep low-cost object stores from AWS, Google, Azure, and other object stores always online and available, by making your snapshots directly searchable by Elasticsearch.