Lessons Learned: Upgrading a 100 node ES Cluster from 1.7 to 7.x Agenda: - Challenges faced with 1.7: Mapping explosion, large cluster state, high CPU usage, EOL - Estimating the optimal cluster size keeping performance and cost in the mind - Cluster provisioning using ECK on AWS EKS with separate hot and cold data tiers to save costs - Designing settings and optimal sizes of indices, aliases, and shards - Implementing rollover and Index Lifecycle Management - Upgrading Java/Spring based microservices using spring-data-elasticsearch and retiring TransportClient - Data Migration from on-prem ES to ECK based ES on AWS: Direct data migration instead of snapshot-restore approach - Performance improvement measurements