In this meetup, Adyen is going to share their experience in delivering operational excellence with Elastic. Adyen is a Dutch Payment Service Provider that that allows businesses to accept e-commerce, mobile, and point-of-sale payments. Adyen has more than 3,500 customers and is listed on the stock exchange Euronext.

Each talk will be approx 10/15min. 

Talk 1: Adyen & General architecture by Willem Pino:

In this introduction I will cover the basics of the problem Adyen is solving for it’s merchants. After this I will discuss how we handle the hundreds of transactions per second on our platform. What were the main scaling issues we ran into and how did we solve it? How do we handle the large amounts of data outside of our ELK stack?

Talk 2: Payment Search by Bengisu Sahin:

How we migrated ~12TB data from Elasticsearch v1.7 to v7.6

The Customer Area is our merchant facing web application where merchants can gain insights into their transactions and perform operational actions. In the Customer Area, the payment search is the most used functionality that allows merchants to search their payments by using an order reference, an email address, a payment method, etc.

At the end of 2014, we migrated the payment search from Lucene to ElasticSearch when the version of Elasticsearch was 1.7. Since then, we have done the first update last year, by jumping six major versions, to version 7.6. During this meetup, I will share our journey of this challenging update.

Talk 3: Monitoring by Diego Costa:

How we enable anyone at Adyen to keep track of their share of the platform in real time

The goal of the monitoring team at Adyen is to provide solutions that can monitor the whole platform for merchant-facing issues and that are flexible enough to be used by all the teams. This must occur on a (near) real time-basis. To accomplish this, we leverage the ELK stack to build the necessary tools. The monitoring solution looks upon analytic events, that are freely created by the programmers, as the source data for all the configurable monitors. This ‘auxiliary’ event exists in parallel and complements the logging infrastructure. Next, the monitors accommodate the analytic events into multiple use cases, such as spikes, drops, underperforming, anomaly detection, forecasting, etc. And finally, the alerts are delivered to the right individuals.

Talk 4: Logging by Lucian Grosu:

Growing a cluster from 1TB to 1PB+ of data and then throwing it all away to start from scratch

I'll be talking about how we scaled one of our logging clusters from a few machines to multiple racks worth of servers, what choices we made along the way. Emphasis has to be made on the role that the log sources have on the choices made when designing specific parts of the logging pipeline. In the end, we'll look into why, even after successfully scaling to this size, we had to start from scratch.