Elasticsearch for central log collection in OpenShift 3 by Tobias Brunner
Elasticsearch is used in OpenShift 3 for central log collection of all application logs. This talk will give an insight how it is done, how Elasticsearch runs itself as part of the Kubernetes cluster, how scaling is done and how the data is protected.
The project linked-swissbib aims to extend the Swissbib catalogue - the metacatalogue of all Swiss university libraries among others - towards a semantically enriched bibliographic metadata platform. To accomplish this goal we split the originally monolithic Swissbib records into several distinct bibliographic concepts, transform these newly created documents into JSON-LD and partially enrich them with resources from third-party providers like DBPedia. The modularly built platform linked-swissbib will provide different modes of access to the customers, currently planned are a search catalogue and a RESTful API.
As central hub for the enriched data Elasticsearch plays an important role in our architecture. In this talk I will discuss how we transform our data to meet the requirements set by the search engine, how we structure the different concepts in the index to be as flexible and as generic as possible for a range of applications and why we chose Elasticsearch for storing our 99m+ documents.