Loading…
DevConf.US 2019 has ended
DevConf.us 2019 is the 2nd annual, free, Red Hat sponsored technology conference for community project and professional contributors to Free and Open Source technologies held at the Boston University in the historic city of Boston, USA.

When: Thursday, August 15 to Saturday, August 17, 2018

Venue: Boston University, George Sherman Union Building
Friday, August 16 • 09:00 - 09:45
Scalable kafka deployment on OpenShift for ML

Sign up or log in to save this to your schedule and see who's attending!

The Open Data Hub (ODH) is a scalable data lake platform for data scientists and their machine learning (ML) activities. We have an internal deployment of ODH that processes RHEL build logs and test results. It aggregates and replicates the data from internal and upstream CI systems. AIOps then uses ML techniques to analyze the data, specifically for anomaly detection. The central component is Apache Kafka deployed across multi data centers on a common OpenShift platform which ensures high availability of the core of the ODH system. The presenters will discuss this use case where Kafka is deployed as a caching tool in a highly available and horizontally scalable architecture running on OpenShift, processing hundreds of gigabytes of log data per minute and thousands of messages per second.

Speakers
avatar for Peter MacKinnon

Peter MacKinnon

Principal Software Engineer, RH - Raleigh - Red Hat Tower
Pete MacKinnon is a Principal Software Engineer in the AI Center of Excellence at Red Hat. He is actively involved in the open source Kubeflow project with Google and Microsoft engineers to bring TensorFlow machine learning workloads to container environments (Kubernetes and OpenShift... Read More →
avatar for Maulik Shah

Maulik Shah

Softwate Engineer at the AI Center of Excellence at Red Hat, Red Hat Inc.
Hi I am a Software Engineer with the AICoE at Red Hat. Before this I did my Masters in Computer Science @ Boston University. At Redhat I work as a Data Engineer which involves ferrying massive amounts of data across systems and I also work on monitoring a


Friday August 16, 2019 09:00 - 09:45
East Balcony GS Union, BU

Attendees (19)