Abstract

For this meetup we are excited to be joined by two great speakers! Ben Hindman, Co-Founder of Mesosphere and Creator of Apache Mesos, and Joerg Schad, Technical Lead for Community Projects will be talking about running Kubernetes-As-A-Service alongside Apache Spark, HDFS, Data Science and more.

What You’ll Learn At This Meetup:

1) Kubernetes-As-A-Service Alongside {Spark, HDFS, Data Science ..}-As-A-Service:

Kubernetes helps developers get their containers up and running. But what’s the best way to run Kubernetes itself? What if you want to give each of your teams their own Kubernetes cluster?

Come here us talk about why and how we built Kubernetes-as-a-Service. Moreover, we’ll describe the other “as-a-Service” options you can run along with Kubernetes, such as Kafka, Cassandra, HDFS, and Spark, in order to quickly get a stack deployed to help with your data engineering and data science projects. We’ll finish it all off with a live demo that puts all the pieces together!

2) Kubeflow++:

In the second part we will look at how this concept extends to Data Science.

Kubeflow is makes it very easy for data scientist to build their own data science pipeline with Jupyter Notebooks, TensorFlow, TensorBoard and Model serving. However, to build a production grade data science pipeline requires some additional components.

As examples consider data preparation (which is frequently performed using Apache Spark or Apache Flink), data storage (using HDFS, Cassandra), or request streaming (using Apache Kafka).

The choice of tools often depends the concrete environment Kubeflow is running in: While Cloud Providers such as Google Cloud, AWS, or Azure have managed solutions for these challenges, an on-prem environment might require alternative Open-Source solutions.

So in this talk we look at building a complete deep learning pipeline around Kubeflow and answer topics such as:

• Data Preparation/Cleansing
• Data and Model Storage
• Model Serving
• Monitoring
• Infrastructure Management across multiple tenants

Event link: Hybrid Cloud {Kubernetes, Spark, HDFS, Data Science, …} As-A-Service

Media

Slides

Photos