Skip to content

Latest commit

 

History

History
171 lines (103 loc) · 5.51 KB

README.md

File metadata and controls

171 lines (103 loc) · 5.51 KB
Raw

MNIST on Kubeflow

This example guides you through the process of taking an example model, modifying it to run better within Kubeflow, and serving the resulting trained model.

Follow the version of the guide that is specific to how you have deployed Kubeflow

  1. MNIST on Kubeflow on GCP
  2. MNIST on Kubeflow on AWS
  3. MNIST on Kubeflow on Azure
  4. MNIST on Kubeflow on IBM Cloud
  5. MNIST on Kubeflow on vanilla k8s

MNIST on Kubeflow on GCP

Follow these instructions to run the MNIST tutorial on GCP

  1. Follow the GCP instructions to deploy Kubeflow with IAP

  2. Launch a Jupyter notebook

    • The tutorial has been tested using the Jupyter Tensorflow 1.15 image

  3. Launch a terminal in Jupyter and clone the kubeflow examples repo

    git clone https://github.com/kubeflow/examples.git git_kubeflow-examples

    • Tip When you start a terminal in Jupyter, run the command bash to start a bash terminal which is much more friendly then the default shell

    • Tip You can change the URL from '/tree' to '/lab' to switch to using Jupyterlab

  4. Open the notebook mnist/mnist_gcp.ipynb

  5. Follow the notebook to train and deploy MNIST on Kubeflow

MNIST on Kubeflow on AWS

Follow these instructions to run the MNIST tutorial on AWS

  1. Follow the AWS instructions to deploy Kubeflow on AWS

  2. Launch a Jupyter notebook

    • The tutorial has been tested using the Jupyter Tensorflow 1.15 image

  3. Launch a terminal in Jupyter and clone the kubeflow examples repo

    git clone https://github.com/kubeflow/examples.git git_kubeflow-examples

    • Tip When you start a terminal in Jupyter, run the command bash to start a bash terminal which is much more friendly then the default shell

    • Tip You can change the URL from '/tree' to '/lab' to switch to using Jupyterlab

  4. Open the notebook mnist/mnist_aws.ipynb

  5. Follow the notebook to train and deploy MNIST on Kubeflow

MNIST on Kubeflow on Azure

Follow these instructions to run the MNIST tutorial on Azure

  1. Follow the Azure instructions to deploy Kubeflow on Azure

  2. If you do not already have a notebook server, create a new server

  3. Launch a Jupyter notebook server

    • The tutorial has been tested using the Jupyter Tensorflow 1.15 image

  4. Launch a terminal in Jupyter and clone the kubeflow examples repo

    git clone https://github.com/kubeflow/examples.git git_kubeflow-examples

    • Tip When you start a terminal in Jupyter, run the command bash to start a bash terminal which is much more friendly then the default shell

    • Tip You can change the URL from '/tree' to '/lab' to switch to using Jupyterlab

  5. Open the notebook mnist/mnist_azure.ipynb

  6. Follow the notebook to train and deploy MNIST on Kubeflow

MNIST on Kubeflow on IBM Cloud

Follow these instructions to run the MNIST tutorial on IBM Cloud

  1. Follow the IBM Cloud instructions to deploy Kubeflow on IBM Cloud with IBM Cloud Block Storage.

  2. Launch a Jupyter notebook

    • The tutorial has been tested using the Jupyter Tensorflow 1.15 and 2.1 image with persistent storage.

  3. Launch a terminal in Jupyter and clone the kubeflow examples repo

    git clone https://github.com/kubeflow/examples.git git_kubeflow-examples

    • Tip When you start a terminal in Jupyter, run the command bash to start a bash terminal which is much more friendly then the default shell

    • Tip You can change the URL from '/tree' to '/lab' to switch to using Jupyterlab

  4. Open the notebook mnist/mnist_ibm.ipynb

  5. Follow the notebook to train and deploy MNIST on Kubeflow

MNIST on Kubeflow on Vanilla k8s

  1. Follow these instructions to deploy Kubeflow.

  2. Setup docker credentials.

  3. Launch a Jupyter Notebook

  • The tutorial is run on Jupyter Tensorflow 1.15 image.
  1. Launch a terminal in Jupyter and clone the kubeflow/examples repo
git clone https://github.com/kubeflow/examples.git git_kubeflow-examples

  1. Open the notebook mnist/mnist_vanilla_k8s.ipynb

  2. Follow the notebook to train and deploy on MNIST on Kubeflow

Prerequisites

Configure docker credentials

Why do we need this?

Kaniko is used by fairing to build the model every time the notebook is run and deploy a fresh model. The newly built image is pushed into the DOCKER_REGISTRY and pulled from there by subsequent resources.

Get your docker registry user and password encoded in base64

echo -n USER:PASSWORD | base64

Create a config.json file with your Docker registry url and the previous generated base64 string 

{
	"auths": {
		"https://index.docker.io/v1/": {
			"auth": "xxxxxxxxxxxxxxx"
		}
	}
}

Create a config-map in the namespace you're using with the docker config

kubectl create --namespace ${NAMESPACE} configmap docker-config --from-file=<path to config.json>

Source documentation: Kaniko docs