03B - Training Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training Models\n",
    "\n",
    "The central goal of machine learning is to train predictive models that can be used by applications. In Azure Machine Learning,  you can use scripts to train models leveraging common machine learning frameworks like Scikit-Learn, Tensorflow, PyTorch, SparkML, and others. You can run these training scripts as experiments in order to track metrics and outputs - in particular, the trained models.\n",
    "\n",
    "## Connect to Your Workspace\n",
    "\n",
    "The first thing you need to do is to connect to your workspace using the Azure ML SDK.\n",
    "\n",
    "> **Note**: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "\n",
    "# Load the workspace from the saved config file\n",
    "ws = Workspace.from_config()\n",
    "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Training Script\n",
    "\n",
    "You;re going to use a Python script to train a machine learning model based on the diabates data, so let's start by creating a folder for the script and data files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "training_folder = 'diabetes-training'\n",
    "os.makedirs(training_folder, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you're ready to create the training script and save it in the folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $training_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "diabetes = pd.read_csv('diabetes.csv')\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Set regularization hyperparameter\n",
    "reg = 0.01\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "# Save the trained model in the outputs folder\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use an Estimator to Run the Script as an Experiment\n",
    "\n",
    "Previously, you ran experiment scripts using a **RunConfiguration** and a **ScriptRunConfig**. You can use the same approach to run a training script, but using an **Estimator** is generally easier as it abstracts both of these configurations in a single object.\n",
    "\n",
    "In this case, we'll use a generic **Estimator** object to run the training experiment. Note that the default environment for this estimator does not include the **scikit-learn** package, so you need to explicitly add that to the configuration. The conda environment is built on-demand the first time the estimator is used, and cached for future runs that use the same configuration; so the first run will take a little longer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.estimator import Estimator\n",
    "from azureml.core import Experiment\n",
    "\n",
    "# Create an estimator\n",
    "estimator = Estimator(source_directory=training_folder,\n",
    "                      entry_script='diabetes_training.py',\n",
    "                      compute_target='local',\n",
    "                      conda_packages=['scikit-learn']\n",
    "                      )\n",
    "\n",
    "# Create an experiment\n",
    "experiment_name = 'diabetes-training'\n",
    "experiment = Experiment(workspace = ws, name = experiment_name)\n",
    "\n",
    "# Run the experiment based on the estimator\n",
    "run = experiment.submit(config=estimator)\n",
    "run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As with any experiment run, you can use the **RunDetails** widget to view information about the run and get a link to it in Azure Machine Learning studio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.widgets import RunDetails\n",
    "\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also retrieve the metrics and outputs from the **Run** object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register the Trained Model\n",
    "\n",
    "Note that the outputs of the experiment include the trained model file (**diabetes_model.pkl**). You can register this model in your Azure Machine Learning workspace, making it possible to track model versions and retrieve them later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Estimator'},\n",
    "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Parameterized Training Script\n",
    "\n",
    "You can increase the flexibility of your training experiment by adding parameters to your script, enabling you to repeat the same training experiment with different settings. In this case, you'll add a parameter for the regularization rate used by the Logistic Regression algorithm when training the model.\n",
    "\n",
    "Again, lets start by creating a folder for the parameterized script and the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, shutil\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "training_folder = 'diabetes-training-params'\n",
    "os.makedirs(training_folder, exist_ok=True)\n",
    "\n",
    "# Copy the data file into the experiment folder\n",
    "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's create a script containing a parameter for the regularization rate hyperparameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $training_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "import argparse\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# Set regularization hyperparameter\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--reg_rate', type=float, dest='reg', default=0.01)\n",
    "args = parser.parse_args()\n",
    "reg = args.reg\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "# load the diabetes dataset\n",
    "diabetes = pd.read_csv('diabetes.csv')\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use a Framework-Specific Estimator\n",
    "\n",
    "You used a generic **Estimator** class to run the training script, but you can also take advantage of framework-specific estimators that include environment definitions for common machine learning frameworks. In this case, you're using Scikit-Learn, so you can use the **SKLearn** estimator. This means that you don't need to specify the **scikit-learn** package in the configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.sklearn import SKLearn\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "# Create an estimator\n",
    "estimator = SKLearn(source_directory=training_folder,\n",
    "                    entry_script='diabetes_training.py',\n",
    "                    script_params = {'--reg_rate': 0.1},\n",
    "                    compute_target='local'\n",
    "                    )\n",
    "\n",
    "# Create an experiment\n",
    "experiment_name = 'diabetes-training'\n",
    "experiment = Experiment(workspace = ws, name = experiment_name)\n",
    "\n",
    "# Run the experiment\n",
    "run = experiment.submit(config=estimator)\n",
    "\n",
    "# Show the run details while running\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, we can get the metrics and outputs from the run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register A New Version of the Model\n",
    "\n",
    "Now that you've trained a new model, you can register it as a new version in the workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "# Register the model\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'Parameterized SKLearn Estimator'},\n",
    "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "# List registered models\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you've finished exploring, you can close this notebook and shut down your Compute Instance."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}