-1-overview.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Goals\n",
    "\n",
    "- Train a deep neural network!\n",
    "- On not-a-toy-problem\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Bonus Goals\n",
    "- Get `> 80%` accuracy, or\n",
    "- Explain `0.687`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Warnings\n",
    "\n",
    "- Not much theory or explanation\n",
    "- Not much time to get everything done\n",
    "- Requires motivation and inquisitiveness\n",
    "- Reading and running someone else's code (aka Programming)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### Worse\n",
    "\n",
    "Reading and running a researcher's code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Tools\n",
    "\n",
    "+ `conda` & `git` - for reproducible setups\n",
    "+ `python3` - programming language of choice\n",
    "+ `jupyter` - reproducible workflows\n",
    "+ `keras` - frontend for deep learning\n",
    "+ `tensorflow` - backend for ML, esp. deep learning\n",
    "+ `numpy` - non-ML numerical processing \n",
    "+ `librosa` - audio processing\n",
    "+ [`audioset`](https://research.google.com/audioset/) - for embeddings (more later)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Programming"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Transforming data from one form to another\n",
    "\n",
    "```\n",
    "INPUTS ---magic---> OUTPUTS\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "where:\n",
    "\n",
    "`magic` is a very specific set of instructions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Audio Segmentation\n",
    "\n",
    "\n",
    "- `INPUT` - audio\n",
    "- `OUTPUT` - frame-wise labels\n",
    "- [picture](https://goo.gl/images/NYHTSK)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "### `magic`\n",
    "\n",
    "- Metric based\n",
    "- Model based (today)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Model Based Audio Segmentation\n",
    "\n",
    "![workflow.png](workflow.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Acquire and Prepare Data\n",
    "\n",
    "`01-data-acquisition-analysis-splitting.ipynb`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Preprocessing\n",
    "\n",
    "Extract [VGGish](https://github.com/tensorflow/models/tree/master/research/audioset) embeddings for each audio in the splits and save separate `tfrecord` for each split.\n",
    "+ Load audio into 16KHz, normalized, single-channel array\n",
    "+ Extract log-mel-spectograms on Hann windows of 0.025s, every 0.010s \n",
    "+ Prepare frames of size 0.96s for VGGish, every 0.12s \n",
    "+ Get 128-dimensional embeddings every 0.12s from VGGish\n",
    "+ Relevant files: \n",
    "    * `02-feature-extraction-vggish_embedding.py`\n",
    "    * `feat_ext.py` - for reusable functions/classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Supervised Training Workflow\n",
    "\n",
    "What we will be doing today\n",
    "\n",
    "[picture](https://goo.gl/images/J4V6mu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- *Make sure your computer won't run out of power*\n",
    "- Duplicate the base notebook `03-training-keras_2mlp_clsw0.ipynb`\n",
    "    - Rename to `<nn>-training-keras_<model_info>.ipynb`\n",
    "    - Modify and train a [Multi-Layer Perceptron (MLP)](https://www.google.de/search?q=multilayer+perceptron)\n",
    "    - Evaluate on the validation split\n",
    "    - Save learned weights"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python [conda env:ml-school]",
   "language": "python",
   "name": "conda-env-ml-school-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}