Skip to content

Latest commit

 

History

History
345 lines (321 loc) · 19.5 KB

README.md

File metadata and controls

345 lines (321 loc) · 19.5 KB
Raw

Study Notes

Talk by Jakub Langr, author of GANs in Action

VAE = Variable auto encoder Some GANs still use auto encoders as part of architecture Most approaches use some form of maximum likelihood estimate (stats, not just ML) We start with a random seed, just to make sure we don't always get the same example Generate function is trainable

Generative = generate

Adversarial = they compete against each other

Network = there's two of them

the random seed forms the latent space

from this vector, what gives me a 4, what gives me a 9

how do we control the generation?

latent representation of the output distribution

random seed allows us to manipulate generation

i.e. if we JUST want to generate 4's

Discriminator supervised

binary classifcatiion

real or fake

Discriminator tries to fool generator, vice versa

likelihood that input example is real

Generator (forger) takens in a vector of random numbers

generates fake examples

Analogy generator = forger, trying to generate fake examples

discriminator = detective (what is real / fake)

back propogagtes error

Real data is fed in, and generator generates fake examples

GANs learn iteratively classification error - fed back into discriminator

Generation gets better each time

inspired by game theory

non cooperative game

minimise likelihood that real data is discriminated (classified) as fake

Generative modelling largely unsolved doesn't generalise well

can't generalise across domains

"What I cannot create, I do not understand"

Generator may fail if you initialise in the wrong place

so it's best to reduce dimensionality to begin with

ML Crash course by Google

ML Crash course

    pyplot module
      sudo apt-get -y install python3-pip
      curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
      python3 get-pip.py --force-reinstall
      sudo python3 get-pip.py --force-reinstall
      pip3 -V
      sudo pip3 install launchpadlib
      sudo pip3 install matplotlib
      python3                           <=== to access the above installed packages
    
    >>> import numpy as np
    >>> import matplotlib.pyplot as plt
    >>> 
    >>> x = np.arange(0, 5, 0.1);
    >>> y = np.sin(x)
    >>> plt.plot(x,y)
    >>> np.correlate(x, y)
    array([-21.37121514])
    >>> plt.acorr(x)
    (array([-10,  -9,  -8,  -7,  -6,  -5,  -4,  -3,  -2,  -1,   0,   1,   2,
             3,   4,   5,   6,   7,   8,   9,  10]), array([ 0.70105133,  0.73024119,  0.75965368,  0.78926407,  0.81904762,
            0.84897959,  0.87903525,  0.90918986,  0.93941868,  0.96969697,
            1.        ,  0.96969697,  0.93941868,  0.90918986,  0.87903525,
            0.84897959,  0.81904762,  0.78926407,  0.75965368,  0.73024119,
            0.70105133]), <matplotlib.collections.LineCollection object at 0x7fd920d876d8>, <matplotlib.lines.Line2D object at 0x7fd91f4f2be0>)
    >>> plt.acorr(x, y)
    /usr/lib/python3/dist-packages/matplotlib/pyplot.py:2497: MatplotlibDeprecationWarning: The 'hold' keyword argument is deprecated since 2.0.
      mplDeprecation)
    (array([-10,  -9,  -8,  -7,  -6,  -5,  -4,  -3,  -2,  -1,   0,   1,   2,
             3,   4,   5,   6,   7,   8,   9,  10]), array([ 0.70105133,  0.73024119,  0.75965368,  0.78926407,  0.81904762,
            0.84897959,  0.87903525,  0.90918986,  0.93941868,  0.96969697,
            1.        ,  0.96969697,  0.93941868,  0.90918986,  0.87903525,
            0.84897959,  0.81904762,  0.78926407,  0.75965368,  0.73024119,
            0.70105133]), <matplotlib.collections.LineCollection object at 0x7fd91f506240>, <matplotlib.lines.Line2D object at 0x7fd91f506908>)
    >>> plt.xcorr(x, y)
    (array([-10,  -9,  -8,  -7,  -6,  -5,  -4,  -3,  -2,  -1,   0,   1,   2,
             3,   4,   5,   6,   7,   8,   9,  10]), array([-0.26817328, -0.26478721, -0.26063553, -0.25578273, -0.25030029,
           -0.24426598, -0.23776309, -0.2308796 , -0.22370727, -0.21634076,
           -0.20887669, -0.15340143, -0.09629586, -0.03803299,  0.02090262,
            0.08001968,  0.13882507,  0.19682883,  0.25354895,  0.3085163 ,
            0.36127922]), <matplotlib.collections.LineCollection object at 0x7fd91f5062e8>, <matplotlib.lines.Line2D object at 0x7fd91f5134e0>)

            <=== Better understanding of this is required, maybe look at more examples
    cm module                           
            <===  more to learn, haven't seen much examples
    
    gridspec module
      gridspec is a module which specifies the location of the subplot in the figure
      Good example to start with: https://matplotlib.org/users/gridspec.html

    heatmap function
     http://seaborn.pydata.org/generated/seaborn.heatmap.html - lots of good examples, 
             <=== silly question, how do we display the graphs generated by the heatmap function()?
              Sort of answer - https://stackoverflow.com/questions/26597116/seaborn-plots-not-showing-up
  • pandas (for data manipulation)
    DataFrame class
    http://pandas.pydata.org/pandas-docs/stable/dsintro.html#dataframe
  • NumPy (for low-level math operations)
    linspace function
    random function
    array function
    arange function
      Why is this named 'arange', its like the tail function except returns the whole sequence except the last element
        - [scikit-learn](http://scikit-learn.org/) (for evaluation metrics)
    metrics module
    http://scikit-learn.org/stable/modules/classes.html#module-sklearn.metrics
    
    Install: http://scikit-learn.org/stable/install.html

ML Recipe course by Josh Gordon

        f(x) = y or y = f(x)
            where y is label (target)
            and x is data or feature
            f(…) is a function that transforms x to y i.e. classifier or activation function

(run from inside the tensorflow-for-poets-2 folder)
tensorboard --logdir tf_files/training_summaries &

wget https://upload.wikimedia.org/wikipedia/commons/c/ce/Bandung_Rose.jpg

python scripts/label_image.py --image Bandung_Rose.jpg