Convolutional Neural Network from scratch in Matlab
In order to understand how Convolutional Neural Networks work, we've built a convolutional neural network from scratch in Matlab, using barebones Matrix operations to realize this vision.
We're using the MNIST dataset (the Hello World dataset of Machine Learning) to evaluate our CNN. Our CNN consists of:
- Convolution Layer (32 filters) - Relu
- Convolution Layer (64 filters) - Relu
- Max Pooling Layer
- Fully Connected Layer (128 nodes) - Relu
- Fully Connected Layer (10 nodes) - Softmax (Stable Version)
The loss function being used is Categorical Cross Entropy.
We're using the stochachistic gradient descent algorithm with fixed learning rate and gradient clipping to train the Neural Network.
On small samples it is giving promising results, however the effectiveness of the CNN can't be commented on right now since we're evaluating its performance
and accuracy since it is taking a long time to train, in no small part due to the code not being vectorized in some places and since we only have a CPU at our disposal.
After training our Convolutional Neural Network on the full training set, we were able to achieve a 97.58% accuracy on the test set.
We used the following references in order to derive the equations and adapt them as per our needs.