Weekly report 4

Weekly report 4

Hours used: 13

This week I worked almost only on backpropagation. I finished my notes about backpropagation and wrote a long document about deriving the gradient, and learned a lot in the process. Implementing it with matrices was rather unintuitive, but I used this and this site as guidelines. The latter had an implementation, but I didn't use their approach.

I struggled a lot: everything seemed to be the wrong shape or size all the time. I managed to finish the frame of the network, but there is most likely a bug somewhere: it can't overfit to XOR-problem data in 500 epochs. The loss doesn't actually change at all. The bulk of my time was spent debugging this, so I'm really annoyed that the network still does not work (it's a neural net-does-not-work).