Learning Neural Networks

Artificial intelligence (AI) is slowly beginning to dominate over human beings in everything from Space Invaders, Jeopardy, and even gambling. This explosion in AI, propelled by a dramatic increase in computational power , demonstrates the amazing potential of AI to solve some of humanity’s biggest problems or on the flip side, kill us all.

But how does it work?

At the core of many artificial intelligence research is something called a neural network. It works very similar to how the neurons in your brain works.

It takes a series of inputs and spits out an output based on a series of matrices that mimics neural structure. Initially it is completely random and thus completely off (similar to how initially we are terrible at tasks such as programming), but over time it learns on its own through a method called backpropagation (similar to how experience slowly build our expertise) and becomes amazing at the task that the neural network is given.

$$\frac{dE}{dw_{jk}}=-(t_k-o_k)\cdot\sigma(\Sigma_jw_{jk}\cdot o_j)(1-\sigma(\Sigma_jw_{jk}\cdot o_j))\cdot o_j$$

The mathematics behind neural networks involve a great deal of calculus and a dash of linear algebra, in particular matrices, as such it can be a daunting task to learn. (I will definitely through the process in a separate post, but that is if I survive college apps). But there are already amazing resources to learn how to create a neural network and here are some below.

Make Your Own Neural Network

The book, Make Your Own Neural Network, assumes that you have absolutely no experience with matrices or programming (knowing calculus, though, would help), and as such is a great tool to learn how to create a neural network. It explores neural networks by teaching how to create one to classify hand-written numbers. The book explains matrices and Python programming going so far as to explain methods and classes. Furthermore, it builds up to neural networks with lessons on classifiers, linear regression, and gradient descent so the reader can see why neural networks are successful. I learned how to create a neural network this way, and I would definitely recommend this to beginners.

Coursera - Machine Learning

The Coursera Machine Learning course explores machine learning algorithms such as support vector machines, recommendation systems, neural networks, unsupervised learning, and more. This course is much heavier on the calculus and gradient calculations behind neural networks so knowing calculus is pretty much required, and it uses the Octave language which is relatively easy to pick up on. It is centered around machine learning which builds a solid foundation in understanding the logic behind neural networks. The videos are incredibly thorough but slightly time-intensive: each week has 6-8 videos each 15-20 minutes each. You can also get a certificate for this course (which might or might not be helpful).

Neural Network and Deep Learning Website

This website serves as a tutorial for neural networks by teaching the hello world of the neural network world: recognizing hand-written numbers. The website explains how neural networks work in great detail with drawings and graphs that really helps in understanding them; however, the calculus notation is probably more than what is needed for a beginner. The code is explained in a lot of detail and I would definitely recommend it.


If you are an audio learner then 3Blue1Brown is for you. The youtube channel has some amazing videos on machine learning and their animations really make neural networks easier to digest and understand, particular emphasis on great animation. The narration is great as well as how the topic is presented. 3Blue1Brown also cover topics such as linear algebra, quantum physics, and higher dimensions making it a channel worth subscribing to.

Siraj Raval

Siraj Raval is a channel dedicated to the latest computer science topics such as bitcoin, deep learning, and generative adversarial networks. The creator, Siraj Raval, puts a lot of effort into each video with each filled with fun animations. Following along with his tutorials on recurrent neural networks, tensorflow, and art generation, I learned a ton from his videos and was able to create some really interesting artificial intelligences. While it would be necessary to pause to digest what he is saying, it is almost crucial to subscribe to his channel if you are a beginner to artificial intelligence.