In my previous post, we briefly discussed about the neural network and its inner mechanics.In this post, we are going to code our own neural network in python.
Python is a programming language which is used for communicating with computer.To move forward with this post , please refer to my previous post for clear understanding.
First , let us see the code and decode it step by step.
That’s it..We implemented a simple neural network from scratch in 12 lines of code.The code and the algorithm we discussed in the previous post looks identical but with difference in python syntax.Let’s talk about the code line by line
1.) The input is set to 1 (input1 = 1)
2.) The original y value is 7.8 (y = 7.8).Our task is to find the weight which when multiplied by input1 gives y
3.) At the initial stage, weight is initialized to 1 (Some random guess).
4.) for loop runs from 0 till 100.Why? .In our algorithm, we used to find the weights by iteration i.e., we will guess some random number and then compute the error if the error is high then we will update the weights accordingly.This forms a loop.So we used for loop.
5.) yh = input1 * weight . In this , we compute our network output(not original y) and we use it to compare with original y
6.) error = y – yh .We compute the error term between the original output and our network output.
7.) If the error is 0, then we found the correct weight and we print the weight.
8.) Else , we compute the derivative of error with respect to the weight. If you compute ,it comes out to be the input1 term.How?
The negative term of the derivative indicates that, when we increase the weight the error decreases.
9.) The derivative term is divided by itself and multiplied by 0.1.Why?.Derivative only tells us whether to increase(If the derivative is negative) or decrease(If the derivative is positive) the weight , but doesn’t tell how much to increase.So to increase it by step by step, the term is divided and multiplied by 0.1.The 0.1 used here is the learning rate of our network.It allows us to update the weight step by step so that we arrive at the proper weight.Try yourselves adjusting the learning rate and see the results.To understand this , let us say our input is 1 and the y value is 4. Let us say if we update the weight by 5, then it exceeds the original weight (4) and we may not get the correct weight.So, to avoid this we update the weight by a tiny amount.
10. ) Then the weight is updated by the dedw.
To understand it more, try yourselves running the code in your system and use print statement where you feel you are unable to understand.All researchers doing deep learning does the same.Then why not you?..
Happy Coding !!!!
Harish's Blog 