-by gagan
lets implement a neural network from scratch, in this implementation I will not be using any libraries such as PyTorch, TensorFlow, etc.. The aim of this blog is to understand what happens behind the hood, something which I always felt was hidden by using these libraries. Even if you are a complete beginner, you can follow through this, however I would advise you to start off with the basics of ML and linear algebra and some basic calculus before implementing this.
In the last blog we learnt about what neural networks were and how they evolved over time( if you haven’t read it, i do recommend to read this blog ). Now that we have the basics covered we are gonna implement this with code and some math. and yes, don’t get intimidated by the math part, i strongly advice you to spend a bit of time working out the math part using pen and paper and tinkering for a while to truly gain an intuition on the whats and the whys. That being said lets start off.
lets start with something very simple, this is a neuron taking a single input and giving the summation $z=w.x+b$ , y is the output after being activated by an activation function, here its the sigmoid function.
lets implement this in python:
import numpy as np
def sigmoid(x):
return 1 / (1 + np.exp(-x))
x = 0.7
w = 1.2
b = -0.4
z = w * x + b
y = sigmoid(z)
print(y)
We can write the same thing by creating a class named “Neuron”, the process of forward movement of the neural network by passing the weights and biases is known as feedforward.
class Neuron:
def __init__(self, weights, bias):
self.weights=weights
self.bias=bias
def feedforward(self, input):
out=np.dot(self.weights, input) + self.bias
return sigmoid(out)
# implementing a sample
w=np.array([1,2])
b=2
neuron1=Neuron(w, b)
x=np.array([3, 5])
final=neuron1.feedforward(x)
print(final)
the first layer is the input layer having 3 neurons, the 2nd layer is the hidden layer having 4 neurons and the 3rd layer is the output layer having 1 neuron activated by sigmoid function. here w11 means weight of the first neuron for first layer, w12 means weight of the second neuron for first layer, etc..
here is the code:
x=np.array([1.0, 2.0, -2.5])
w1=np.array([[2.0, 1.2, 3.2, 2.45], [2.23, 3.10, 3.30, 4.5], [-0.98, 2.09, -3.2, 0.8]])
b1=np.array([2.3, 4.5, 3.0, -1.2])
a1=sigmoid(np.dot(x, w1)+b1)
print("a1: \\n",a1)
w2=np.array([1.2, -0.90, 0.3, 2.0])
b2=3.22
a2=sigmoid(np.dot(w2, a1)+b2)
print(a2)
