MP Neuron

The MP neuron is mankind’s first simplified mathematical model of the neuron. This model was developed by McCulloch and Pitts in 1943. The MP neuron model is also known as the linear threshold gate model. They are widely used in proving logic functions.

Now let’s look into the model. It has 4 basic components :

1. The model takes inputs (x1, x2, …., xm)
2. Applies Adder function(g) and
3. Takes decision in Activation function (f)
4. Gives an output Y

Characteristics of MP Neuron :

1. When MP neurons are modeled as neural networks, they are connected by directed weighted paths in a neural network which we will study later.
2. When we pass the Adder function value in the Activation function of an MP neuron, there are 2 possibilities: the neuron may fire (label 0) or it does not fire (label 1).
3. The activation function is based on the threshold value. There is a fixed threshold for each neuron and if the net input to the neuron is greater than the threshold then the neuron fires.

Weights(w): It is the parameter that shows the contributing power of the input feature towards the output. A low weight value will have no change on the input and a high weight will have a more significant change on the output.

Adder Function(g): It is an Aggregation function that performs the sum of the product of the inputs with the weights and gets Adder value.

Activation Function(f): It is the mathematical function that decides whether neuron input is relevant for model prediction or not.

Threshold value(b):

It is the value in the activation function based on which the activation function takes its decision. Initially, we take any value from 0 to n (where n is maximum adder value) and then iterate over it and find the total loss of the model. Then we can choose the value of the threshold, such that the loss is minimum. This is the brute force method by which we calculate the threshold value.

From the above image :

g(x) is the aggregated sum of all weighted inputs

y is the final value predicted by an activation function

b is the threshold value which is calculated by the brute force method

Let us discuss the representation of the boolean functions in MP neurons. In these examples, the threshold value is calculated by the brute force method which we study later in this blog.

AND function :

Here x1 , x2 ,x3 are inputs and w1, w2 and w3 are its weight respectively

Threshold value = 3

Aggregated value :

g(x) : x1 + x2 + x3

Activation function :

g(x) >3 Output=1

g(x)<=3 Output=0

Example:

Inputs= 0, 1, 1

weights= 1, 1, 1

g(x) = 0 x 1+ 1 x 1 + 1 x 1 =2

2<=3

Output = 0 . So we implement AND function through MP neuron

OR function :

Here x1, x2, x3 are inputs and w1, w2 and w3 are its weight respectively

Threshold value= 1

Aggregated value :

g(x) : x1 + x2 + x3

Activation function :

g(x) >=1 Output=1

g(x)<=1 Output=0

Example:

Inputs= 0, 1, 0

Weights= 1, 1, 1

g(x) = 0 x 1 + 1 x 1 +0 x 1 =1

1>=1

Output = 1 .So we implement OR function through MP neuron

Till now we saw how models can be used for a boolean function, Now let’s try if we can use it in a real-world scenario.

A real-life simple example of MP Neuron :

Let us understand the working procedure of MP Neuron through an example.

A student wants to purchase a competitive programming book. The shopkeeper shows 3 books in which the first book is written by author A, the second book is written by author B, and the third book is written by author A. he prices of books are 100, 180, and 150 respectively.

Now he comes is in a dilemma about which book to buy so he uses the MP neuron model to solve this problem.

Consider : Input 0 = author A and Input 1 = author B

Let’s take the average of prices to find a threshold value, we get 143. Here 100, 180, and 150 are weights of the 3 books respectively.

Adder Function Value : (Input * Weights)

0 x 100 + 1 x 180 +0 x 150 = 180

Activation function :

If 180 < 143 output = 0

If 180 > 143 output =1

So final output =1

Since output 1 is for author B

Therefore student choose the book by author B

This is the procedure through which MP Neuron works.

Problem Statement :

Now comes the problem statement to get a clear understanding of the MP neuron model.

Now this problem statement has more features and we also consider loss, error, and accuracy.

You all may go to the movie theatre, some people don’t go to their nearest movie theatre and go to expensive ones. How they choose which movie theatre is best for them.

They choose from their experience by visiting the theatres and by considering other factors like ticket prices, location, whether it is multiplex or not, Time and comfortability to reach the theatre, seats available, etc.

Given below is the dataset in which four features are given based on which it can be predicted if a person would visit a particular theatre or not :

This dataset has 6 rows (input) and 5 features (columns) out of which 4 are input features and 1 are output features.

Features are Ticket Price, Total Seats, Multiplex, Reachability, and the last feature Preferred which will predict the output.

To pass our table dataset in our model we have to convert its values into binary format 0 or 1.

But there should be some pattern to convert our data values of different ranges in the table into binary.

For yes/ no and easy/difficult, we can easily assign 0 and 1.

So the simple solution is we have to take average values of all features and assign 0 and 1 according to low high from average as shown in below table :

Now the next step is to assign weights to each feature and find the Adder Value using Adder function (g).

We assign values to the weights by considering which feature is more contributing towards the output. Since Ticket Price, Total Seats, and Reachability is more contributing towards the output hence its weights w1, w2, w4 is assigned 1 and Multiplex is least contributing towards the output, therefore weight w3 is assigned zero.

Adder Function = Ticket price x w1+ Total seats x w2 + Multiplex x w3 + Reachability x w4

Next step is to pass our adder function in the Activation function

There are a lot of activation function we study in deep learning like

1. Sigmoid / Logistic function
2. TanH / Hyperbolic Tangent
3. Relu function
4. Leaky Relu
5. Softmax etc.

But we use the Sigmoid function here because it produces output only 0 or 1.

We require Threshold value b to use the activation function. So we analyze the output of Adder and then by brute force decide the threshold value.

So by approaching the brute force method to find threshold value we have 3 values available b=1, 2, and 3. We then predicted output y for each particular threshold respectively, as shown in the last 3 columns of the above table. Now we have to choose the right one.

To decide the correct threshold value we have to find loss and for the loss, we calculated Mean square error and Mean Difference for each threshold value.

Here,

On calculating Mean Square Error, we found it is least for b=2. Therefore we choose threshold value as 2 as its error is minimum so its loss is also minimum.

Now final table we get :

Now test our model for the last row

Input = ( 0 , 1 , 1 ,0 )

Weights =(1, 1, 0, 1)

Adder value = 0 x 1 + 1 x 1 + 1 x 0 + 0 x 1 = 1

Threshold value = 2

Activation function :

Adder value≥2 :Output=1

Adder value<2 :Output=0

Now, 1<2

Predicted output: 0

Real Output: 0

This also shows theta 2 is the correct threshold value

This is the whole solution for our problem statement using Mp Neuron.

Geometrical Representation of MP Neuron

Let us take an example to understand the geometrical representation of MP neuron

I use another dataset of previous examples for simplicity.

We can take Ticket Price at x-axis and Total Seats on the y-axis while plotting on the x-y graph.

Threshold value, b= 1

Mathematical equation through activation function :

Ticket price x w1+ Total Seats x w2 > =Threshold Value , Output =1

x1 + x2-b =0

Therefore, x1 + x2–1 =0

We will get a graph after plotting :

Points on the line and above the line are giving output 1 and points below the line give output 0. Our decision boundary as a line works to predict the correct output. This is a geometrical representation of the MP neuron model.

Similarly, we can also make boolean functions for AND, OR & NOT, etc.

Nowadays in our models, we no longer use the MP neuron. Many advanced models have come up which I will discuss in later blogs.

We try to understand the coding part of MP Neuron in the next blog.

Thanks for your time!