• LOGIN
  • No products in the cart.

204.5.9 Building a Neural Network in Python

Using package neuraolab.
Link to the previous post : https://statinfer.com/204-5-8-neural-network-algorithm-demo/

Building the Neural Network

The good news is…

  • We don’t need to write the code for weights calculation and updating.
  • There ready-made codes, libraries and packages available in Python.
  • The gradient descent method is not very easy to understand for a non mathematics students.
  • Neural network tools don’t expect the user to write the code for the full length back propagation algorithm.

Building the Neural Network in Python

  • We have a couple of packages available in Python.
  • We need to mention the dataset, input, output & number of hidden layers as input.
  • Neural network calculations are very complex. The algorithm may take sometime to produce the results.
  • One need to be careful while setting the parameters. The run-time changed based on the input parameter values.

Python Code Options

  • Step 1 : import the neurolab
  • Step 2 : We need a epouch(error pouch) error = [], In this we store error value of each iteration or epoach.
  • Step 3 : Create a network
    • Here nl.net.newff([[0, 1],[0,1]],[4,1],transf=[nl.trans.LogSig()] * 2) is feed forward neural network
    • 1st argument is min max values of predictor variables; can be a list of list
    • 2nd argument is no.of nodes in each layer i.e 4 in hidden 1 in output layer
    • transf is transfer function applied in each layer
  • Step 4 : Train Network
    • net.train() outputs error which is appended to error variable and has a few parameters
  • Step 5 : Simulate network
    • net.sim(input) gives the output for the network

Practice : Building the neural network in Python

  • Build a neural network for XOR data.
  • Dataset: Emp_Productivity/Emp_Productivity.csv
  • Draw a 2D graph between age, experience and productivity.
  • Build neural network algorithm to predict the productivity based on age and experience.
  • Plot the neural network with final weights.

Manual Calculation of Weights XOR Dataset

In [37]:
## Importing the dataset
xor_data =pd.read_csv("datasets\\Gates\\xor.csv")
xor_data.shape
Out[37]:
(4, 3)
In [38]:
xor_data.head()
Out[38]:
input1 input2 output
0 1 1 0
1 1 0 1
2 0 1 1
3 0 0 0
In [39]:
#Neural network building
#We use 'neurolab' package for implementing neural nets in python
#We need to manually downlaod and install  neurolab
import neurolab as nl
import numpy as np
import pylab as pl
In [40]:
error = []        #In this we store error value of each iteration or epoach

# Create network with 1 hidden layer and random initialized
#nl.net.newff() is feed forward neural network
#1st argument is min max values of predictor variables
#2nd argument is no.of nodes in each layer i.e 4 in hidden 1 in o/p
#transf is transfer function applied in each layer
net = nl.net.newff([[0, 1],[0,1]],[4,1],transf=[nl.trans.LogSig()] * 2)
net.trainf = nl.train.train_rprop
In [41]:
# Training network
#net.train outputs error which is appended to error variable
error.append(net.train(xor_data[["input1"]+["input2"]], xor_data[["output"]], show=0, epochs = 100,goal=0.001))
In [42]:
#plotting epoches Vs error
#we can use this plot to specify the no.of epoaches in training to reduce time
pl.figure(1)
pl.plot(error[0])
pl.xlabel('Number of epochs')
pl.ylabel('Training error')
pl.grid()
pl.show()
In [43]:
# Simulate network(predicting)
predicted_values = net.sim(xor_data[["input1"]+["input2"]])
In [44]:
#converting predicted values into classes by using threshold
predicted_class=predicted_values
predicted_class[predicted_values>0.5]=1
predicted_class[predicted_values<=0.5]=0
In [45]:
#predicted classes
predicted_class
Out[45]:
array([[ 0.],
       [ 1.],
       [ 1.],
       [ 0.]])
In [46]:
#confusion matrix
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(xor_data[['output']],predicted_class)
print('ConfusionMatrix : ', ConfusionMatrix)

#accuracy
accuracy=(ConfusionMatrix[0,0]+ConfusionMatrix[1,1])/sum(sum(ConfusionMatrix))
print('Accuracy : ', accuracy)

#accuracy is 100% it classifying every training observation correctly
error=1-accuracy
print('Error :', error)
ConfusionMatrix :  [[2 0]
 [0 2]]
Accuracy :  1.0
Error : 0.0

Building the neural network in Python on EMP_Productivity datasets

In [47]:
Emp_Productivity_raw = pd.read_csv("datasets\\Emp_Productivity\\Emp_Productivity.csv")
Emp_Productivity_raw.head(10)
Out[47]:
Age Experience Productivity Sample_Set
0 20.0 2.3 0 1
1 16.2 2.2 0 1
2 20.2 1.8 0 1
3 18.8 1.4 0 1
4 18.9 3.2 0 1
5 16.7 3.9 0 1
6 16.3 1.4 0 1
7 20.0 1.4 0 1
8 18.0 3.6 0 1
9 21.2 4.3 0 1
In [48]:
#Draw a scatter plot that shows Age on X axis and Experience on Y-axis. Try to distinguish the two classes with colors or shapes.
import matplotlib.pyplot as plt

fig = plt.figure()
ax1 = fig.add_subplot(111)

ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==0], s=10, c='b', marker="o", label='Productivity 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==1], s=10, c='r', marker="+", label='Productivity 1')
plt.legend(loc='upper left');
plt.show()
In [49]:
####Building neural net
import neurolab as nl
import numpy as np
import pylab as pl

error = []
# Create network with 1 layer and random initialized
net = nl.net.newff([[15, 60],[1,20]],[6,1],transf=[nl.trans.LogSig()] * 2)
net.trainf = nl.train.train_rprop
In [50]:
# Train network
error.append(net.train(Emp_Productivity_raw[["Age"]+["Experience"]], Emp_Productivity_raw[["Productivity"]], show=0, epochs = 500,goal=0.02))

# Simulate network
predicted_values = net.sim(Emp_Productivity_raw[["Age"]+["Experience"]])
In [51]:
#Converting Predictive values into Predected Classes
predicted_class=predicted_values
predicted_class[predicted_values>0.5]=1
predicted_class[predicted_values<=0.5]=0

#Predcited Classes
predicted_class[0:10]
Out[51]:
array([[ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.],
       [ 0.]])
In [52]:
#confusion matrix
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(Emp_Productivity_raw[['Productivity']],predicted_class)
print('Confusion Matrix : ', ConfusionMatrix)

#accuracy
accuracy=(ConfusionMatrix[0,0]+ConfusionMatrix[1,1])/sum(sum(ConfusionMatrix))
print('Accuracy : ', accuracy)

error=1-accuracy
print('Error : ', error)
Confusion Matrix :  [[75  1]
 [ 3 40]]
Accuracy :  0.966386554622
Error :  0.0336134453782
In [53]:
#plotting actual and prected classes
Emp_Productivity_raw['predicted_class']=pd.DataFrame(predicted_class)

import matplotlib.pyplot as plt

fig = plt.figure()
ax1 = fig.add_subplot(111)

ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==0], s=40, c='g', marker="o", label='Productivity 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==1], s=40, c='g', marker="x", label='Productivity 1')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.predicted_class==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.predicted_class==0], s=30, c='b', marker="1", label='Predicted 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.predicted_class==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.predicted_class==1], s=30, c='r', marker="2", label='Predicted 1')

plt.legend(loc='upper left');
plt.show()
In [54]:
# Simulate network with 'Age': 40; 'experience':12
x = np.array([[40,12]])
net.sim(x)
Out[54]:
array([[ 0.00012387]])

With the threshold 0.5 the above output will be ‘0’.

The next post is about local vs global minimum.

Link to the next post : https://statinfer.com/204-5-10-local-vs-global-minimum/

0 responses on "204.5.9 Building a Neural Network in Python"

Leave a Message