Link to the previous post : https://statinfer.com/204-5-8-neural-network-algorithm-demo/
Building the Neural Network
The good news is…
- We don’t need to write the code for weights calculation and updating.
- There ready-made codes, libraries and packages available in Python.
- The gradient descent method is not very easy to understand for a non mathematics students.
- Neural network tools don’t expect the user to write the code for the full length back propagation algorithm.
Building the Neural Network in Python
- We have a couple of packages available in Python.
- We need to mention the dataset, input, output & number of hidden layers as input.
- Neural network calculations are very complex. The algorithm may take sometime to produce the results.
- One need to be careful while setting the parameters. The run-time changed based on the input parameter values.
Python Code Options
- Step 1 : import the neurolab
- Step 2 : We need a epouch(error pouch)
error = [], In this we store error value of each iteration or epoach. - Step 3 : Create a network
- Here
nl.net.newff([[0, 1],[0,1]],[4,1],transf=[nl.trans.LogSig()] * 2)is feed forward neural network - 1st argument is min max values of predictor variables; can be a list of list
- 2nd argument is no.of nodes in each layer i.e 4 in hidden 1 in output layer
- transf is transfer function applied in each layer
- Here
- Step 4 : Train Network
- net.train() outputs error which is appended to error variable and has a few parameters
- Step 5 : Simulate network
- net.sim(input) gives the output for the network
Practice : Building the neural network in Python
- Build a neural network for XOR data.
- Dataset: Emp_Productivity/Emp_Productivity.csv
- Draw a 2D graph between age, experience and productivity.
- Build neural network algorithm to predict the productivity based on age and experience.
- Plot the neural network with final weights.
Manual Calculation of Weights XOR Dataset
In [37]:
## Importing the dataset
xor_data =pd.read_csv("datasets\\Gates\\xor.csv")
xor_data.shape
Out[37]:
In [38]:
xor_data.head()
Out[38]:
In [39]:
#Neural network building
#We use 'neurolab' package for implementing neural nets in python
#We need to manually downlaod and install neurolab
import neurolab as nl
import numpy as np
import pylab as pl
In [40]:
error = [] #In this we store error value of each iteration or epoach
# Create network with 1 hidden layer and random initialized
#nl.net.newff() is feed forward neural network
#1st argument is min max values of predictor variables
#2nd argument is no.of nodes in each layer i.e 4 in hidden 1 in o/p
#transf is transfer function applied in each layer
net = nl.net.newff([[0, 1],[0,1]],[4,1],transf=[nl.trans.LogSig()] * 2)
net.trainf = nl.train.train_rprop
In [41]:
# Training network
#net.train outputs error which is appended to error variable
error.append(net.train(xor_data[["input1"]+["input2"]], xor_data[["output"]], show=0, epochs = 100,goal=0.001))
In [42]:
#plotting epoches Vs error
#we can use this plot to specify the no.of epoaches in training to reduce time
pl.figure(1)
pl.plot(error[0])
pl.xlabel('Number of epochs')
pl.ylabel('Training error')
pl.grid()
pl.show()
In [43]:
# Simulate network(predicting)
predicted_values = net.sim(xor_data[["input1"]+["input2"]])
In [44]:
#converting predicted values into classes by using threshold
predicted_class=predicted_values
predicted_class[predicted_values>0.5]=1
predicted_class[predicted_values<=0.5]=0
In [45]:
#predicted classes
predicted_class
Out[45]:
In [46]:
#confusion matrix
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(xor_data[['output']],predicted_class)
print('ConfusionMatrix : ', ConfusionMatrix)
#accuracy
accuracy=(ConfusionMatrix[0,0]+ConfusionMatrix[1,1])/sum(sum(ConfusionMatrix))
print('Accuracy : ', accuracy)
#accuracy is 100% it classifying every training observation correctly
error=1-accuracy
print('Error :', error)
Building the neural network in Python on EMP_Productivity datasets
In [47]:
Emp_Productivity_raw = pd.read_csv("datasets\\Emp_Productivity\\Emp_Productivity.csv")
Emp_Productivity_raw.head(10)
Out[47]:
In [48]:
#Draw a scatter plot that shows Age on X axis and Experience on Y-axis. Try to distinguish the two classes with colors or shapes.
import matplotlib.pyplot as plt
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==0], s=10, c='b', marker="o", label='Productivity 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==1], s=10, c='r', marker="+", label='Productivity 1')
plt.legend(loc='upper left');
plt.show()
In [49]:
####Building neural net
import neurolab as nl
import numpy as np
import pylab as pl
error = []
# Create network with 1 layer and random initialized
net = nl.net.newff([[15, 60],[1,20]],[6,1],transf=[nl.trans.LogSig()] * 2)
net.trainf = nl.train.train_rprop
In [50]:
# Train network
error.append(net.train(Emp_Productivity_raw[["Age"]+["Experience"]], Emp_Productivity_raw[["Productivity"]], show=0, epochs = 500,goal=0.02))
# Simulate network
predicted_values = net.sim(Emp_Productivity_raw[["Age"]+["Experience"]])
In [51]:
#Converting Predictive values into Predected Classes
predicted_class=predicted_values
predicted_class[predicted_values>0.5]=1
predicted_class[predicted_values<=0.5]=0
#Predcited Classes
predicted_class[0:10]
Out[51]:
In [52]:
#confusion matrix
from sklearn.metrics import confusion_matrix as cm
ConfusionMatrix = cm(Emp_Productivity_raw[['Productivity']],predicted_class)
print('Confusion Matrix : ', ConfusionMatrix)
#accuracy
accuracy=(ConfusionMatrix[0,0]+ConfusionMatrix[1,1])/sum(sum(ConfusionMatrix))
print('Accuracy : ', accuracy)
error=1-accuracy
print('Error : ', error)
In [53]:
#plotting actual and prected classes
Emp_Productivity_raw['predicted_class']=pd.DataFrame(predicted_class)
import matplotlib.pyplot as plt
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==0], s=40, c='g', marker="o", label='Productivity 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.Productivity==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.Productivity==1], s=40, c='g', marker="x", label='Productivity 1')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.predicted_class==0],Emp_Productivity_raw.Experience[Emp_Productivity_raw.predicted_class==0], s=30, c='b', marker="1", label='Predicted 0')
ax1.scatter(Emp_Productivity_raw.Age[Emp_Productivity_raw.predicted_class==1],Emp_Productivity_raw.Experience[Emp_Productivity_raw.predicted_class==1], s=30, c='r', marker="2", label='Predicted 1')
plt.legend(loc='upper left');
plt.show()
In [54]:
# Simulate network with 'Age': 40; 'experience':12
x = np.array([[40,12]])
net.sim(x)
Out[54]:
With the threshold 0.5 the above output will be ‘0’.
The next post is about local vs global minimum.
Link to the next post : https://statinfer.com/204-5-10-local-vs-global-minimum/
Related Courses
Python(Batch6)
PRIVATE 28
Tableau (Batch6)
PRIVATE 29
PowerBI (Batch6)
PRIVATE 28
