203.5.11 Hidden Layers and Their Roles

Understanding hidden layers.

In previous section, we studied about Local vs. Global Minimum

In this post we will understand one of the most important part of a Neural Network : Hidden Layers. We will be able to use this information to control the parameters related to hidden layers.

Hidden layers and their role

Multi Layer Neural Network

The Role of Hidden Layers

  • The First hidden layer
  • The first layer is nothing but the liner decision boundaries
  • The simple logistic regression line outputs
  • We can see them as multiple lines on the decision space
  • The Second hidden layer
  • The Second layer combines these lines and forms simple decision boundary shapes
  • The third hidden layer forms even complex shapes within the boundaries generated by second layer.
  • You can imagine all these layers together divide the whole objective space into multiple decision boundary shapes, the cases within the shape are class-1 outside the shape are class-2.

The Number of Hidden Layers

  • There is no concrete rule to choose the right number. We need to choose by trial and error validation.
  • Too few hidden layers might result in imperfect models. The error rate will be high.
  • High number of hidden layers might lead to over-fitting, but it can be identified by using some validation techniques.
  • The final number is based on the number of predictor variables, training data size and the complexity in the target.
  • When we are in doubt, its better to go with many hidden nodes than few. It will ensure higher accuracy. The training process will be slower though.
  • Cross validation and testing error can help us in determining the model with optimal hidden layers.

The next post is about Practice Digit Recognizer.

27th January 2017

0 responses on "203.5.11 Hidden Layers and Their Roles"

Leave a Message

Blog Posts

Hurry up!!!

"use coupon code for FLAT 30% discount"  datascientistoffer        ___________________________________      Subscribe to our youtube channel. Get access to video tutorials.                

Contact Us

Statinfer Software Solutions#647 2nd floor 1st Main, Indira Nagar 1st Stage, 100 feet road,Indranagar Bangalore,Karnataka, Pin code:-560038 Landmarks: Opp. Namma Metro Pillar 48.

Connect with us

linkin fn twitter g

How to become a Data Scientist.?