• No products in the cart.

204.5.11 Hidden Layers and Their Roles

Understanding hidden layers.

Link to the previous post : https://statinfer.com/204-5-10-local-vs-global-minimum/

In this post we will understand one of the most important part of a Neural Network : Hidden Layers. We will be able to use this information to control the parameters related to hidden layers.

Hidden layers and their role

Multi Layer Neural Network

The Role of Hidden Layers

  • The First hidden layer
  • The first layer is nothing but the liner decision boundaries.
  • The simple logistic regression line outputs.
  • We can see them as multiple lines on the decision space.
  • The Second hidden layer.
  • The Second layer combines these lines and forms simple decision boundary shapes.
  • The third hidden layer forms even complex shapes within the boundaries generated by second layer.
  • You can imagine All these layers together divide the whole objective space into multiple decision boundary shapes, the cases within the shape are class-1 outside the shape are class-2.

The Number of Hidden Layers

  • There is no concrete rule to choose the right number. We need to choose by trial and error validation.
  • Too few hidden layers might result in imperfect models. The error rate will be high.
  • High number of hidden layers might lead to over-fitting, but it can be identified by using some validation techniques.
  • The final number is based on the number of predictor variables, training data size and the complexity in the target.
  • When we are in doubt, its better to go with many hidden nodes than few. It will ensure higher accuracy. The training process will be slower though.
  • Cross validation and testing error can help us in determining the model with optimal hidden layers.

The next post is a practice session on digit recognizer.

Link to the next post : https://statinfer.com/204-5-12-practice-digit-recognizer/

Statinfer

Statinfer derived from Statistical inference. We provide training in various Data Analytics and Data Science courses and assist candidates in securing placements.

Contact Us

info@statinfer.com

+91- 9676098897

+91- 9494762485

 

Our Social Links

top
© 2020. All Rights Reserved.