• No products in the cart.

203.5.11 Hidden Layers and Their Roles

Understanding hidden layers.

In previous section, we studied about Local vs. Global Minimum

In this post we will understand one of the most important part of a Neural Network : Hidden Layers. We will be able to use this information to control the parameters related to hidden layers.

Hidden layers and their role

Multi Layer Neural Network

The Role of Hidden Layers

  • The First hidden layer
  • The first layer is nothing but the liner decision boundaries
  • The simple logistic regression line outputs
  • We can see them as multiple lines on the decision space
  • The Second hidden layer
  • The Second layer combines these lines and forms simple decision boundary shapes
  • The third hidden layer forms even complex shapes within the boundaries generated by second layer.
  • You can imagine all these layers together divide the whole objective space into multiple decision boundary shapes, the cases within the shape are class-1 outside the shape are class-2.

The Number of Hidden Layers

  • There is no concrete rule to choose the right number. We need to choose by trial and error validation.
  • Too few hidden layers might result in imperfect models. The error rate will be high.
  • High number of hidden layers might lead to over-fitting, but it can be identified by using some validation techniques.
  • The final number is based on the number of predictor variables, training data size and the complexity in the target.
  • When we are in doubt, its better to go with many hidden nodes than few. It will ensure higher accuracy. The training process will be slower though.
  • Cross validation and testing error can help us in determining the model with optimal hidden layers.

The next post is about Practice Digit Recognizer.

0 responses on "203.5.11 Hidden Layers and Their Roles"

Leave a Message


Statinfer derived from Statistical inference is a company that focuses on the data science training and R&D.We offer training on Machine Learning, Deep Learning and Artificial Intelligence using tools like R, Python and TensorFlow

Contact Us

We Accept

Our Social Links

How to Become a Data Scientist?

© 2020. All Rights Reserved.