simple feed forward neural network with 5 layers code examples

Guri Bee

3 min read

·

21-10-2024

1

0

Share

Demystifying Deep Learning: Building a 5-Layer Feedforward Neural Network

Deep learning, with its ability to tackle complex problems, has revolutionized various fields. One of the fundamental building blocks in deep learning is the feedforward neural network. This article aims to demystify this powerful tool by exploring a simple yet insightful example: a 5-layer feedforward neural network.

Understanding the Building Blocks

Before diving into code, let's understand the core components of a feedforward neural network:

• Layers: A feedforward network is structured in layers. Each layer consists of multiple interconnected neurons, performing simple calculations.
• Input Layer: The first layer, where data enters the network.
• Hidden Layers: The layers between the input and output layers, responsible for complex feature extraction.
• Output Layer: The final layer, producing the network's predictions.
• Weights and Biases: Each connection between neurons has an associated weight and bias, determining the strength of the connection and the neuron's activation threshold.
• Activation Function: Applied to each neuron's output, introducing non-linearity into the network and enabling it to learn complex relationships.

Code Example: Python with TensorFlow

This example demonstrates a simple 5-layer feedforward neural network for classifying handwritten digits using the MNIST dataset. This code is inspired by this GitHub repository by the TensorFlow team:

import tensorflow as tf

# Define the model
model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28, 28)),
  tf.keras.layers.Dense(128, activation='relu'), # Hidden Layer 1
  tf.keras.layers.Dense(128, activation='relu'), # Hidden Layer 2
  tf.keras.layers.Dense(128, activation='relu'), # Hidden Layer 3
  tf.keras.layers.Dense(10, activation='softmax') # Output Layer
])

# Compile the model
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# Train the model
model.fit(x_train, y_train, epochs=5)

# Evaluate the model
loss, accuracy = model.evaluate(x_test, y_test, verbose=0)
print('Accuracy: {}'.format(accuracy))

Explanation:

1. Data Loading: We assume the MNIST dataset is loaded into x_train, y_train, x_test, and y_test variables.
2. Model Definition:
   • We use tf.keras.models.Sequential to create a sequential model, where layers are stacked one after another.
   • Flatten() reshapes the input images into a 1D vector.
   • Three Dense layers with 128 neurons and ReLU activation serve as hidden layers.
   • The final Dense layer has 10 neurons with softmax activation, representing the probabilities of each digit class.
3. Compilation: We use 'adam' optimizer, 'sparse_categorical_crossentropy' loss function (suitable for multi-class classification), and 'accuracy' metric for evaluation.
4. Training: We train the model for 5 epochs, iterating through the training data multiple times.
5. Evaluation: We assess the model's performance on the test set and report the accuracy.

Beyond the Basics: Optimizing and Analyzing

This simple example provides a foundation for building more complex feedforward networks. Here are some key considerations:

• Choosing the right activation function: Different activation functions (like ReLU, sigmoid, tanh) have distinct properties. Experiment to find the best fit for your problem.
• Regularization: Techniques like dropout and L2 regularization can prevent overfitting and improve generalization.
• Hyperparameter tuning: Optimize parameters like the number of layers, neurons per layer, and learning rate to achieve the best performance.
• Early stopping: Monitor training performance to stop training when the model starts overfitting.

Further Exploration

This article provides a stepping stone to exploring the fascinating world of feedforward neural networks. To delve deeper, consider:

• Backpropagation: Understanding how the network learns by adjusting weights and biases through backpropagation is crucial for building intuition about deep learning.
• Convolutional Neural Networks (CNNs): For image-related tasks, CNNs are commonly used and offer advantages over basic feedforward networks.
• Recurrent Neural Networks (RNNs): For sequential data like text or time series, RNNs are powerful tools.

By understanding the basics of feedforward neural networks, you'll be equipped to explore and leverage the power of deep learning for various applications.