Simple Neural Network + Keras

A Simple Neural Network (SNN), also often referred to as a Multi-Layer Perceptron (MLP), is a foundational artificial neural network architecture. It typically consists of an input layer, one or more hidden layers, and an output layer. Each layer is composed of nodes (neurons) that are connected to nodes in the subsequent layer. Information flows forward through the network: input data is fed into the input layer, processed through weighted sums and activation functions in the hidden layers, and finally yields an output from the output layer. SNNs are capable of learning complex patterns from data and are widely used for tasks such as classification, regression, and pattern recognition.

Keras is a high-level, user-friendly API for building and training deep learning models. It runs on top of popular machine learning frameworks like TensorFlow, Theano, or Microsoft Cognitive Toolkit (CNTK), with TensorFlow being the most common backend. Keras was designed for fast experimentation and ease of use, making it an excellent choice for both beginners and experienced practitioners. It simplifies the process of defining complex neural network architectures, handling data preprocessing, training models, and making predictions.

Combining a Simple Neural Network with Keras provides an incredibly efficient and intuitive way to implement, train, and evaluate these models. Keras abstracts away much of the underlying complexity, allowing users to focus on the model's architecture and hyper-parameters. Key Keras components for building SNNs include:
- `Sequential` Model: A linear stack of layers, suitable for most feed-forward networks.
- `Dense` Layer: The most common layer type, representing a fully connected layer where each neuron receives input from all neurons in the previous layer.
- Activation Functions: Non-linear functions applied to the output of each neuron (e.g., `relu` for hidden layers, `sigmoid` for binary classification output, `softmax` for multi-class classification output).
- `compile()` method: Used to configure the learning process by specifying the optimizer (e.g., `adam`, `sgd`), loss function (e.g., `binary_crossentropy`, `categorical_crossentropy`, `mse`), and metrics to monitor during training (e.g., `accuracy`).
- `fit()` method: Trains the model using the provided training data and labels, iterating for a specified number of epochs.
- `predict()` method: Generates predictions for new input data after the model has been trained.

The example code below demonstrates how to build and train a simple neural network using Keras for a binary classification task. It involves generating synthetic data, defining a `Sequential` model with `Dense` layers, compiling the model, training it, and then evaluating its performance.

Example Code

import numpy as np
import tensorflow as tf
from tensorflow import keras
from sklearn.model_selection import train_test_split
from sklearn.datasets import make_classification
from sklearn.metrics import accuracy_score

 1. Generate Synthetic Data for Binary Classification
 X: features, y: labels
X, y = make_classification(n_samples=1000, n_features=20, n_informative=10, n_redundant=10, random_state=42)

 Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

print(f"X_train shape: {X_train.shape}")
print(f"y_train shape: {y_train.shape}")
print(f"X_test shape: {X_test.shape}")
print(f"y_test shape: {y_test.shape}")

 2. Build a Simple Neural Network (MLP) using Keras Sequential API
model = keras.Sequential([
     Input layer and first hidden layer
     units: number of neurons in the layer
     input_shape: required for the first layer, specifies the shape of one input sample (excluding batch size)
    keras.layers.Dense(units=32, activation='relu', input_shape=(X_train.shape[1],)),
    
     Second hidden layer
    keras.layers.Dense(units=16, activation='relu'),
    
     Output layer
     For binary classification, use 1 unit and 'sigmoid' activation
     Sigmoid outputs a probability between 0 and 1
    keras.layers.Dense(units=1, activation='sigmoid')
])

 3. Compile the model
 optimizer: Algorithm to update weights during training (e.g., 'adam' is a good default)
 loss: Function to minimize during training. For binary classification, 'binary_crossentropy' is common.
 metrics: Metrics to monitor during training and evaluation (e.g., 'accuracy')
model.compile(optimizer='adam',
              loss='binary_crossentropy',
              metrics=['accuracy'])

 Display the model summary
print("\nModel Summary:")
model.summary()

 4. Train the model
print("\nTraining the model...")
history = model.fit(X_train, y_train,
                    epochs=50,        Number of times to iterate over the entire training dataset
                    batch_size=32,    Number of samples per gradient update
                    validation_split=0.1,  Fraction of the training data to be used as validation data
                    verbose=0)       Set to 1 or 2 for more verbose output during training

print("Training finished.")

 5. Evaluate the model on the test data
loss, accuracy = model.evaluate(X_test, y_test, verbose=0)
print(f"\nTest Loss: {loss:.4f}")
print(f"Test Accuracy: {accuracy:.4f}")

 6. Make predictions on new data
 Predictions will be probabilities (between 0 and 1)
y_pred_proba = model.predict(X_test)

 Convert probabilities to binary class labels (0 or 1)
y_pred = (y_pred_proba > 0.5).astype(int)

 Calculate accuracy using sklearn (should be similar to model.evaluate accuracy)
sklearn_accuracy = accuracy_score(y_test, y_pred)
print(f"Sklearn Calculated Accuracy: {sklearn_accuracy:.4f}")

print("\nSample Predictions (first 5 test samples):")
for i in range(5):
    print(f"Actual: {y_test[i]}, Predicted Probability: {y_pred_proba[i][0]:.4f}, Predicted Class: {y_pred[i][0]}")

Simple Neural Network + Keras

Example Code

Related Topics