Step-by-Step Python Implementation of Neural Network for Regression

Objective:

We will build a neural network for regression using Python, TensorFlow, and Keras. The model will predict Boston house prices based on various features.

Step 1: Install Dependencies

Before proceeding, ensure you have the required libraries installed. If not, install them using:

pip install tensorflow scikit-learn numpy pandas matplotlib

Step 2: Import Libraries

import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt

Explanation:

• tensorflow and keras are used for building and training the neural network.
• sklearn.datasets provides the California housing dataset (alternative to Boston housing).
• train_test_split splits data into training and testing sets.
• StandardScaler normalizes the data for better performance.

Step 3: Load and Preprocess the Data

# Load the California housing dataset
housing = fetch_california_housing()
X = housing.data  # Feature matrix
y = housing.target  # Target variable (house prices)

# Split the dataset into training (80%) and testing (20%) sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Normalize the data for better performance
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

Explanation:

• fetch_california_housing() loads the dataset (Boston dataset is deprecated).
• X = housing.data contains numerical features (house attributes).
• y = housing.target represents median house prices.
• Data is split into training (80%) and testing (20%).
• StandardScaler() normalizes the dataset to improve performance.

Step 4: Build the Neural Network Model

# Define the neural network model
model = Sequential([
    Dense(64, activation='relu', input_shape=(X_train.shape[1],)),  # Input layer
    Dense(32, activation='relu'),  # Hidden layer
    Dense(1)  # Output layer (single neuron for regression)
])

# Compile the model
model.compile(optimizer='adam', loss='mse', metrics=['mae'])

# Print model summary
model.summary()

Explanation:

• Input Layer: Dense(64, activation='relu') with 64 neurons.
• Hidden Layer: Dense(32, activation='relu') for learning complex patterns.
• Output Layer: Dense(1) (single neuron) to predict a continuous value.
• Activation Functions:
  • ReLU is used in hidden layers for non-linearity.
  • No activation in the output layer (linear output for regression).
• Loss Function:
  • Mean Squared Error (MSE) minimizes the difference between predicted and actual values.
• Optimizer:
  • Adam is used for efficient learning.

Step 5: Train the Model

# Train the model
history = model.fit(X_train, y_train, epochs=100, batch_size=32, validation_data=(X_test, y_test))

Explanation:

• epochs=100 trains the model for 100 cycles.
• batch_size=32 processes 32 samples at a time for weight updates.
• validation_data=(X_test, y_test) helps monitor the model’s performance on unseen data.

Step 6: Evaluate the Model

# Evaluate the model on test data
loss, mae = model.evaluate(X_test, y_test)
print(f"Test MAE: {mae:.4f}")

Explanation:

• evaluate() calculates test loss (MSE) and Mean Absolute Error (MAE).
• A lower MAE indicates better performance.

Step 7: Make Predictions

# Make a prediction on a sample
sample = np.array([X_test[0]])  # Taking one test sample
prediction = model.predict(sample)
print(f"Predicted House Price: {prediction[0][0]:.2f} (in $100,000s)")

Explanation:

• We take one test sample and pass it through the trained model.
• predict() returns the estimated house price.

Step 8: Visualize Training Performance

# Plot training & validation loss
plt.plot(history.history['loss'], label='Train Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('MSE Loss')
plt.legend()
plt.title('Model Training Loss')
plt.show()

Explanation:

• The graph shows how the loss (MSE) decreases over epochs.
• A decreasing validation loss means the model is generalizing well.

Key Takeaways

1. Regression neural networks predict continuous values instead of classes.
2. Data preprocessing (normalization) is essential for better performance.
3. The Sequential API makes it easy to build deep learning models.
4. ReLU activation is preferred for hidden layers.
5. The output layer has a single neuron with no activation for continuous output.
6. Mean Squared Error (MSE) is the preferred loss function for regression.
7. Adam optimizer helps achieve faster convergence.
8. Mean Absolute Error (MAE) is a useful metric for model evaluation.
9. Training loss curves help diagnose overfitting or underfitting.

Next Blog- Perceptron