Blogs

Machine Learning February 02 ,2025

AdaBoost for Regression

Now, let's use AdaBoost for regression on the California Housing dataset.

Step 1: Import Required Libraries

First, import the necessary Python libraries.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.ensemble import AdaBoostClassifier, AdaBoostRegressor
from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report, mean_squared_error
from sklearn.datasets import fetch_california_housing

Step 2: Load and Explore the Dataset

housing = fetch_california_housing()
df_housing = pd.DataFrame(housing.data, columns=housing.feature_names)
df_housing['target'] = housing.target

Step 3: Split dataset

X_train, X_test, y_train, y_test = train_test_split(df_housing.drop(columns=['target']), df_housing['target'], test_size=0.2, random_state=42)

Step 4: Initialize AdaBoost Regressor

adaboost_regressor = AdaBoostRegressor(base_estimator=DecisionTreeRegressor(max_depth=3), n_estimators=50, learning_rate=1.0, random_state=42)

Step 5: Train the model

adaboost_regressor.fit(X_train, y_train)

Step 6: Predict on Test Data

y_pred_reg = adaboost_regressor.predict(X_test)

Step 7: Evaluate Performance

mse = mean_squared_error(y_test, y_pred_reg)
print(f'Mean Squared Error: {mse:.2f}')

Key Takeaways – AdaBoost for Regression

Regression with AdaBoost: Uses weak learners (e.g., decision tree regressors) to minimize prediction errors iteratively.

Step-by-Step Process:

Load California Housing Data – Predicts housing prices.
Split Dataset – 80% training, 20% testing.
Train AdaBoost Regressor – Uses DecisionTreeRegressor (max depth = 3) as the base estimator.
Make Predictions – Predicts housing prices on test data.
Evaluate Performance – Uses Mean Squared Error (MSE) for accuracy.

Model Performance:

Achieved low MSE, indicating good predictive performance.
Ensemble learning improves accuracy over individual weak regressors.

Final Thought: AdaBoost Regressor effectively reduces errors and enhances predictive accuracy by boosting weak learners.

Next Blog- Transfer Learning in Machine Learning

Purnima

You must logged in to post comments.