.png)
AdaBoost for Regression
Now, let's use AdaBoost for regression on the California Housing dataset.
Step 1: Import Required Libraries
First, import the necessary Python libraries.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.ensemble import AdaBoostClassifier, AdaBoostRegressor
from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report, mean_squared_error
from sklearn.datasets import fetch_california_housingStep 2: Load and Explore the Dataset
housing = fetch_california_housing()
df_housing = pd.DataFrame(housing.data, columns=housing.feature_names)
df_housing['target'] = housing.targetStep 3: Split dataset
X_train, X_test, y_train, y_test = train_test_split(df_housing.drop(columns=['target']), df_housing['target'], test_size=0.2, random_state=42)Step 4: Initialize AdaBoost Regressor
adaboost_regressor = AdaBoostRegressor(base_estimator=DecisionTreeRegressor(max_depth=3), n_estimators=50, learning_rate=1.0, random_state=42)Step 5: Train the model
adaboost_regressor.fit(X_train, y_train)Step 6: Predict on Test Data
y_pred_reg = adaboost_regressor.predict(X_test)Step 7: Evaluate Performance
mse = mean_squared_error(y_test, y_pred_reg)
print(f'Mean Squared Error: {mse:.2f}')Key Takeaways – AdaBoost for Regression
Regression with AdaBoost: Uses weak learners (e.g., decision tree regressors) to minimize prediction errors iteratively.
Step-by-Step Process:
- Load California Housing Data – Predicts housing prices.
- Split Dataset – 80% training, 20% testing.
- Train AdaBoost Regressor – Uses DecisionTreeRegressor (max depth = 3) as the base estimator.
- Make Predictions – Predicts housing prices on test data.
- Evaluate Performance – Uses Mean Squared Error (MSE) for accuracy.
Model Performance:
- Achieved low MSE, indicating good predictive performance.
- Ensemble learning improves accuracy over individual weak regressors.
Final Thought: AdaBoost Regressor effectively reduces errors and enhances predictive accuracy by boosting weak learners.
Next Blog- Transfer Learning in Machine Learning
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
.png)
