Day 37: 60 days of Data Science and Machine Learning Series
Advanced Regression Techniques with project ( Part 1) …
Welcome back peeps. Hope your holidays are going well. I’m relaxing/rejuvenating on an island (on a beach) writing this post and sipping Piña colada!
Some of the other best Series —
100 days : Your Data Science and Machine Learning Degree Series with projects
Complete Data Visualization and Pre-processing Series with projects
Projects Videos —
All the projects, data structures, SQL, algorithms, system design, Data Science and ML , Data Analytics, Data Engineering, , Implemented Data Science and ML projects, Implemented Data Engineering Projects, Implemented Deep Learning Projects, Implemented Machine Learning Ops Projects, Implemented Time Series Analysis and Forecasting Projects, Implemented Applied Machine Learning Projects, Implemented Tensorflow and Keras Projects, Implemented PyTorch Projects, Implemented Scikit Learn Projects, Implemented Big Data Projects, Implemented Cloud Machine Learning Projects, Implemented Neural Networks Projects, Implemented OpenCV Projects,Complete ML Research Papers Summarized, Implemented Data Analytics projects, Implemented Data Visualization Projects, Implemented Data Mining Projects, Implemented Natural Leaning Processing Projects, MLOps and Deep Learning, Applied Machine Learning with Projects Series, PyTorch with Projects Series, Tensorflow and Keras with Projects Series, Scikit Learn Series with Projects, Time Series Analysis and Forecasting with Projects Series, ML System Design Case Studies Series videos will be published on our youtube channel ( just launched).
Subscribe today!
Tech Newsletter —
If you are interested, you can join my newsletter through which I send tech interview tips, techniques, patterns, hacks — Software Development, ML, Data Science, Startups and Technology projects to more than 30K readers. You can subscribe to Tech Brew :
Well, this post is going to be the part 2 of Advanced Regression Techniques with project ( Part 1) that can be found in the link below.
In this post, we will cover different regression techniques that you can use to build your model.
Let’s dive in ( before I dive in the sea ;)) —
Declare Features ( X and Y)
X = ht_df[["TotalBsmtSF","1stFlrSF","FullBath","TotRmsAbvGrd","YearBuilt","YearRemodAdd","OverallQual","GrLivArea","GarageCars","GarageArea"]]
y= ht_df['SalePrice']Split into train and test data
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=42)Rescale the data, Cross Validation and Evaluation Metrics
pipeline = Pipeline([
('std_scalar', StandardScaler())
])X_train = pipeline.fit_transform(X_train)
X_test = pipeline.transform(X_test)def cross_val(model):
pred = cross_val_score(model, X, y, cv=5)
return pred.mean()def print_evaluate(t, p):
mae = metrics.mean_absolute_error(t, p)
mse = metrics.mean_squared_error(t, p)
rmse = np.sqrt(metrics.mean_squared_error(t, p))
r2_square = metrics.r2_score(t, p)
print('Mean Squared Error:', mse)
print('Root Mean Squared Error:', rmse)
print('Mean Absoulte Error:', mae)
print('R2 Square:', r2_square)
print('-'* 8)Linear Regression
from sklearn.linear_model import LinearRegressionlinear_reg = LinearRegression(normalize=True)
linear_reg.fit(X_train,y_train)test_pred = lin_reg.predict(X_test)
train_pred = lin_reg.predict(X_train)print('Train set evaluation:')
print_evaluate(y_train, train_pred)print('Test set evaluation:')
print_evaluate(y_test, test_pred)Output —
Test set evaluation:
Mean Squared Error: 1.784092e+09
Root Mean Squared Error: 42238.512698
Mean Absoulte Error: 24612.044039
R2 Square: 0.74433
--------SVM Regression
from sklearn.svm import SVRsvm_reg_model = SVR(kernel='rbf', C=1000000, epsilon=0.001)
svm_reg_model.fit(X_train, y_train)test_pred = svm_reg_model.predict(X_test)
train_pred = svm_reg_model.predict(X_train)print('Train set evaluation:')
print_evaluate(y_train, train_pred)print('Test set evaluation:')
print_evaluate(y_test, test_pred)Output —
Train set evaluation:
Mean Squared Error: 356240223.75021607
Root Mean Squared Error: 18874.327107216726
Mean Absoulte Error: 9666.126442526473
R2 Square 0.9408098767474895
--------
Test set evaluation:
Mean Squared Error: 873770430.0569018
Root Mean Squared Error: 29559.608083614738
Mean Absoulte Error: 20073.42020251248
R2 Square: 0.8747837933981435
--------Random Forest Regression
from sklearn.ensemble import RandomForestRegressorrandom_reg_model = RandomForestRegressor(n_estimators=1500)
random_reg_model.fit(X_train, y_train)test_pred = random_reg_model.predict(X_test)
train_pred = random_reg_model.predict(X_train)print('Train set evaluation:')
print_evaluate(y_train, train_pred)print('Test set evaluation:')
print_evaluate(y_test, test_pred)Output —
Train set evaluation:
Mean Squared Error: 164817856.78464976
Root Mean Squared Error: 12838.140705906357
Mean Absoulte Error: 7629.598502997546
R2 Square 0.972615138305835
--------
Test set evaluation:
Mean Squared Error: 742703900.752294
Root Mean Squared Error: 27252.5943857148
Mean Absoulte Error: 17884.901544103792
R2 Square: 0.893566362649114
--------SGD Regression
from sklearn.linear_model import SGDRegressorsgd_reg_model = SGDRegressor(n_iter_no_change=150, penalty=None, eta0=0.0001, max_iter=1000, tol=1e-3)
sgd_reg_model.fit(X_train, y_train)test_pred = sgd_reg_model.predict(X_test)
train_pred = sgd_reg_model.predict(X_train)print('Train set evaluation:')
print_evaluate(y_train, train_pred)print('Test set evaluation:')
print_evaluate(y_test, test_pred)Output —
Train set evaluation:
Mean Squared Error: 1491518433.5065699
Root Mean Squared Error: 38620.18168660746
Mean Absoulte Error: 23894.374559726748
R2 Square 0.7521808206179814
--------
Test set evaluation:
Mean Squared Error: 1433731798.722035
Root Mean Squared Error: 37864.6510445037
Mean Absoulte Error: 24187.28838012137
R2 Square: 0.7945381865248764
--------Elastic Net
from sklearn.linear_model import ElasticNetelastic_model = ElasticNet(alpha=0.1, l1_ratio=0.9, selection='random', random_state=42)
elastic_model.fit(X_train, y_train)test_pred = elastic_model.predict(X_test)
train_pred = elastic_model.predict(X_train)print('Train set evaluation:')
print_evaluate(y_train, train_pred)print('Test set evaluation:')
print_evaluate(y_test, test_pred)Output —
Train set evaluation:
Mean Squared Error: 1467441330.190119
Root Mean Squared Error: 38307.19684589462
Mean Absoulte Error: 23951.8806845575
R2 Square 0.7561812860844062
--------
Test set evaluation:
Mean Squared Error: 1390440655.6722608
Root Mean Squared Error: 37288.61294915997
Mean Absoulte Error: 24287.598088519848
R2 Square: 0.8007420502923858
--------Lasso Regression
from sklearn.linear_model import Lassolasso_model = Lasso(alpha=0.1,
precompute=True,
selection='random',
random_state=42)
lasso_model.fit(X_train, y_train)test_pred = lasso_model.predict(X_test)
train_pred = lasso_model.predict(X_train)print('Train set evaluation:')
print_evaluate(y_train, train_pred)print('Test set evaluation:')
print_evaluate(y_test, test_pred)Output —
Train set evaluation:
Mean Squared Error: 1467114732.084582
Root Mean Squared Error: 38302.933726864605
Mean Absoulte Error: 23994.145203746753
R2 Square 0.7562355511023124
--------
Test set evaluation:
Mean Squared Error: 1387789082.8139398
Root Mean Squared Error: 37253.04125590205
Mean Absoulte Error: 24355.84074739878
R2 Square: 0.8011220355647481
--------Ridge Regression
from sklearn.linear_model import Ridgeridge_model = Ridge(alpha=1.0, solver='cholesky', tol=0.0005, random_state=42)
ridge_model.fit(X_train, y_train)
pred = ridge_model.predict(X_test)test_pred = ridge_model.predict(X_test)
train_pred = ridge_model.predict(X_train)print('Train set evaluation:')
print_evaluate(y_train, train_pred)print('Test set evaluation:')
print_evaluate(y_test, test_pred)Output —
Train set evaluation:
Mean Squared Error: 1467118179.7562702
Root Mean Squared Error: 38302.978732159594
Mean Absoulte Error: 23989.845101710034
R2 Square 0.7562349782638216
--------
Test set evaluation:
Mean Squared Error: 1388018226.771261
Root Mean Squared Error: 37256.11663567824
Mean Absoulte Error: 24348.388207169395
R2 Square: 0.8010891979496099
--------If you want to learn more about these Regression models, go to (References) : https://scikit-learn.org/stable/supervised_learning.html#supervised-learning
Day 38 : Coming soon!
Follow and Stay tuned. Keep coding :)
For other projects, tune to —
Build Machine Learning Pipelines( With Code)
Recurrent Neural Network with Keras
Clustering Geolocation Data in Python using DBSCAN and K-Means
Facial Expression Recognition using Keras
Hyperparameter Tuning with Keras Tuner
Custom Layers in Keras
That’s it fellas. Peace out and keep coding :)
Stay Tuned and of-course let me end this post with a quote by Steve Jobs ;)
“You have to be burning with an idea, or a problem, or a wrong that you want to right. If you’re not passionate enough from the start, you’ll never stick it out.”
