Free AI web copilot to create summaries, insights and extended knowledge, download it at here

2611

Abstract

n-linear relationship between dependent and independent variables, such as:</p><figure id="c7b3"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*P6ThSiui592I-Ql0vAA_iA.png"><figcaption></figcaption></figure><figure id="e3b5"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*90lxAW1T3wb1HmI7NuMAmw.png"><figcaption></figcaption></figure><h1 id="fb9b">2. Classification Models</h1><p id="32a8">Classification models are used to predict a <b><i>discrete</i></b> output (or dependent) variable from input (or independent) variables (unlike regression models that predict a <b><i>continuous</i></b> dependent variable). We can think of each of the discrete values of the dependent variable as “classes” and hence the naming of these models. One example is the machine learning algorithm that decides whether an email is spam or not.</p><p id="84ea">Now that we understand what a classification model is, let’s look at some of the most common classification models:</p><h2 id="7ce7">2.1. Logistic Regression</h2><p id="27d4">Logistic Regression, despite its name, is a classification algorithm where the dependent variable can only take two classes (e.g. yes vs. no or 0 vs. 1). Classification problems with only two possible outcomes for the dependent variables are called binary classifications.</p><p id="2b35">Logistic regression relies on the logistic function, which is a sigmoid curve with the following equation:</p><figure id="c656"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*j0-qUWrEZz1kRg1mvaROXQ.png"><figcaption></figcaption></figure><figure id="060c"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*CKDSnLiP0zdoJUeLMP-xjw.png"><figcaption></figcaption></figure><p id="a0d0">With some simplifying assumptions, logistic regression plot is as follows:</p><figure id="c15c"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*LcpG7owU81klhpkUHVI0Xw.png"><figcaption></figcaption></figure><p id="a0ae">Assumtions:</p><figure id="30c3"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*Hd4NUNz1zDcJByFwoo8aNw.png"><figcaption></figcaption></figure><p id="2bac">A logistic regression model calculates the probability based on the formula above, which will always end up between 0 and 1. Then when the probability is between 0 and 0.5, assigns it to one class, such as <code>0</code> or <code>not-spam</code> and then when the probability is between 0.5 and 1, assigns it to the other outcome, such as <code>1</code> or <code>spam</code>.</p><h2 id="d9d2">2.2. Naive Bayes</h2><p id="315f">Naive Bayes calc

Options

ulates the probability of a data point belonging to a certain class. It is called Naive because it assumes the occurrence of a certain feature is independent of the occurrence of other features. And it is called Bayes since it relies on Bayes’ Theorem, described below. For the purposes of this post, you can just remember that Naive Bayes is a type of classification model.</p><figure id="5af8"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*iag3dfrvxcSVl3YFqcPyOw.png"><figcaption></figcaption></figure><figure id="9e64"><img src="https://cdn-images-1.readmedium.com/v2/resize:fit:800/1*tdtxbdG4uQ7efd3Zy1u1ig.png"><figcaption></figcaption></figure><h1 id="ea95">3. Regression vs. Classification</h1><p id="0258">Most important difference between regression and classification models is that the former is used to predict continuous outcomes, while the latter is used to predict discrete outcomes.</p><blockquote id="f9ad"><p><b>Pro Tip:</b> It is important to keep in mind that sometimes the goal is not to just pick one of the regression or classification models and rather to understand what purpose each one serves and then take advantage of the differences between regression and classification models. In many machine learning pipelines, we end up using both. For example, if I call a customer service number, they may first use a classification model (based on my location, time of the day, etc.) to route my incoming call to a specific queue and then use a separate regression model to predict the wait time and inform me accordingly. In other words, the pipeline took advantage of the strengths of both approaches — the outcome of the classification model was used as an upstream input to the regression model to improve the overall performance of the pipeline, which was predicting the customer wait time.</p></blockquote><h1 id="dafe">Notebook</h1><p id="4dfb">I used a notebook to create some of the formula and plots in this post. In case you are curious to see how this is done, I’ve included it below for reference.</p> <figure id="4df7"> <div> <div>

            <iframe class="gist-iframe" src="/gist/fmnobar/2909e9164dce7b352a5a3320cd3793e6.js" allowfullscreen="" frameborder="0" height="undefined" width="undefined">
          </div>
        </div>
    </figure></iframe></div></div></figure><h1 id="9990">Thanks for Reading!</h1><p id="4292">If you found this post helpful, please <a href="https://medium.com/@fmnobar">follow me on Medium</a> and subscribe to receive my latest posts!</p></article></body>

Classification vs. Regression in Machine Learning — Which One Should I Use?

Overview

Model selection in Machine Learning exercises is one of the foundational questions that we will need to find an answer for as a Data Scientist. In this post we continue our journey of learning about Data Science Role Requirements by learning more about model selection and specifically the differences between a regression and classification model.

We will first go over an introduction of each of the regression and classification models and then will do a comparison between the two.

Let’s get started!

Example of a Logistic vs. Linear Regression (Source: scikit-learn.org)

1. Regression Models

Regression models are used to predict a continuous output (or dependent) variable from input (or independent) variables. Some examples of such continuous output variables are quantities such as height, weight, salary, probability, etc.

Now that we understand what a regression model is, let’s look at some of the most common regression models:

1.1. Simple Linear Regression

This is when one independent variable (x) is used to predict a dependent variable (y). Equation of a line is one such example:

1.2. Multiple (or Multivariable) Linear Regression

A more generalized linear regression where more than one independent variables are used to predict a dependent variable, such as:

1.3. Polynomial Regression

A regression describing a non-linear relationship between dependent and independent variables, such as:

2. Classification Models

Classification models are used to predict a discrete output (or dependent) variable from input (or independent) variables (unlike regression models that predict a continuous dependent variable). We can think of each of the discrete values of the dependent variable as “classes” and hence the naming of these models. One example is the machine learning algorithm that decides whether an email is spam or not.

Now that we understand what a classification model is, let’s look at some of the most common classification models:

2.1. Logistic Regression

Logistic Regression, despite its name, is a classification algorithm where the dependent variable can only take two classes (e.g. yes vs. no or 0 vs. 1). Classification problems with only two possible outcomes for the dependent variables are called binary classifications.

Logistic regression relies on the logistic function, which is a sigmoid curve with the following equation:

With some simplifying assumptions, logistic regression plot is as follows:

Assumtions:

A logistic regression model calculates the probability based on the formula above, which will always end up between 0 and 1. Then when the probability is between 0 and 0.5, assigns it to one class, such as 0 or not-spam and then when the probability is between 0.5 and 1, assigns it to the other outcome, such as 1 or spam.

2.2. Naive Bayes

Naive Bayes calculates the probability of a data point belonging to a certain class. It is called Naive because it assumes the occurrence of a certain feature is independent of the occurrence of other features. And it is called Bayes since it relies on Bayes’ Theorem, described below. For the purposes of this post, you can just remember that Naive Bayes is a type of classification model.

3. Regression vs. Classification

Most important difference between regression and classification models is that the former is used to predict continuous outcomes, while the latter is used to predict discrete outcomes.

Pro Tip: It is important to keep in mind that sometimes the goal is not to just pick one of the regression or classification models and rather to understand what purpose each one serves and then take advantage of the differences between regression and classification models. In many machine learning pipelines, we end up using both. For example, if I call a customer service number, they may first use a classification model (based on my location, time of the day, etc.) to route my incoming call to a specific queue and then use a separate regression model to predict the wait time and inform me accordingly. In other words, the pipeline took advantage of the strengths of both approaches — the outcome of the classification model was used as an upstream input to the regression model to improve the overall performance of the pipeline, which was predicting the customer wait time.

Notebook

I used a notebook to create some of the formula and plots in this post. In case you are curious to see how this is done, I’ve included it below for reference.

Thanks for Reading!

If you found this post helpful, please follow me on Medium and subscribe to receive my latest posts!