fit() vs transform() vs fit_transform() in Python scikit-learn
What’s the difference between fit, transform and fit_transform methods in sklearn
scikit-learn (or commonly referred to as sklearn) is probably one of the most powerful and widely used Machine Learning libraries in Python. It comes with a comprehensive set of tools and ready-to-train models — from pre-processing utilities, to model training and model evaluation utilities.
Transformers are among the most fundamental object types in sklearn, which implement three specific methods namely fit(), transform()and fit_transform(). Essentially, they are conventions applied in scikit-learn and its API. In this article, we are going to explore how each of these work and when to use one over the other.
Note that in this article we are going to explore the aforementioned functions using specific examples, but the concepts explained here are applicable to most (if not all) transformers that implement these methods.
Subscribe to Data Pipeline, a newsletter dedicated to Data Engineering
Before explaining the intuition behind fit(), transform()and fit_transform(), it is important to first understand what a transformer is in scikit-learn API.
What are transformers in scikit-learn
In scikit-learn, a transformer is an estimator that implements the transform() and/or fit_transform() methods.
What does fit() do in transformers
In scikit-learn transformers, the fit() method is used to fit the transformer to the input data and perform the required computations to the specific transformer we apply.
As an example, let’s assume that we need to scale our features using the preprocessing transformer called MinMaxScaler. In the code below, we first create some sample data and we subsequently split it into training and testing sets. Then we instantiate a MinMaxScaler and fit the training data in order to compute the minimum and maximum to be used for later scaling.
It is very important to highlight the importance of splitting the data into test and train sets before applying any pre-processing step such as scaling. Test data points represent real-world data. Therefore, we must run fit() only over the training instances to avoid introducing future information to our model.
Now that we have successfully called fit() over the training instances, we can now access the fitted parameters, as shown below.
Note that every transformer has different parameters that you can access once the data is fitted. You can find which parameters you can access in the official documentation and specifically in the ‘Attributes’ section of the sklearn transformer you are working with. Typically, fitted parameters use an underscore _ as a suffix.
What does transform() do
Now the transform() method of sklearn transformers, will transform the input data into some transformed spaced. The output is usually an array or a sparse matrix with equal number of samples (n_samples) as the input data. The transformation will be performed based on the parameters that were computed during fit.
Building on top of the example of the previous section, we can now go ahead and transform the training and testing instances based on the data seen only in the training data.
Note that if you attempt to run transform() without first running fit() you will receive a
What does fit_transform() do
scikit-learn transformers also implement fit_transform() method which is responsible for fitting the transformer and then return the transformed training instances. It is equivalent to calling fit().transform(). The most basic reason why you should prefer fit_transform() over running fit() and transform() separately is efficiency. The base class fit_transform().
Once again, we must highlight that in inductive learning, where we typically split our data into training and testing sets we must not apply fit_transform() over the entire dataset, but instead only over the training data points. Subsequently, you can then run transform() over the testing instances, so that they are transformed based on the fitted parameters that were computed during fit_transform().
The example below illustrates how to run fit_transform() over the training instances and then apply the transformer over the testing instances as well.
And as we can see, the result is equivalent to running fit() on training instances and then transform() over both the training and testing data points:
For more details around feature scaling and data normalisation in general, as well as to how to avoid data leakage, you can refer to the Medium article below.
Conclusion
In this article, we discussed what is the purpose of the three most commonly implemented functions of sklearn transformers, namely fit(), transform()and fit_transform() . We explored what each does and what their differences are as well as in what use-cases you should use one over the other.
fit() method will perform the computations which are relevant in the context of the specific transformer we wish to apply to our data, while transform() will perform the required transformation over the testing instances, based on the learned parameters during fit. On the other hand, fit_transform() will perform both steps in a single command and might also be a little bit more optimised when compared to applying fit and transform steps individually. Remember that fit_transform() must only be applied over the training set otherwise data leakage will occur.
A very similar topic is probably the comparison between fit() , predict() and fit_predict() methods which are implemented by scikit-learn estimators that are used to train models and perform predictions on the specified data points. If you want to learn more about them you can read my Medium article below.
