Free AI web copilot to create summaries, insights and extended knowledge, download it at here

3926

Abstract

egorical variables. If the number of categories are few compared to the total number values, it is better to use the category data type instead of object. It saves a great amount of memory depending on the data size.The following code will go over columns with object data type. If the number of categories are less than 5 percent of the total number of values, the data type of the column will be changed to category.<div id="0b82"><pre>cols = marketing.select_dtypes(include='object').columns for col in cols: ratio = len(marketing[col].value_counts()) / len(marketing) if ratio < 0.05: marketing[col] = marketing[col].astype('category')</pre></div>We have done three steps of data cleaning and manipulation. Depending on the task, the number of steps might be more.Let’s create a pipe that accomplish all these tasks.The pipe function takes functions as inputs. These functions need to take a dataframe as input and return a dataframe. Thus, we need to define functions for each task.<div id="4e5c"><pre>def drop_missing(df): thresh = len(df) * 0.6 df.dropna(axis=1, thresh=thresh, inplace=True) return df </pre></div><div id="3a6e"><pre>def remove_outliers(df, column_name): low = np.quantile(df[column_name], 0.05) high = np.quantile(df[column_name], 0.95) return df[df[column_name].between(low, high, inclusive=True)]</pre></div><div id="dcd6"><pre>def to_category(df): cols = df.select_dtypes(include='object').columns for col in cols: ratio = len(df[col].value_counts()) / len(df) if ratio < 0.05: df[col] = df[col].astype('category') return df</pre></div>You may argue that what the point is if we need to define functions. It does not seem like simplifying the workflow. You are right for one particular task but we need to think more generally. Consider you are doing the same operations many times. In such case, creating a pipe makes the process easier and also provides cleaner code.We have mentioned that the pipe function takes a function as input. If the function we pass to the pipe function has any arguments, we can pass it to the pipe function along with the function. It makes the pipe function even more efficient.For instance, the remove_outliers function takes a column name as argument. The function removes the outliers in that column.We can now create our pipe.</

Options

p><div id="df66"><pre>marketing_cleaned = (marketing. pipe(drop_missing). pipe(remove_outliers, 'Salary'). pipe(to_category))</pre></div>It looks neat and clean. We can add as many steps as needed. The only criterion is that the functions in the pipe should take a dataframe as argument and return a dataframe. Just like with the remove_outliers function, we can pass the arguments of the functions to the pipe function as an argument. This flexibility makes the pipes more useful.One important thing to mention is that the pipe function modifies the original dataframe. We should avoid changing the original dataset if possible.To overcome this issue, we can use a copy of the original dataframe in the pipe. Furthermore, we can add a step that makes a copy of the dataframe in the beginning of the pipe.<div id="95d3"><pre>def copy_df(df): return df.copy()</pre></div><div id="f711"><pre>marketing_cleaned = (marketing. pipe(copy_df). pipe(drop_missing). pipe(remove_outliers, 'Salary'). pipe(to_category))</pre></div>Our pipeline is complete now. Let’s compare the original dataframe with the cleaned to confirm it is working.<div id="b937"><pre>marketing.shape (1000,10)</pre></div><div id="e53b"><pre>marketing.dtypes Age object Gender object OwnHome object Married object Location object Salary int64 Children int64 History object Catalogs int64 AmountSpent int64 </pre></div><div id="3641"><pre>marketing_cleaned.dtypes (900,10)</pre></div><div id="e065"><pre>marketing_cleaned.dtypes Age category Gender category OwnHome category Married category Location category Salary int64 Children int64 History category Catalogs int64 AmountSpent int64</pre></div>The pipeline is working as expected.<h2 id="8b85">Conclusion</h2>The pipes provide cleaner and more maintainable syntax for data analysis. Another advantage is that they automatize the steps of data cleaning and manipulation.If you are doing the same operations over and over, you should definitely consider creating a pipeline.Thank you for reading. Please let me know if you have any feedback.</article></body>

Asynchronously loaded Javascript — how to do it with promises.

Javascript: How to execute code from an asynchronously loaded script although when it is not loaded completely (Promises)

Sometimes you have to fire an action or execute code before a script is completely loaded in the browser (for example an asynchronously loaded external script). To prevent that these actions resulting in an error or get completely lost, you can use a promise which resolves, when the script is loaded. You can then fire the actions inside the promise and be sure that everything get executed in the correct order when the script is loaded.

Concept Code (copy paste to console)

var x = new Promise(function (resolve, reject) {
    setTimeout(resolve, 3000);
});
x.then(function () {
    console.log('1');
});
x.then(function () {
    console.log('2');
});
x.then(function () {
    console.log('3');
});

This should fire: “1,2,3” after 3 seconds when the promise is resolved. That’s the basic concept. Now we bring this to a real world scenario.

Real world example: loadScriptAsync()

We instantiate a new promise and create inside the promise a new script tag. We bind our promise resolver to the onload event, which is fired when the script is completely loaded and then resolve the promise.

The promise is saved in the “scriptLoaded” variable.

<script>
var loadScriptAsync = function(uri){
  return new Promise((resolve, reject) => {
    var tag = document.createElement('script');
    tag.src = uri;
    tag.async = true;
    tag.onload = () => {
      resolve();
    };
  var firstScriptTag = document.getElementsByTagName('script')[0];
  firstScriptTag.parentNode.insertBefore(tag, firstScriptTag);
});
}
var scriptLoaded = loadScriptAsync('external-script.js');
</script>

Firing the event via the scriptLoaded promise

Now every time when you want to use methods from the external script you can use the promise like this:

<script>
scriptLoaded.then(function(){
  window.extvar.execute('test');
});
</script>

Copy & paste script

Try it! Create an index.html and copy this code.

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Test</title>

<script>
var loadScriptAsync = function(uri){
  return new Promise((resolve, reject) => {
    var tag = document.createElement('script');
    tag.src = uri;
    tag.async = true;
    tag.onload = () => {
      resolve();
    };
  var firstScriptTag = document.getElementsByTagName('script')[0];
  firstScriptTag.parentNode.insertBefore(tag, firstScriptTag);
});
}
var scriptLoaded = loadScriptAsync('external-script.js');
</script>

<script>
scriptLoaded.then(function(){
  window.extvar.execute('test');
});
</script>

</head>
<body>
</body>
</html>

Example external script: external-script.js

Use this as an external script, if you don’t have already one.

window.extvar = {};
window.extvar.execute = function(a){ console.log('external event: ' + a); }
console.log('external script loaded');

Of course that script does not handle errors or reject the promise when the script is not loaded etc... I just want to make the concept clear.

New at Promises?

Have a look in this nice introduction:

More about the basic principles behind promises you can find here:

https://github.com/mattdesl/promise-cookbook#the-problem

Happy async coding!