avatarHardik Raval

Free AI web copilot to create summaries, insights and extended knowledge, download it at here

3926

Abstract

egorical variables. If the number of categories are few compared to the total number values, it is better to use the category data type instead of object. It saves a great amount of memory depending on the data size.</p><p id="4b45">The following code will go over columns with object data type. If the number of categories are less than 5 percent of the total number of values, the data type of the column will be changed to category.</p><div id="0b82"><pre>cols = marketing<span class="hljs-selector-class">.select_dtypes</span>(include=<span class="hljs-string">'object'</span>)<span class="hljs-selector-class">.columns</span> <span class="hljs-keyword">for</span> col <span class="hljs-keyword">in</span> cols: ratio = <span class="hljs-built_in">len</span>(marketing<span class="hljs-selector-attr">[col]</span><span class="hljs-selector-class">.value_counts</span>()) / <span class="hljs-built_in">len</span>(marketing) <span class="hljs-keyword">if</span> ratio < <span class="hljs-number">0.05</span>: marketing<span class="hljs-selector-attr">[col]</span> = marketing<span class="hljs-selector-attr">[col]</span><span class="hljs-selector-class">.astype</span>(<span class="hljs-string">'category'</span>)</pre></div><p id="56aa">We have done three steps of data cleaning and manipulation. Depending on the task, the number of steps might be more.</p><p id="7ed1">Let’s create a pipe that accomplish all these tasks.</p><p id="af89">The pipe function takes functions as inputs. These functions need to take a dataframe as input and return a dataframe. Thus, we need to define functions for each task.</p><div id="4e5c"><pre>def drop_missing(df): thresh = len(df) * 0.6 df.dropna(<span class="hljs-attribute">axis</span>=1, <span class="hljs-attribute">thresh</span>=thresh, <span class="hljs-attribute">inplace</span>=<span class="hljs-literal">True</span>) return df </pre></div><div id="3a6e"><pre>def remove_outliers(df, <span class="hljs-built_in">column_name</span>): low = np.quantile(df[<span class="hljs-built_in">column_name</span>], <span class="hljs-number">0.05</span>) high = np.quantile(df[<span class="hljs-built_in">column_name</span>], <span class="hljs-number">0.95</span>) <span class="hljs-keyword">return</span> df[df[<span class="hljs-built_in">column_name</span>].<span class="hljs-keyword">between</span>(low, high, inclusive=<span class="hljs-keyword">True</span>)]</pre></div><div id="dcd6"><pre>def to_category(<span class="hljs-built_in">df</span>): cols = df.select_dtypes(include=<span class="hljs-string">'object'</span>).columns <span class="hljs-keyword">for</span> col <span class="hljs-keyword">in</span> cols: ratio = len(<span class="hljs-built_in">df</span>[col].value_counts()) / len(<span class="hljs-built_in">df</span>) <span class="hljs-keyword">if</span> ratio < 0.05: <span class="hljs-built_in">df</span>[col] = <span class="hljs-built_in">df</span>[col].astype(<span class="hljs-string">'category'</span>) <span class="hljs-built_in">return</span> <span class="hljs-built_in">df</span></pre></div><p id="4e1f">You may argue that what the point is if we need to define functions. It does not seem like simplifying the workflow. You are right for one particular task but we need to think more generally. Consider you are doing the same operations many times. In such case, creating a pipe makes the process easier and also provides cleaner code.</p><p id="f187">We have mentioned that the pipe function takes a function as input. If the function we pass to the pipe function has any arguments, we can pass it to the pipe function along with the function. It makes the pipe function even more efficient.</p><p id="4d8c">For instance, the remove_outliers function takes a column name as argument. The function removes the outliers in that column.</p><p id="b1c0">We can now create our pipe.</

Options

p><div id="df66"><pre>marketing_cleaned = (<span class="hljs-name">marketing</span>. pipe(<span class="hljs-name">drop_missing</span>). pipe(<span class="hljs-name">remove_outliers</span>, 'Salary'). pipe(<span class="hljs-name">to_category</span>))</pre></div><p id="10f5">It looks neat and clean. We can add as many steps as needed. The only criterion is that the functions in the pipe should take a dataframe as argument and return a dataframe. Just like with the remove_outliers function, we can pass the arguments of the functions to the pipe function as an argument. This flexibility makes the pipes more useful.</p><p id="b9fe">One important thing to mention is that the pipe function modifies the original dataframe. We should avoid changing the original dataset if possible.</p><p id="2d9c">To overcome this issue, we can use a copy of the original dataframe in the pipe. Furthermore, we can add a step that makes a copy of the dataframe in the beginning of the pipe.</p><div id="95d3"><pre><span class="hljs-keyword">def</span> <span class="hljs-title function_">copy_df</span>(<span class="hljs-params">df</span>): <span class="hljs-keyword">return</span> df.copy()</pre></div><div id="f711"><pre>marketing_cleaned = (<span class="hljs-name">marketing</span>. pipe(<span class="hljs-name">copy_df</span>). pipe(<span class="hljs-name">drop_missing</span>). pipe(<span class="hljs-name">remove_outliers</span>, 'Salary'). pipe(<span class="hljs-name">to_category</span>))</pre></div><p id="0425">Our pipeline is complete now. Let’s compare the original dataframe with the cleaned to confirm it is working.</p><div id="b937"><pre>marketing.<span class="hljs-built_in">shape</span> (<span class="hljs-number">1000</span>,<span class="hljs-number">10</span>)</pre></div><div id="e53b"><pre>marketing.dtypes <span class="hljs-type">Age</span> <span class="hljs-keyword">object</span> <span class="hljs-type">Gender</span> <span class="hljs-keyword">object</span> <span class="hljs-type">OwnHome</span> <span class="hljs-keyword">object</span> <span class="hljs-type">Married</span> <span class="hljs-keyword">object</span> <span class="hljs-type">Location</span> <span class="hljs-keyword">object</span> <span class="hljs-type">Salary</span> <span class="hljs-type">int64</span> <span class="hljs-type">Children</span> <span class="hljs-type">int64</span> <span class="hljs-type">History</span> <span class="hljs-keyword">object</span> <span class="hljs-type">Catalogs</span> <span class="hljs-type">int64</span> <span class="hljs-type">AmountSpent</span> <span class="hljs-type">int64</span> </pre></div><div id="3641"><pre><span class="hljs-title">marketing_cleaned</span>.dtypes (<span class="hljs-number">900</span>,<span class="hljs-number">10</span>)</pre></div><div id="e065"><pre>marketing_cleaned.dtypes Age category Gender category OwnHome category Married category Location category Salary <span class="hljs-built_in">int64</span> Children <span class="hljs-built_in">int64</span> History category Catalogs <span class="hljs-built_in">int64</span> AmountSpent <span class="hljs-built_in">int64</span></pre></div><p id="9b63">The pipeline is working as expected.</p><h2 id="8b85">Conclusion</h2><p id="2c89">The pipes provide cleaner and more maintainable syntax for data analysis. Another advantage is that they automatize the steps of data cleaning and manipulation.</p><p id="a29e">If you are doing the same operations over and over, you should definitely consider creating a pipeline.</p><p id="ae6e">Thank you for reading. Please let me know if you have any feedback.</p></article></body>

How Being Like ‘Salt’ Can Forever Change Your Life

The crucial role of ‘salt’ in modern life

Photo by Tiard Schulz on Unsplash

Think about it,

When was the last time you praised the salt in a dish?

It’s not something that usually comes to our mind.

Yet the effect of salt is undeniable.

Salt may seem like a simple element.

However, it adds flavor without wanting to be recognized.

It remains hidden and blends seamlessly with other ingredients to enhance the overall flavor.

The right amount of salt can transform a bland dish into a culinary masterpiece.

I first stumbled across the concept of “be like salt” while reading Jay Shetty’s book “Think Like a Monk.”

Inspired by his teacher, Radhanath Swami, Jay encourages us to manifest the qualities of salt in our lives.

Life, like a well-cooked dish, requires balance.

The balance that can improve the taste of our existence.

It has the potential to change the way you see yourself.

Not just you but also the people around you.

You feel dissatisfied if you focus too much on one aspect and neglect the others.

Just like a dish with too much or too little salt can spoil your evening.

Jay Shetty’s teachings remind us that wisdom lies in the balance of our time.

He suggests that we pay due attention to our health, family, profession, and spiritual needs.

It’s about finding a happy balance.

Radhanath Swami takes the analogy to another level and emphasizes the quality of humility.

He expresses it beautifully,

Like salt, true yogis serve without attachment to recognition or praise. They find joy in the act of giving, appreciating others, and letting go of the need for constant acknowledgment.

He encourages us to be indispensable and yet unnoticed.

To be practically unrecognizable when you’re at your best.

I wondered,

Why does it matter?

Both Radhanath Swami and Jay Shetty advocate us to strive to be the “salt of the earth.”

In other words,

It’s about being indispensable, reliable, and quietly making a difference.

It’s all about staying humble, balanced, and salty.

Being humble makes us more likable and trustworthy.

1. Finding balance in life

Too much salt overwhelms the taste buds.

Too little leaves the dish inadequate.

Similarly, an imbalance in the focus on health, family, career, and spiritual needs can lead to dissatisfaction.

The wisdom lies in maintaining balance.

It ensures that no aspect of our lives is neglected.

As Jay Shetty wisely puts it,

“Wisdom is to balance our time.”

2. Serving selflessly

Salt adds flavor to a dish without seeking praise.

Radhanath Swami emphasizes the quality of humility.

In the same way, true yogis serve without attachment to recognition or praises.

This is a powerful lesson in selflessness and humility.

He encourages us to find joy in giving.

Appreciate others and let go of the ego.

In the same way, real satisfaction comes from serving without expecting applause.

Radhanath Swami beautifully expresses,

“In giving, we really do receive food that satisfies the heart.”

It’s a reminder to appreciate the simple joy of making a positive difference without needing constant recognition.

3. Giving up ego for a greater purpose

Salt remains hidden in a bowl.

It blends seamlessly with the other ingredients to enhance the overall flavor.

Similarly, Radhanath Swami asks us to give up our sense of separateness.

He asks us to work in harmony with others for the greater good.

The ego often stands in the way of cooperation.

Like salt, we can add the flavor of cooperation to the preparation of our lives.

Imagine a world in which people come together.

Put aside personal differences for the good of all.

This is a vision inspired by the selfless nature of salt.

It sacrifices its identity to contribute to a more palatable outcome.

As Radhanath Swami wisely puts it,

“Add the salt of balance, unselfishness, and cooperation to the preparation of your life to enhance its flavor.”

Consider the three key qualities:

Take responsibility,

Avoid self-praise,

And exercise moderation.

Taking responsibility

Be the salt who takes the blame.

In the world of sports, successful leaders often take responsibility for failures.

Just as the salt is the first to take the blame when a dish doesn’t turn out well.

Strong leaders admit their mistakes.

This sense of responsibility creates a culture of accountability.

It inspires others to do the same.

Leadership is not about pointing fingers.

A study by Northwestern University indicates an interesting aspect of human behavior.

People are more willing to take the blame for others when they have received a favor.

This corresponds to the humility of salt.

It sometimes admits mistakes, even if there are none.

Taking the blame, like salt, strengthens character and trust within a team or community.

In the words of Jacob Brown,

“Lead dogs take the most snow in their face.”

Avoiding self-praise

Be the silent high flyer.

Salt never brags about its presence in a dish.

Successful people often work hard in silence.

Modesty becomes a key factor in building relationships and trust.

As the proverb wisely advises,

“Work hard in silence. Let your success be your noise.”

Research by social psychologist Susan Speer shows the negative effects of direct bragging.

Boasting about one’s own achievements can lead to integrity and genuine care for others being called into question.

Instead, embrace the salty quality of quiet accomplishments.

Let your actions speak louder than words.

Exercising moderation

Do you remember your mother’s advice on moderation?

Salt works on the same principle.

Just as too much or too little salt can ruin a dish, extremes in life can lead to dissatisfaction.

The words of the Greek poet Hesiod echo this sentiment.

“Observe due measure; moderation is best in all things.”

It emphasizes the importance of moderation in all things.

The gray areas of life enable us to walk a balanced path.

Similar to the optimal amount of salt in a dish.

As the proverb wisely adds,

“Everything in moderation. Including moderation.”

Why does salt matter?

Of course, I wondered while reading the book.

Why should I strive to be like the salt?

The answer lies in the virtue of humility.

Psychologists Kibeom Lee and Michael C. Ashton underlines the importance of honesty and humility in various areas of life.

People with a higher H-factor (humility) are generally more successful in business, relationships, politics, and society.

The true connection comes when the ego is put aside.

Success is humbly acknowledged.

Final thoughts

Earn the wisdom to be like salt.

Take responsibility, avoid self-praise, and keep yourself in moderation.

Let your actions speak volumes and quietly contribute to the flavor of the world.

Finally, strive to be known as the “salt of the earth.”

“Indispensable, reliable, and quietly making a difference.”

Enjoyed the story? Don’t miss out on future storiesSubscribe Now!

Did the story resonate with you? Join my free email newsletter for bite-sized wisdom and insightful productivity tips.

Inspired or moved? Show your supportBuy me a coffee (in fact a tea) and make my day!

Life Lessons
Self-awareness
Mindfulness
Meditation
Spirituality
Recommended from ReadMedium