Memory and Garbage Collection in Python

Memory allocation and deallocation

Introduction

Have you wondered what happens when you create an object in Python? Where does it get stored? Or what happens if more than two kinds of data get stored in the same variable? How does the memory handle these kinds of processes? This article will help you understand the backend processes of python memory.

In python programming, the concept of memory allocation and deallocation is an automatic method. Unlike C# and C++, Python users do not have to pre-allocate or deallocate memory using dynamic memory allocation.

Python follows two kinds of strategies:

1. Garbage Collection
2. Reference counting

Reference counting

Reference counting is nothing but deallocating objects when there’s no reference to them in a program. Let us have a better understanding using an example:

• Let us consider the value ‘10’ stored in ‘My_nem’.

First Example:

My_nem = 10

Here, python is creating an object of integer type having value as ‘10’, with some memory address(eg: 1000). And here, ‘My_nem’ is a pointer to the stated object.

This means that ‘My_nem’ is the reference to the memory address of that object. Hence the current reference count is 1.

Reference count = 1

Second Example:

My_nem = 10
Nem = My_nem

Here, we are not assigning ‘Nem’ to the value 10. Instead, we are taking the reference of ‘My_nem’ and assigning that to ‘Nem’.

In simple words, ‘Nem’ does not get stored as a separate object. It is pointing to the same object in the memory. Hence, sharing the memory address. And in terms of reference count, it now adds up to 2.

Reference count = 2

• Let us now consider another scenario, where ‘My_nem’ either falls out of scope or gets assigned to a different object in the memory, and the second line of code is still present:

What happens now is that the reference goes away and, the reference count goes back to being 1.

Reference count = 1

• In the final scenario, consider the second line of code also goes out of scope.

Now there is nothing left, and the reference count drops to 0. And at this point, the Python memory manager recognizes that there is no object left. And hence it sees the memory as garbage and throws away the object.

Reference count = 0

This is where the concept of garbage collection arises.

Garbage Collection

Garbage collection is nothing but releasing the memory when the object is no longer in use. The system removes the unused object from the memory slot and reuses it for new objects. It can be denoted as a computer’s automated recycling system.

Now, what if you don’t want the object to get automatically erased by Python’s memory manager? There is a way to save this object in the memory even though it’s empty. This concept is established by creating reference cycles.

Reference cycle:

A reference cycle is nothing but more than one object referencing each other. A reference cycle is created only when the object’s reference count is unable to reach a minimum value of ‘1’. One of the easiest ways to generate a reference cycle is by creating an object referring to itself. The program below will show you how it works.

Program:

1→ def ref_cycle():
2→     A = [ ]
3→     A.append(A)
4→ ref_cycle()

Explanation:

In this program, we are creating a function containing an empty list. The third line containing the append function is where the reference cycle gets generated. This means that the function ‘A’, contains a reference to itself, making the default reference count of the object as 1. Due to this function, the object ‘A’ will not be free once the function returns. Hence causing the memory, ‘A’ to be stored until the Python Garbage Collector is invoked.

Making an object eligible for Garbage collection

There is a way to collect the discarded objects using the Python program. The below program shows a way to make an object eligible for garbage collection.

Program:

A = [ ]
A.append(1)
A.append(2)

del A

Explanation:

From the above program, an empty list ‘A’ is created and, the reference count is 2 due to the two append functions. The list ‘A’ gets deleted later; making it unable to use it again. Hence, the object becomes garbage. But the list can never be freed from the memory because of the reference cycles generated; making the object eligible for garbage collection.

Automatic Garbage Collection

Since reference cycles take a lot of computational processes to discover, it is much better when the garbage collector gets scheduled. Python schedules this collection based on the object’s allocation and deallocation thresholds. In other words, the garbage collector can be automated only if:

No. of allocations — No. of deallocations > threshold no

Also, to find the threshold for new objects, all you have to do is, import the gc module and request the thresholds.

Program:

import gc
print(“Python GC thresholds:”, gc.get_threshold())

#Sample Output:
Python GC thresholds: (800, 20, 20)

Explanation:

From the output obtained from the above program, the default threshold is 800. This means that the number of allocations minus the number of deallocations is greater than 800 thereby, successfully running the garbage collector.

Manual Garbage Collection

Calling the garbage collector during the program’s execution manually; is an excellent idea to understand how the memory is handled and consumed by reference cycles.

Program:

import gc
c = gc.collect()

print(“Garbage collector: c — “, “%d objects” % c)

#Output:
Garbage collector: c — 0 objects

Explanation:

In the above program, we have imported the gc module to collect the number of objects in the garbage collector and print it as the output.

Conclusion

To conclude, in Python, we never deal with memory addresses directly. This topic is just for programmers to understand the basic backend of Python of how it handles memory and storage. I strongly recommend referring to more articles and applying these concepts using python programs you can experiment with on this topic.

I hope you like the article. Reach me on my LinkedIn and twitter.

Recommended Articles

1. 8 Active Learning Insights of Python Collection Module 2. NumPy: Linear Algebra on Images 3. Exception Handling Concepts in Python 4. Pandas: Dealing with Categorical Data 5. Hyper-parameters: RandomSeachCV and GridSearchCV in Machine Learning 6. Fully Explained Linear Regression with Python 7. Fully Explained Logistic Regression with Python 8. Data Distribution using Numpy with Python 9. Decision Trees vs. Random Forests in Machine Learning 10. Standardization in Data Preprocessing with Python