Lazy Loading in Python: What It Is and How to Use It

Become a better developer by understanding one of the trickiest and most common programming techniques, also learn how to avoid the lazy loading bugs

In computer programming, lazy loading is a technique that allows you to defer the loading of an object until it’s actually needed. This can be useful in situations where loading an object takes a long time or uses a lot of memory, and you want to avoid doing it until it’s absolutely necessary.

Python supports lazy loading through a number of mechanisms, including generators, iterators, and lazy evaluation. In this post, we’ll take a closer look at each of these techniques and see how they can be used to implement lazy loading in Python.

Generators

In Python, a generator is a special type of iterator that allows you to generate a sequence of values on the fly. Instead of computing all the values upfront and storing them in memory, a generator computes each value as it’s needed.

Generators are defined using the yield keyword. When you call a generator function, it doesn't actually execute the function right away. Instead, it returns a generator object that you can use to iterate over the values.

Here’s an example of a generator function that generates the first n Fibonacci numbers:

def fibonacci(n):
    a, b = 0, 1
    for i in range(n):
        yield b
        a, b = b, a + b

When you call the fibonacci function with an argument of 10, it returns a generator object:

>>> fib = fibonacci(10)
>>> fib
<generator object fibonacci at 0x7fba57c10660>

You can then use a for loop to iterate over the values generated by the generator:

>>> for x in fib:
...     print(x)
...
1
1
2
3
5
8
13
21
34
55

The values are generated on the fly as you iterate over the generator object. This means that the values are not computed until they’re actually needed.

Iterators

In Python, an iterator is an object that generates a sequence of values. Like generators, iterators allow you to generate the values on the fly instead of computing them all upfront.

To define an iterator in Python, you need to implement two methods: __iter__ and __next__. The __iter__ method should return the iterator object itself, while the __next__ method should return the next value in the sequence.

Here’s an example of an iterator that generates the first n even numbers:

class EvenNumbers:
    def __init__(self, n):
        self.n = n
        self.current = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.n:
            raise StopIteration
        result = self.current * 2
        self.current += 1
        return result

When you create an instance of the EvenNumbers class and iterate over it, it generates the first n even numbers:

>>> evens = EvenNumbers(5)
>>> for x in evens:
...     print(x)
...
0
2
4
6
8

The values are generated on the fly as you iterate over the iterator object. This means that the values are not computed until they’re actually needed.

Lazy Evaluation

Lazy evaluation is a technique that allows you to defer the evaluation of an expression until it’s actually needed. In Python, lazy evaluation is implemented using the lambda keyword and the map and filter functions.

Here’s an example of lazy evaluation using the map

squares = map(lambda x: x * x, [1, 2, 3, 4, 5])

When you print the squares object, you'll see that it's a map object:

>>> print(squares)
<map object at 0x7fba57c2c748>

The map function doesn't actually compute the squares of the numbers upfront. Instead, it returns a map object that you can use to generate the squares on the fly.

To generate the squares, you can iterate over the map object:

>>> for x in squares:
...     print(x)
...
1
4
9
16
25

The squares are generated on the fly as you iterate over the map object. This means that the squares are not computed until they're actually needed.

The filter function works in a similar way. It allows you to generate a sequence of values that satisfy a certain condition. Here's an example of lazy evaluation using the filter function:

evens = filter(lambda x: x % 2 == 0, [1, 2, 3, 4, 5])

When you print the evens object, you'll see that it's a filter object:

>>> print(evens)
<filter object at 0x7fba57c2c748>

The filter function doesn't actually compute the even numbers upfront. Instead, it returns a filter object that you can use to generate the even numbers on the fly.

To generate the even numbers, you can iterate over the filter object:

>>> for x in evens:
...     print(x)
...
2
4

The even numbers are generated on the fly as you iterate over the filter object. This means that the even numbers are not computed until they're actually needed.

Lazy Loading in Practice

Lazy loading can be a powerful technique for optimizing your Python code. By deferring the loading of objects until they’re actually needed, you can reduce memory usage and improve performance.

For example, suppose you’re working with a large dataset that’s too big to fit into memory. You could use lazy loading to read the data from the disk on an as-needed basis. This would allow you to work with the data without having to load it all into memory at once.

Another use case for lazy loading is with web applications. If you have a web page that includes a lot of images, you could use lazy loading to defer the loading of the images until the user actually scrolls down to see them. This would improve the performance of your web page by reducing the amount of data that needs to be loaded upfront.

Conclusion

Lazy loading is a technique that allows you to defer the loading of objects until they’re actually needed. Python supports lazy loading through a number of mechanisms, including generators, iterators, and lazy evaluation. By using lazy loading in your Python code, you can reduce memory usage and improve performance.