10 Ways of Iterating Over a List in Python

Can you loop without using the for keyword?

Python is a great programming language. It’s a simple yet powerful tool that can accomplish even complex tasks in just a few lines of code. In addition, Python is highly flexible, providing countless standard and non-standard ways of achieving the same goal.

In this article, I will show you ten ways of looping through a list in Python.

The Classic For-Each Loop

The first and most common method of looping through a list is by using Python’s for loop directly on the list:

The Classic For-Range Loop

This is another traditional looping method so banal that it doesn’t need presentation:

Non-Pythonic While Loop

Now we begin to look at some less common approaches, such as manually updating the list index in a while loop:

Still boring, right? It’s now time to warm up the party with the following method.

Hacking Built-In Functions

Hacking is the art of achieving a goal by non-standard means, for example, by making stuff (software, objects, etc.) behave in a way it was not intended to.

So, let’s hack the sorted() built-in function. It normally takes in an iterable and returns its sorted copy. However, you can pass an optional key to define your own sorting criteria. But what if you used this key function to perform operations on the list items?

Isn’t that also how code injection attacks such as SQL injection work? Indeed it is.

Manually Managing List Iterators

This is a more technical approach that involves iterators: objects that allow the traversing of a container, generally a list, with no knowledge of its internal structure.

This is also how the classic for-each loop works internally.

Treating the List as a Non-destructive Queue

None of these methods are particularly performant, but how about popping and subsequently reappending the list items for every iteration?

Note that even though we loop over the length range of the list, we don't ever use the subscript operator [].

Parse the String Representation of a List

This approach takes advantage of the built-in string representation of lists. Note, however, that this algorithm only works for primitive types, as complex objects are represented as their type name and memory address, which cannot be easily converted to a Python object without implementing a type-specific parser.

However, isn't that kinda cheating since we used list comprehension and a for-each loop? If you can’t stand for loops, don’t worry, the next alternative has no fors in it.

Iterate Using Recursion

Using a more functional approach, we can recursively loop through a list and apply a predicate to its elements:

And who said that loops and recursion cannot go well together?

Parse the Bytes Representation of the List

Even though raw bytes and Python don’t usually work nicely together, you can still convert the list to a byte array and then reconstruct the items from the raw bytes. Note that also this method doesn’t work for complex objects because we cannot reconstruct an object from its raw bytes.

That was pretty inelegant, though. Let’s see if we can do better with the following method. How about iterating over a list in Python without using Python?

Handle the Iteration in C

For this approach, we will write a C extension that takes care of iterating over the Python list. To keep the article concise, I’m not going to explain in detail the full process of creating a C extension for Python. If you’re interested in learning more about this topic, check out this guide.

First of all, let’s create the C code. If you don’t understand C, you might just read the comments.

Then you create the setup.py script to build the extension:

And finally, you build and install the C extension:

python3 setup.py build
python3 setup.py install --user

Now you can easily import the new C module in your Python script as it was a normal library. Here’s the code:

Indeed you can iterate over a list in Python without using Python. However, you should note that, since C is a statically typed programming language, you cannot pass any argument type to functions written in it.

In case you wanted to implement a generic c_iter() function that can take any list type, you would need to handle each specific type differently in the C file or simply define a different C function for every data type (c_iter_int(), c_iter_float(), c_iter_str(),etc.).

Conclusion

There are countless ways of looping through a list in Python. Some make sense and are considered best practices, such as using a for-each loop (Python loops compared). Others are considered proof-of-concept hacks and should not be used in any production codebase.

I hope you enjoyed this article. If you know any other interesting way of looping through a list in Python, please share it in a comment. Thanks for reading!

This was the second article of a series where I explore various interesting and over-engineered approaches to achieve simple tasks in Python. In case you missed it, you can find it here below: