Summary

The provided content explains the concepts of iterables and iterators in Python, detailing their differences, usage, and the various data types and functions that support them.

Abstract

In Python, iterables are objects that can be iterated over using a for loop, while iterators are objects that implement the iterator protocol, which includes the __iter__() and __next__() methods. Iterators produce a stream of data, returning each element one at a time and raising a StopIteration exception when exhausted. Iterables can be converted into iterators using the iter() function, and iterators can be advanced using the next() function. The content also discusses how certain built-in functions and operators work with iterators, the limitations of iterators, and how to handle infinite iterators. Additionally, it covers the use of iterators with data types like strings, lists, tuples, ranges, dictionaries, and sets, as well as with file objects. The article concludes by summarizing the key differences between iterables and iterators and provides resources for further reading.

Opinions

The author emphasizes that iterators can be a source of infinite data streams, which can be particularly useful in certain programming contexts.
The author suggests that understanding the iterator protocol is crucial for Python developers to effectively work with iterators.
There is an opinion that the ability to convert iterators to lists, tuples, or dictionaries using constructors like list(), tuple(), and dict() is a powerful feature in Python.
The author points out that the max() and min() functions do not support infinite iterators, which is an important consideration when working with iterators.
The article conveys that the len() function is not supported by iterators, which could be seen as a limitation when dealing with iterators compared to iterables.
The author highlights the utility of the map(), filter(), zip(), and enumerate() functions, as well as the itertools module, for creating and manipulating iterators.
The author notes that the in and not in operators can be problematic with infinite iterators, as they may never return a result.
Sequence unpacking is presented as a feature that works well with iterators, provided that the number of variables matches the number of elements in the iterator.
The article expresses that iterators have inherent limitations, such as the inability to go backward, reset, or copy them, which developers should be aware of.
The author concludes with a reminder that iterators and iterables are fundamental concepts in Python, with iterators being a subset of iterables.

Iterable vs Iterator in Python

Let’s learn about iterables and iterators.

Iterator

In Python, an iterator is an object which implements the iterator protocol, which consist of the methods __iter__() and __next__() . — python docs

An iterator is an object representing a stream of data.
It returns the data one element at a time.
A Python iterator must support a method called __next__() that takes no arguments and always returns the next element of the stream.
If there are no more elements in the stream,__next__() must raise the StopIteration exception.
Iterators don’t have to be finite.It’s perfectly reasonable to write an iterator that produces an infinite stream of data.

Iterable

In Python,Iterable is anything you can loop over with a for loop.

An object is called an iterable if u can get an iterator out of it.

Calling iter() function on an iterable gives us an iterator.
Calling next() function on iterator gives us the next element.
If the iterator is exhausted(if it has no more elements), calling next() raises StopIteration exception.

Topics Covered in this story

How to convert an object into Iterator

Using iter() function.

The built-in iter() function takes an arbitrary object and tries to return an iterator that will return the object’s contents or elements, raising TypeError if the object doesn’t support iteration. — python docs

Iterables vs Iterators.

Both iterables and iterators can be iterated using for loop.

2. Iterables supports only iter() function.But iterators supports both iter() and next() function.

Iterable:

Iterator:

3. Iterators are also iterables.

We can get an iterator from an iterable by calling iter() function.Similarly, we can call iter() function on the iterator itself. It will return the iterator object itself.

Data types that support Iterators

Text Sequence type- str

Strings support Iterator. iter() on a string returns an iterator.

Sequences in Python: List, tuple, range supports Iterator.

1.Converting list(iterable) into iterator object.

2. Converting tuple(iterable) into iterator object.

3.Converting range() into iterator object.

Mapping: Dictionary supports Iterator.

iter()function will loop over dictionary keys only.

Dictionary has methods like 1.values()- which will iterate over values 2.items() -which will iterate over (key, value) pairs.

items()-loop over (key,value) pair in the dictionary.

File objects are implemented as iterators.

Files also support iteration by calling the readline() method until there are no more lines in the file. readline()-Read until newline or EOF and return a single str. If the stream is already at EOF, an empty string is returned.

colors.txt

red
green
blue

How to convert an iterator to list/tuple/dict

Iterators can be materialized as lists or tuples by using the list() ortuple() constructor functions.The dict() constructor can accept an iterator that returns a finite stream of (key, value) tuples.

Functions which returns an iterator

Functions that return iterators (Image Source: Author)

Generator Generators are a special class of functions that return an iterator that returns a stream of values. Any function containing a yield keyword is a generator function.

2. Generator expression Generator expression returns an iterator object only. Refer to my story for generator expressions.

Built-in functions that return an iterator.

map() Return an iterator that applies a function to every item of iterable, yielding the results. — python docs

Refer to my story of map() vs starmap()

2. filter() Returns an iterator from those elements of iterable for which function returns true. iterable may be either a sequence, a container that supports iteration, or an iterator. — python docs

Refer to my story of filter vs filterfalse

3. zip() zip(iter a, iter b ..) —Returns a zip object which is an iterator. It takes one element from each iterable and returns them in a tuple.

4. enumerate(): enumerate(iterable, start=0) Return an enumerate object which is an iterator. iterable must be a sequence, an iterator, or some other object which supports iteration. The __next__() method of the iterator returned by enumerate()returns a tuple containing a count (from the start which defaults to 0) and the values obtained from iterating over iterable. — python docs

5.itertools module:

The itertools module contains a number of commonly-used iterators as well as functions for combining several iterators.

Refer to my story for itertools module.

6.reversed()

reversed() function is used to reverse the elements in the sequences (list,tuple,range()).It is also used to reverse the key elements in the dictionary. reversed() function returns an iterator object.

The built-in functions which supports Iterator.

1.max() max() also supports a single iterator argument and returns the largest element.

2. min() min() also supports a single iterator argument and returns the smallest element. If iterator is infinite,max() and min() will never return.It supports only finite iterators.

Operators:

in, not in

The "in" and "not in" operators also support iterators. X in iterator is true if X is found in the stream returned by the iterator. If the iterator is infinite, and if the element X never appears in the stream, the "in" and "not in" operators will never return.

Sequence unpacking

Sequence unpacking also supports iterators.

Sequence unpacking requires the list of variables on the left to have the same number of elements in the iterator.

for loop

In the statement for X in Y, Y must be an iterator or some object(iterable) for which iter() can create an iterator. These two statements are equivalent:

for i in iter(obj):
    print(i)

for i in obj:
    print(i)

Limits of Iterator

We can only go forward in an iterator.
We can’t make a copy of it.
No way to get the previous element.
We can’t reset the iterator.

The iterator protocol only specifies the __next__() method. Functions may therefore consume all of the iterator’s output, and if you need to do something different with the same stream, you’ll have to create a new iterator.

StopIteration

Raised by the built-in function next()and an iterator’s __next__() method to signal that there are no further items produced by the iterator.

Conclusion

Iterators can be infinite or finite.
max(),min() doesn’t support infinite iterators.
len() is not supported by iterators.
for loop can be used to iterate over iterable and also iterator.
Iterators: 1. We can iterate using for loop 2. We can use the next() method which will return the next element in the iterator. 3. We can convert the iterator to list using list() constructor, tuple using tuple () constructor. 4.The dict() constructor can accept an iterator that returns a finite stream of (key, value) tuples
Difference between iterables and iterators: 1. Iterators support iter() and next() function. Iterables support only iter() function. 2. Iterators are also iterables but not vice versa.

Resources(Python Documentation):

Watch this space for more articles on Python and DataScience. If you like to read more of my tutorials, follow me on Medium, LinkedIn, Twitter.

Thanks for reading!