How to Flatten Deeply Nested JSON Objects in Non-Recursive Elegant Python

It is dangerous to flatten deeply nested JSON objects with a recursive python solution. Because the python interpreter limits the depth of stack to avoid infinite recursions which could result in stack overflows. And from performance standpoint, recursion is usually slower than an iterative solution.

The purpose of this article is to share an iterative approach for flattening deeply nested JSON objects with python source code and examples provided, which is similar to bring all nested matryoshka dolls outside for some fresh air iteratively.

Traditional recursive python solution for flattening JSON

The following function is an example of flattening JSON recursively. Code at line 16 and 20 calls function “flatten” to keep unpacking items in JSON object until all values are atomic elements (no dictionary or list).

In the following example, “pets” is 2-level nested. The value for key “dolphin” is a list of dictionary.

Loading the flattened results to a pandas data frame, we can get

Using an iterative approach to flatten deeply nested JSON

The function “flatten_json_iterative_solution” solved the nested JSON problem with an iterative approach. The idea is that we scan each element in the JSON file and unpack just one level if the element is nested. We keep iterating until all values are atomic elements (no dictionary or list).

• chain.from_iterable() at line 27 is used to get chained inputs from a single iterable argument, i.e. chain.from_iterable([‘ABC’, ‘DEF’]) → A B C D E F
• starmap() at line 27 is used to make an iterator that computes the function “unpack” using arguments obtained from dictionary.items()

Using the new iterative solution “flatten_json_iterative_solution” for example “pets”:

Unit test with another example “pets2” which is 3-level nested at key “dolphin”. A list [{“bird”:”bluejay”},{“fish”:”dolphin”}] is packed as the value of key “tiger”.

In this article, I describe an iterative solution for flattening deeply nested JSON object. Feel free to use the source code in practice. Please send me a message if you have any idea of optimizing the iterative solution.

Congratulations! You just finished reading an article and emancipated some matryoshka dolls at the meaning time.

Sign up for Udemy course 🦞:

Recommender System With Machine Learning and Statistics