How to query collections in Firestore under a certain path?
A simple solution to get content across different sub-collections
Cloud Firestore is a flexible, scalable database for mobile, web, and server development from Firebase and Google Cloud. It consists of collections and documents. When working with such a database, there often are cases in which we would like to query multiple collections at once. But unfortunately, this is not possible. And it makes sense since the queries in Firestore are shallow, meaning that can only get documents from the collection that the query is run against. There is no way to get documents from different collections or sub-collections in a single query. Firestore doesn’t support queries across different collections in one go unless we are using a collection group query.
But how about limiting the results only to sub-collections that exist under a particular document? So I’m going to explain in this article, the most simple and efficient way for solving this problem.
Let’s assume we have an application that displays landmarks from all over the world, from the smallest to the largest cities. So let’s take a simple example of a Firestore Database structure for such an app:
Firestore-root
|
--- countries (collection)
|
--- UK (document)
| |
| --- name: "United Kingdom"
| |
| --- largestCities (sub-collection)
| | |
| | --- LDN (document)
| | | |
| | | --- name: "London"
| | | |
| | | --- landmarks (sub-collection)
| | | |
| | | --- $landmarkId (document)
| | | | |
| | | | --- name: "Big Ben"
| | | |
| | | --- $landmarkId (document)
| | | |
| | | --- name: "Buckingham Palace"
| | |
| | --- MCR (document)
| | |
| | --- name: "Manchester"
| | |
| | --- landmarks (sub-collection)
| | |
| | --- $landmarkId (document)
| | |
| | --- name: "Manchester Library"
| |
| --- smallestCities (sub-collection)
| |
| --- CRD (document)
| |
| --- name: "Cardiff"
| |
| --- landmarks (sub-collection)
| |
| --- $landmarkId (document)
| |
| --- name: "Cardiff Castle"
|
--- US (document)
|
--- name: "United States"
|
--- largestCities (sub-collection)
|
--- NY (document)
|
--- name: "New York"
|
--- landmarks (sub-collection)
|
--- $landmarkId (document)
|
--- name: "Central Park"- Please ignore the abbreviations, as I only used them as examples. I really don’t know if it actually exists.
As you can see, there is a top-level collection called “countries”. Each Country document within this collection contains a sub-collection called largestCities and another one called smallestCities. Within each City document, there is another sub-collection called “landmarks” that contains Landmark objects. In short, it looks like this:
countries/$countryId/largestCities/$cityId/landmarks/$landmarkIdAnd:
countries/$countryId/smallestCities/$cityId/landmarks/$landmarkIdNow, the question that arises is:
How to get all landmarks of a particular country, no matter the size of the city?
Since Cloud Firestore version 19.0.0, when the collectionGroup(String collectionId) method was introduced, we are allowed to perform a query that will include all the documents in the database that are contained in a collection or subcollection with the given collectionId.
So assuming we have the following root reference:
val rootRef = FirebaseFirestore.getInstance()We can perform a collection group query like this:
And the result that we get:
countries/UK/largestCities/LDN/landmarks/QcTj6X5Upqa4gWfIX***
countries/UK/largestCities/LDN/landmarks/Qs6iR3BYSuOF7gTZo***
countries/UK/largestCities/MCR/landmarks/HXbzW9rwPgtMr7Ohw***
countries/UK/smallestCities/CRD/landmarks/Q3UmT9hTBnHgs7wAY***
countries/US/largestCities/NY/landmarks/L07Dzgv97kcYpBFQO***Meaning that we get all landmarks of all countries. Hold on, but this is not what we want. We need to get only the landmarks from a single country, for example, from the UK.
How can we do that?
A relatively simple solution would be to add the Country, as a property of the Landmark object like this:
Firestore-root
|
--- countries (collection)
|
--- UK (document)
| |
| --- name: "United Kingdom"
| |
| --- largestCities (sub-collection)
| | |
| | --- LDN (document)
| | | |
| | | --- name: "London"
| | | |
| | | --- landmarks (sub-collection)
| | | |
| | | --- $landmarkId (document)
| | | | |
| | | | --- name: "Big Ben"
| | | | |
| | | | --- country: "UK" //Newly added
| | | |
| | | --- $landmarkId (document)
| | | |
| | | --- name: "Buckingham Palace"
| | | | |
| | | | --- country: "UK" //Newly added
| | |
| | --- MCR (document)
| | |
| | --- name: "Manchester"
| | |
| | --- landmarks (sub-collection)
| | |
| | --- $landmarkId (document)
| | |
| | --- name: "Manchester Library"
| | |
| | --- country: "UK" //Newly added
| |
| --- smalltesCities (sub-collection)
| |
| --- CRD (document)
| |
| --- name: "Cardiff"
| |
| --- landmarks (sub-collection)
| |
| --- $landmarkId (document)
| |
| --- name: "Cardiff Castle"
| |
| --- country: "UK" //Newly added
|
--- US (document)
|
--- name: "United States"
|
--- largestCities (sub-collection)
|
--- NY (document)
|
--- name: "New York"
|
--- landmarks (sub-collection)
|
--- $landmarkId (document)
|
--- name: "Central Park"
|
--- country: "US" //Newly addedAnd perform the following query:
rootRef.collectionGroup("landmarks")
.whereEqualTo("country", "UK")
.get()
.addOnCompleteListener { /* ... */ }But is this the best solution? The simplest answer is, no! Why? Because of the following two main reasons:
- To perform such a query, an index is required. This can be done, by creating it manually in the Firebase Console, or you’ll find in your logs a message that sounds like this:
FAILED_PRECONDITION: The query requires an index. You can create it here: https://...You can simply click on that link, or copy and paste the URL into a web browser and the index will be created automatically. However, there is a downside. Currently, there is no way you can create such an index programmatically. Even if you create the required index directly in the Firebase Console, it won’t help, since the names of the countries are dynamically added, and manually creating an index for each country, doesn’t sound like an option. Besides that, it may also lead you to reach the maximum number of indexes very quickly.
Maximum number of composite indexes for a database: 200
After building the index, we get the following result, without having the US in the results:
countries/UK/largestCities/LDN/landmarks/QcTj6X5Upqa4gWfIX***
countries/UK/largestCities/LDN/landmarks/Qs6iR3BYSuOF7gTZo***
countries/UK/largestCities/MCR/landmarks/HXbzW9rwPgtMr7Ohw***
countries/UK/smallestCities/CRD/landmarks/Q3UmT9hTBnHgs7wAY***2. Since we know that there are some limitations when it comes to how much data we store in a single document, according to the official documentation regarding usage and limits:
Maximum size for a document: 1 MiB (1,048,576 bytes)
So adding more data into the document might also lead to reaching this limitation too.
So is this the single option we have?
No, it’s not! The key to solving this issue without the above limitations is to use a collection group query for finding all matching document paths that start with the given country’s document path like this:
val countriesRef = rootRef.collection("countries")
val ukDocRef = countriesRef.document("UK")
rootRef.collectionGroup("landmarks")
.orderBy(FieldPath.documentId())
.startAt(ukDocRef.path)
.endAt(ukDocRef.path + "\uf8ff")
.get()
.addOnCompleteListener {
it.apply {
if (isSuccessful) {
for (document in result) {
Log.d(TAG, document.reference.path)
}
} else {
Log.d(TAG, "Error: ", exception)
}
}
}The result of the above code is the same as before, which is all the landmarks that correspond only to the UK. Please also note that the \uf8ff is the last character in Unicode, so acts as an end guard.
Benefits:
- No index is required.
- Fewer data to be added in the document. This means the document will have a smaller size. If you are worried about the size of the document by chance, you can always check against the maximum 1 MiB quota using the FirestoreDocument-Android library.
However, there is a downside. Considering having the following “new” country:
--- USA (document) //Newly added
|
--- name: "United States of America"
|
--- largestCities (sub-collection)
|
--- SF (document)
|
--- name: "San Francisco"
|
--- landmarks (sub-collection)
|
--- $landmarkId (document)
|
--- name: "Golden Gate Bridge"The result of the following query:
val countriesRef = rootRef.collection("countries")
val usDocRef = countriesRef.document("US") //Changed from UK to US
rootRef.collectionGroup("landmarks")
.orderBy(FieldPath.documentId())
.startAt(usDocRef.path)
.endAt(usDocRef.path + "\uf8ff")
.get()
.addOnCompleteListener { /* ... */ }Will be:
countries/US/largestCities/NY/landmarks/L07Dzgv97kcYpBFQO***
countries/USA/largestCities/SF/landmarks/8naAYp5MrHwuJNBIq***And not only:
countries/US/largestCities/NY/landmarks/L07Dzgv97kcYpBFQO***As you probably expected. So be aware of this constraint. However, there is a solution that can help us get over this limitation, which would be to use only document IDs that are generated either by the CollectionReference’s add(Object data) method or by the DocumentReference’s set(Object data) method. In this way, you’ll always have totally different document IDs having the same length and the queries will work as usual without the chances of hotspots.
Conclusion
I tried to explain in this article the simplest way that currently exists for querying multiple collections in Firestore that exist under a certain path. So I hope you found this article useful and if you have any questions regarding this topic, feel free and leave a comment in the section below.
If you wanna support me, please join me!
You can see it on youtube:
Credit to Samuel Jones
#BetterTogether 🔥