Databricks List Files from a Path — DBUTILS VS FS
Databricks has at least four ways to interact with the file system, namely the following.
- DBUTILS — Databricks Package
- FS — Magic Command
- OS — Python Libraray
- SH — Magic Command
OS and SH are primary for the operating systems files and dbfs files.
In This Article, we look at all examples to list the file from Databricks data sets.
Databricks has plenty of Datasets for learning and practice in /databricks-datasets/ path. The link to the article explains how to access and what all data sets are available.
DBUTILS
To know more about the DBUTILS, Follow the link:
DBUTILS to list files from databricks data sets
dbutils.fs.ls("/databricks-datasets/")Output:
This output is not in a user-readable format. We can also use the display command to render in a table format. Learn more about the Display
display(dbutils.fs.ls("/databricks-datasets/"))
FS — Magic Command
Let's straight jump into the snippet to use FS magic command. To know more about the magic commands, look at the article.
%fs
ls /databricks-datasets/output:
Please look at the short articles-list to learn about Databricks and Spark.
We hope that you will find this article insightful. Please share the link with your friends, family, and colleagues.
Do you like to encourage us to spread the insights on more topics? Please enable us with a cup of coffee.
It just takes a moment to follow us. Let's help each other to spread the knowledge. Follow us on Medium, Insights and Data, LinkedIn, and Twitter to stay up-to-date with our latest articles.
It costs just 16 Cents per Day to become a referred member in Medium through us (Subscribe), unlocks the full potential of reading our articles, and may more in Medium.
