Download Million Of Images At Once Using Programming Skills

A simple yet interesting mini-project to add to your resume

While surfing the web I often come across situations where I needed to download images from the internet.

Though it is very straightforward to download images by just right-clicking and then saving them from the menu.

But what if I needed to download more the one image, and importantly, all in bulk or all at once?

It is quite simple, to be honest, anyone can do it.

In this post, I will share with you how you can use Python to download an image from the internet.

I was very excited while doing this task in python, for the fact that how dynamic and amazing this process can get

Just because I am doing it in Python, I could do endlessly different things with downloading an Image from the internet.

So let us get into the exciting stuff.

Getting Everything Ready!

You might be eager to jump into the coding part but first, there are some Imports and Presets we need to perform to get going with our main objective.

1. Code editor
2. Downloads

Code Editor

PyCharm is the Python IDE I use, and it creates a virtual environment for me every time I start a new project.

If you’re testing a program for the first time, a virtual environment is ideal because it offers a lot of flexibility and is forgiving.

Note: In this program, we will focus on how to download images from the given list of links, that way multiple images can be taken care of.

So go over the internet and copy the links of images you wish to download using your very own Image downloading program.

Downloads

To continue with our project we need to download a few libraries. We will be downloading them using pip.

It is quite easy to do. You just need to type pip install requests in the terminal window and that library gets downloaded on your computer.

Note: Use the terminal window provided in pycharm to install the library in that environment only.

We need to download in total two libraries, as mentioned below:

1. requests
2. beautifulsoup4

But why do we need these libraries?

Requests

Making HTTP requests is a really complicated task, with lots of moving parts and things to be careful about.

The requests library is the most popular library for making HTTP requests in Python.

It hides the difficulties and complexities of making requests behind a beautiful, simple API, allowing you to concentrate on interacting with services and consuming data in your app.

Beautiful Soup

Beautiful Soup is a Python package for extracting data from HTML and XML files for web scraping needs.

It generates a tree from the page source code, which can be used to extract data in a more legible and hierarchical manner.

Download Multiple Images With a URL List

The URL for my eg is listed below, it is a copyright-free image from unsplash.com

https://images.unsplash.com/photo-1519060825752-c4832f2d400a?ixlib=rb1.2.1&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8&auto=format&fit=crop&w=1170&q=80

And there you go, this is the entire code.

I would highly recommend keeping reading to fully understand the code, as I am going to go through each line step by step. Enjoy!

1. This is all the imports we are doing for our program

import requests
import urllib.request
from bs4 import BeautifulSoup

2. List that holds the links for the images you want to download.

# A list is used so that multiple images can be downloaded with a loopUrls_img = [“URL for the file"] 

linksUrls_faulty stores all the links of images that could not be downloaded

# A list to store all the faulty 
linksUrls_faulty = []

3. This is a loop that goes through all the links provided in the list for downloading the images.

for image in Urls_img:

    count += 1
    
    #Naming the file using our counter variable
    name = f"Image #{count}.png"
    print(f"This is the image name: {name}")

A counter variable is used to keep track of the current Image number that is being downloaded.

This count variable is also used in the naming of the file. For eg if the count variable is equal to 3 here, then the image will be named Image #3

4. In this part we are sending HTTPS requests to each individual link using the request library that we downloaded.

req = requests.get(image, stream=True)

req=request.get() sends requests and stores the data that is received from the server

5. Check status code

# We need to check that the status code is 200 before doing anything else
    if req.status_code == 200:

If req.status_code==200 check for the status that is received back from the server.

If the code received is 200, that means everything went well while connecting to the server. Or else, some problem incurred during the connection.

You might be wondering what these 200 code is all about.

HTTP response status codes show whether or not a particular HTTP request was completed successfully.

In simple terms, it is a universal way for communicating the current status of the HTTPS request.

There are a few other codes as well, but we won’t be going in-depth on these, do say me if you all want me to write on that!

For eg the very popular 404 not found, status code you have been seeing, which simply means not found.

The 200 code, send a message OK!, which means we are good to go ahead.

Since we have checked the server status, we are good to send the request

# Now let's send a request to the image URL:
    req = requests.get(image, stream=True)

5. Write the file

# This command below will allow us to write the data to a file as binary
        with open(name, 'wb') as f:
            for data in req:
                f.write(data)

This is a standard way how to open a file in Python.

We use the inbuilt Python function open() If you want to read more about how to open files in Python, do check my previous post, dedicated to file manipulation in Python.

The write() the function writes the data that in the opened file, the data being written was downloaded using the .get() method previously.

6. Store the faulty links

else:
        # We will write all the faulty URLs back to the Urls_faulty    list:
        Urls_faulty.append(image)

In this line of code, we simply append the URL that could not be downloaded for some reason.

Conclusion

“Imagination is more important than knowledge. Knowledge is limited. Imagination encircles the world.”

— Albert Einstein

Coding can be simple and fun to do. It helps you put your creativity to use and helps you learn many things.

With that said, you can now download images in bulk through Python.

Thank you for reading till the end. If you guys want another tutorial like this, hit me up in the comments below!

I love hearing from you all and will continue to do my best for many to come. au revoir!