avatarEivind Kjosbakken

Summary

This article provides instructions on how to terminate a Python script after a specified duration, which is particularly useful for time-sensitive tasks like web scraping.

Abstract

The article discusses a method to automatically stop a Python script after it has been running for a predetermined amount of time. This technique is especially beneficial for tasks such as web scraping, where one might want to limit the duration of the script's execution. The author illustrates the use of Python's built-in time module to track the script's runtime and the optional tqdm package to provide a progress bar with additional details. The article includes code examples for both while and for loops, demonstrating how to integrate time checks to terminate the script. The author also advises on reducing the overhead caused by frequent time checks and provides a link to a real-world application of this method on their GitHub repository.

Opinions

  • The author recommends the tqdm package for its utility in displaying loop progress, considering it a valuable tool for scripts involving loops.
  • The author acknowledges the potential overhead introduced by time checks within loops and suggests optimizing this by reducing the frequency of these checks.
  • The article implies that the method described is efficient and practical, as evidenced by the author's own use case of scraping Wikipedia articles within a time constraint.
  • The author encourages readers to explore further related topics by providing links to other articles they have written, indicating a belief in the interconnectedness and relevance of these subjects to the Python programming community.

How to stop Python script after X seconds

This article will talk about how to stop after a script based on time. This could be useful when for example running scripts where you need to run it for a while, but you are not entirely sure how long a task will task. This could, for example, be perfect for web scraping, where you want to scrape many sites, but not let your computer run for too long. This is exactly the task I used it for, where I wanted to scrape as many Wikipedia articles as possible, but only wanted my computer to run for an hour.

First, you need some imports:

The time package is a basic package with Python and does not need any pip installation

import time

The tqdm package is not necessary for you to install to stop a script after X seconds, but it is a package I recommend when running loops, as it tells you some useful information

Install tqdm (write in terminal):

pip install tqdm

Import tqdm into your Python script

from tqdm import tqdm

Example of how tqdm works with a for loop that stops after 5 seconds:

tqdm is a very useful package. You can see a GIF of what it outputs to the terminal here. It shows a percentage progress bar, how many iterations are done, how many iterations in total, time passed, estimated time till finish, and number of iterations per second. You can see the script here stops after 5 seconds, with the code explained below.

Stop Python script after 5 seconds:

Now I will run a for loop, with a pass, which would (for my computer) take about 10 seconds to run. This will depend a lot on your computer, and how many applications are currently running on your computer, but the point is to have a loop that runs for some amount of time, but we stop the loop based on time before it has run its course.

#stop a Python script after 5 seconds
startTime = time.time()
timeToRun = 5
endTime = startTime + timeToRun


for i in tqdm(range(80000000)):
    if (time.time() >= endTime):
       print("breaking from time")
       break
    pass

To explain the code. First, we grab the current time with time.time(). This returns a large number, which is equal to the number of seconds that have passed since January 1. 1970. This is useful since we can now take this current time, and add X seconds to it, where X represents the number of seconds we want a script to run. In my code, X is equal to 5, as I want the script to stop after 5 seconds. I therefore set the “endTime” variable as the current time (“startTime”), and then add the 5 seconds I want my script to run. Then I have a check in my loop saying that if the current time is bigger than the “endTime”, stop the loop. My loop then stops after 5 seconds, instead of running its full course.

It should be noted that adding this if statement adds some overhead to the loop, which is not optimal. If your loop runs many times, you could perhaps not do this check every loop, as grabbing the value from time.time() takes some time (especially if you do it a lot of times).

In general how to stop a Python script in X seconds:

While loop:

startTime = time.time()
endTime = startTime + X
while <condition>:
  if (time.time() >= endTime):
    break
  … (the code you want in your loop)

For loop:

startTime = time.time()
endTime = startTime + X
for i in range(<number of times the loop should run>):
  if (time.time() >= endTime):
    break
  … (the code you want in your loop)

Note that X is in seconds, so if you want to run the script for an hour for example, you just put 3600 in there (60 seconds * 60 minutes), and so on, if you want to have it run for several hours/days…

And that is how simple it is to stop a Python script after X number of seconds. Remember if you do this inside a method that uses a loop, and you want to stop the whole method after X number of seconds, then use “return” instead of “break”

To see how I have used the concept explained in this article, in a real system, check out this code from my GitHub: ScrapingScriptFromGitHub.

If you want to check some other related articles I have written, please check out:

Python
Time
Useful Tips
Recommended from ReadMedium