Scale Python Applications Using Nginx

Scale Python Applications To Reduce Downtime, Redundancy, And Increase Reliability

First The Usecase

• Let’s consider we have a Python web service application running on a local machine on port 70.
• Also, consider that multiple client applications send a large number of requests, at times millions of concurrent requests, to the Python application.

• As a result, the application is under constant traffic load and it is slow to respond back to the client applications. The client applications start receiving timeout exceptions. And the Python application is considered not reliable.

How do we balance the load? This is where a load balancer comes in. A load balancer can route client requests across all servers whilst ensuring the workload of the applications is balanced.

1. Article Aim

This article aims to demonstrate an end-to-end example of how we can scale a Python application with the help of a load balancer.

1. I will present a Python web service application.
2. Then I will demonstrate how we can easily scale the Python application and use a load balancer to distribute client requests
3. This will improve network efficiency, ensure high availability of the applications, and the applications will be able to serve the requests quickly.
4. And the best part, the client code is not required to be amended. This means the clients can keep calling the same host and port as the load balancer will take care of the routing of the requests.

Result: A reliable, fast and trusted infrastructure with reduced downtime.

1. NGINX As A Load Balancer

I am going to use NGINX to balance the load.

• Nginx is one of the most useful tools to add to your software architecture.
• Essentially Nginx can work as a load balancer, which implies that it can handle the client requests and forward the requests in a smarter way so that the load is balanced.
• The client applications do not need to know that the Nginx is serving the traffic, how many application instances are running, and which ports they are running on.
• Additionally, NGINX can terminate connections, perform security checks, log traffic, and add caching among other features. NGINX can also handle failover scenarios.

And the best part: NGINX is extremely easy to set up

NGINX can also help architects design their applications in a smarter way.

NGINX supports various balancing algorithms, from the round robin to the least connections to the random choices algorithms.

2. Install NGINX Load Balancer If It’s Not Installed Already

This is a one-off setup exercise and we only need to do it if you do not have NGINX installed already.

1. Download NGINX from: http://nginx.org/en/docs/windows.html
2. Unzip the file. Open the unzipped folder.

3. Start NGINX by double clicking on the nginx.exe

4. Open the conf folder and open nginx.conf file in Notepad and enter:

worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    #to read external configuration.
    include "C:/nginx/conf/sites-enabled/*.conf";
}

3. End to End Example: Let’s Scale A Python Application

The first step is to create a Python application and launch multiple instances of the application on different ports.

1. Create a new python application and a virtual environment

2. Install the following python package

pip install tornado==6.2

3. Create a new file: fintechexplained_runner.py

Copy the following code snippet into the file.

import asyncio
import os
import sys

import tornado.web

class FinTechHandler(tornado.web.RequestHandler):
    def get(self):
        self.write(f"Hello from Process Id = {os.getpid()}")

def make_app():
    return tornado.web.Application([
        (r"/", FinTechHandler),
    ])

async def main():
    app = make_app()
    port = sys.argv[1]
    print(f'Running on {port}')
    app.listen(int(port))
    shutdown_event = asyncio.Event()
    await shutdown_event.wait()

if __name__ == "__main__":
    asyncio.run(main())

This is a simple application but it can demonstrate the use case and how a load balancer can help.

1. This code uses the tornado python package to start a web server on a port.
2. The port is a command line argument that we can pass in.
3. There is aget endpoint that serves the request and returns “Hello from Process Id =< ProcessId>” back to the client.

4. Open Python Terminal or Command Prompt

Navigate to the folder where the fintechexplained_runner.py file is and run the following commands

fintechexplained_runner.py 71

fintechexplained_runner.py 72

fintechexplained_runner.py 73

fintechexplained_runner.py 74

fintechexplained_runner.py 75

Now we have created 5 instances of the Python application and they are running on ports 71, 72, 73, 74, and 75.

In a production environment, I would test this by creating 5 docker containers to provide isolation.

5. Test The Application

• Open your browser. Enter the URL: http://localhost:71 and you will see:

• Now enter the URL: http://localhost:72 and it will print:

Note the process ID is different, implying that there is a separate Python instance running to serve the request.

6. Add NGINX As A Load Balancer in front of the applications

• Remember the Python application is running the ports: 71–75
• Create a directory: C:/nginx/conf/sites-available
• Create a configuration file named fintechexplained.conf within the sites-available directory and add the following configuration:

upstream fintechexplained {
 server localhost:71;
 server localhost:72;
 server localhost:73;
 server localhost:74;
 server localhost:75;
}

server {
  listen       70;
  server_name  localhost;
  location / {
  proxy_pass http://fintechexplained;
 }
}

This code is informing Nginx to direct the traffic to ports 71–75 to serve the requests that are sent to port 70. NGINX can balance the load using various algorithms. By default, it will use the Round Robin algorithm.

Best Practice: Create a config file per service. Add the configuration files in the site-available folder. And then add shortcuts to the configuration files that are required to be load-balanced in the site-enabled folder.

We are done

And that’s how simple it is

7. Start NGINX

Open the command prompt terminal and enter:

nginx -s reload

nginx

8. Test Load Balancing

Go to http://localhost:70

• We can see that this request was served by the process running on port 72 (as demonstrated above)
• Refresh the page but don’t change the URL. The browser will display:

• We can see that this request was served by the process running on port 72
• Refresh the page again and NGINX will forward the traffic to another python process.

This is because NGINX is balancing the requests. This is how NGINX balances the load of Python applications. We can now serve 1000s of traffic and handle them in an extremely fast manner.

This is how we can build fast, scalable Python applications with 0 downtime.

NGINX is an extremely useful tool

3. Benefits Of NGINX

Lastly, I wanted to highlight the key benefits of Nginx.

• Not NGINX can serve specific routes, it can make applications reliable, reduce downtime and serve millions of requests without overloading one server.
• It also has an inbuilt cache so that the same request can automatically get the same response.
• NGINX can also be used to control security and terminate SSL
• NGINX can be used to monitor the activity
• NGINX can also be used as a Mail Proxy Server

4. Summary

This article demonstrated how we can scale a Python application by using the NGINX tool. This can help us increase the reliability of our infrastructure. Plus it can reduce downtime as NGINX can server millions of requests in a fast manner without overloading a single instance.

It can also switch over to a secondary server if the primary server goes down via configuration. Plus the client code does not need to change. Everything is handled on the server side.

NGINX is an extremely powerful tool and I recommend highly that you explore the tool and attempt to use it within your architecture.