Building A Simple RAG System With Fastapi (2)

In part 1 of this series of articles, I created the outline of our fastapi application. I also created the first service we will need for our RAG system to work: the file parser.

The file parser is responsible for the extraction of text content from input files. I made sure to respect the open closed principle so that the file parser can be easily extended by registering a new specialised parser, and is closed for modification at the same time. I did that by using an elegant solution using a parser factory. Here is the link to the first article of the series:

One thing I didn’t do in my first article is programming defensively.

Defensive programming is a methodology that involves writing code in a way that anticipates and handles potential errors or unexpected usage scenarios. It is an approach that aims to ensure the program continues to function correctly under a variety of unforeseen circumstances.

When applied correctly, can significantly enhance the quality, security, and reliability of software. However, it’s important to strike a balance to avoid overly complex or inefficient code. Excessive checks and validations might impact performance. Overusing defensive programming can lead to overly complex code.

Here is how I rewrote the app.py file in a defensive manner. I added logic to check the file extension of uploaded files. I added a try…catch to handle any error during the process of saving the uploaded file to the disk. I used shutil for secure file writing.

I also decided to rewrite the file parser defensively and add type annotation. Type annotation is useful to make the code base easier to understand, among other things.

Now, one thing I want to do is to save information about the file in a database, so that when later on the user wants to ask questions about a specific file, I will know how to retrieve it. In our case, let’s use a PostgreSQL database to store the file name and the file content. The database will also automatically create a unique file id, that I will later use to retrieve information about a specific file. For each file, I also want to launch the process of creating content chunks and their corresponding embeddings as soon as the file is uploaded, and potentially do that in the background.

I will do that tomorrow. Stay tuned if you are interested.

Also, do not hesitate to let me know your thoughts in the comments.