Building S3 File Storage With Next.js: 1. Building a Local Development Environment

This article is part of a series of articles about building a file storage application with Next.js, PostgreSQL, and Minio S3. Here are the links to the other articles in the series:

1. Building a Local Development Environment
2. Main concepts and shared components
3. Upload and download files using Next.js API route (4MB limit)
4. Upload and download files using presigned URLs
5. Delete files from S3
6. Deploying locally using Docker Compose

You can find the full source code for this tutorial on GitHub

Introduction

As a developer working on a full-stack application, you need to have a local development environment that is as close as possible to the production environment. It will allow you to test and debug your application locally before deploying it to production.

Almost every full-stack application needs a database and a file storage so let’s build a basic full-stack application that can save and retrieve data from a database and upload and download files from a file storage.

You can run your own PostgreSQL and Minio S3 server locally, or even use a cloud service like AWS RDS and S3. But it will take some time to set up and configure. Using docker-compose will make it super easy to set up a local development environment for your full-stack application. After you test it locally, all you need to switch to the production environment is to change the environment variables.

Additionally, you can test your application end-to-end (for example, using Cypress) in a local environment that is as close as possible to the production environment. Having pre-configured docker-compose file will make it easy to set up a CI/CD pipeline like GitHub Actions or GitLab CI.

Once you have a docker-compose file, you and your team can use it to set up the same local development environment on any machine in a few minutes with just one command. Overall, it will save you a lot of time and make your life easier.

In this article, we will look at how to build a local development environment for a full-stack Next.js application with Prisma ORM connected to PostgreSQL as a database and Minio S3 as a file storage using Docker-Compose.

You can find the full source code for this tutorial on GitHub

When you are ready to deploy your application to production, you can use any type of database:

• Supabase (I love this one)
• Vercel Postgres
• AWS RDS
• Google Cloud SQL
• Azure Database for PostgreSQL
• Heroku Postgres
• DigitalOcean Managed Databases
• ScaleGrid
• …

The same goes for the file storage, you need one that is compatible with the S3 protocol:

• AWS S3
• Google Cloud Storage (I tested it, it works)
• Wasabi (One of the cheapest options)
• Backblaze B2
• DigitalOcean Spaces
• …

Prerequisites

To follow this tutorial, you need to have Docker and Docker-Compose installed on your machine. You can find the instructions on how to install Docker and Docker-Compose on the official Docker website.

When I firstly faced the task of setting up a local development environment for Next.js, Prisma and PostgreSQL, I tried to use T3 Docker tutorial but it didn’t work for me. However, I use it as a starting point for this tutorial.

Building a local development environment

1. Create a Next.js application

Let’s start by creating a Next.js application. We will use the T3 stack (TypeScript, TailwindCSS, and Prisma ORM) for this tutorial to skip installing and configuring all the dependencies which is out of the scope of this article. You can find more information about the T3 stack.

Run the following command to create a new Next.js:

npm create t3-app@latest

After you run the command, you will be asked several questions:

   ___ ___ ___   __ _____ ___   _____ ____    __   ___ ___
  / __| _ \ __| /  \_   _| __| |_   _|__ /   /  \ | _ \ _ \
 | (__|   / _| / /\ \| | | _|    | |  |_ \  / /\ \|  _/  _/
  \___|_|_\___|_/‾‾\_\_| |___|   |_| |___/ /_/‾‾\_\_| |_|

◇  What will your project be called?
│  local-nextjs-postgres-s3
│
◇  Will you be using TypeScript or JavaScript?
│  TypeScript
│
◇  Will you be using Tailwind CSS for styling?
│  Yes
│
◇  Would you like to use tRPC?
│  No
│
◇  What authentication provider would you like to use?
│  None
│
◇  What database ORM would you like to use?
│  Prisma
│
◇   EXPERIMENTAL  Would you like to use Next.js App Router?
│  No
│
◇  Should we initialize a Git repository and stage the changes?
│  Yes
│
◇  Should we run 'npm install' for you?
│  Yes
│
◇  What import alias would you like to use?
│  ~/

2. Configure Next.js to work with Docker

According to the Next.js documentation

Next.js can automatically create a standalone folder that copies only the necessary files for a production deployment including select files in node_modules.

To reduce image size we need to add output: "standalone" in the next.config.js file. The next.config.js file should look like this:

const config = {
  reactStrictMode: true,
  output: "standalone",
  // ...
};

3. Add .dockerignore file

I prefer having a separate folder for the docker files, so I created a folder called compose in the root of the project. We need to add a .dockerignore file to this folder to exclude unnecessary files from the Docker image. The .dockerignore file should look like this:

.env
Dockerfile
.dockerignore
.next
.git
.gitignore
node_modules
npm-debug.log
README.md

4. Configure Prisma to work with Docker

In the prisma/schema.prisma file, we need to

• Change the provider from sqlite to postgresql:
• Add binaryTargets to the generator block. It will allow us to use the Prisma CLI inside the Docker container. You need to your binaryTargets specific to your OS and architecture. For example, for M1 Mac, I use "linux-musl-arm64-openssl-3.0.x". To support other OS and architectures, you need to add them to the binaryTargets array. More about binaryTargets in the Prisma documentation.

I want to use the same schema.prisma file for local development on machines with different OS and architectures and also for CI/CD pipelines on GitHub Actions. So in my case, the prisma/schema.prisma file looks like this:

generator client {
    provider      = "prisma-client-js"
    binaryTargets = ["native", "linux-musl-arm64-openssl-3.0.x", "linux-musl-openssl-3.0.x", "rhel-openssl-1.0.x"]
}

datasource db {
  provider = "postgresql"
  url      = env("DATABASE_URL")
}

5. Create a Dockerfile for the Next.js application

Inside the compose folder, create a file called web.Dockerfile with the following content:

FROM node:18-alpine

RUN mkdir app
COPY ../prisma  ./app
COPY ../package.json ../package-lock.json ./app
WORKDIR /app

RUN npm ci
CMD ["npm", "run", "dev"]

We will use the node:18-alpine image as a base image. It is a lightweight image that contains Node.js 18 and npm. We will copy the /prisma folder, package.json and package-lock.json files to the /app folder and run npm ci to install the dependencies.

Using ‘clean install’ (npm ci) instead of 'install' (npm i) is a good practice for Docker images. It will ensure that the dependencies are installed from the package-lock.json file and not from the node_modules cache. This is faster than 'install', which is especially important for CI/CD pipelines where you want to keep the build time as short as possible.

6. Create a docker compose file

Docker compose file is used to define and run multi-container Docker applications with a single command docker-compose up.

Here we will not go into details about docker-compose files. In general, our docker-compose file creates 3 services: web (Next.js application built with our Dockerfile), db (PostgreSQL database), and minio (Minio S3 file storage). Remember to add volumes for the database and file storage services. Otherwise, the data will be lost when you stop the containers.

It is generally not recommended to store environment variables in the docker-compose file. However, in this particular scenario, for educational purposes and given that we are exclusively using it for local development and testing, it looks acceptable. If you do not want to store secrets in the docker-compose file, you should use a .env file and use ${VARIABLE_NAME} syntax to reference the variables. More about environment variables in docker-compose files Docker compose file reference.

Inside the compose folder, create a file called docker-compose.yml with the following content:

version: "3.9"
services:
  web:
    build:
      context: ../
      dockerfile: compose/web.Dockerfile
      args:
        NEXT_PUBLIC_CLIENTVAR: "clientvar"
    ports:
      - 3000:3000
    volumes:
      - ../:/app
    environment:
      - DATABASE_URL=postgresql://postgres:postgres@db:5432/myapp-db?schema=public
      - S3_ENDPOINT=minio
      - S3_PORT=9000
      - S3_ACCESS_KEY=minio
      - S3_SECRET_KEY=miniosecret
      - S3_BUCKET_NAME=s3bucket
      # For example, if you want to use Google OAuth API keys, you can store them in
      # the .env file and add the following variables:
      # - GOOGLE_CLIENT_ID=${GOOGLE_CLIENT_ID}
      # - GOOGLE_CLIENT_SECRET=${GOOGLE_CLIENT_SECRET}
    depends_on:
      - db
      - minio
  db:
    image: postgres:15.3
    container_name: postgres
    ports:
      - 5432:5432
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: myapp-db
    volumes:
      - postgres-data:/var/lib/postgresql/data
    restart: unless-stopped
  minio:
    image: bitnami/minio:latest
    ports:
      - "9000:9000"
      - "9001:9001"
    volumes:
      - minio_storage:/data
volumes:
  postgres-data:
  minio_storage:

Run the application

Depending on where you have your docker-compose file located and what options you want to use,

you need to run the following command:

# If you have docker files in the root of the project
docker-compose up

# In our case, we have dockerfile and docker-compose file in the `compose` folder, so we need to run:
docker-compose -f compose/docker-compose.yml up

# --- Optional ---
# For running the application with secrets ${VARIABLE_NAME} stored in the .env file, we would need to run:
docker-compose -f compose/docker-compose.yml --env-file .env up

# If you want to run the application in the background, you can use the -d flag:
docker-compose -f compose/docker-compose.yml up -d

It will run build the Next.js application and run it on port 3000. It will also download the PostgreSQL and Minio S3 docker images and run them on ports 5432 and 9000 respectively.

You can access the application at http://localhost:3000, the PostgreSQL database at http://localhost:5432 (login: postgres, password: postgres), and Minio S3 at http://localhost:9000 (login: minio, password: miniosecret). Credentials for the database and file storage are stored in the docker-compose file. You can change them if you want.

Conclusion

In this article, we looked at how to build a local development environment for a full-stack Next.js application with Prisma ORM connected to PostgreSQL as a database and Minio S3 as a file storage using Docker-Compose.

References and further reading:

• Containerize T3 stack and deploy it as a single container using Docker
• Next.js deployment documentation
• Official Next.js Docker example
• Docker Compose file reference
• How to use .dockerignore and its importance
• Securing Docker Builds: A Comprehensive Guide to .dockerignore Usage and Best Practices
• Prisma schema reference
• Minio S3 Docker image

In the next article, we add a file upload functionality and database integration to our application.

I hope you found this article useful. If you have any questions or comments, please let me know in the comments below.

Here are the links to the other articles in the series:

You can find the full source code for this tutorial on GitHub

Enjoyed the read? Hit 👏 like it’s a high-five — to motivate me to bring more stories!