The Simple Guide: Deep Learning Installation with RTX 4090 (CUDA, cuDNN, Tensorflow, PyTorch)

This tutorial is tested with RTX4090. All the commands in this tutorial will be done inside the “terminal”.

Purpose: Identify the proper drivers and software versions for getting the RTX4090 (or any RTX40-series) GPU working

For RTX30-Series: https://readmedium.com/the-simple-guide-deep-learning-with-rtx-3090-cuda-cudnn-tensorflow-keras-pytorch-e88a2a8249bc

For RTX20-Series & Full Deep Learning Installation guide: https://readmedium.com/the-ultimate-guide-ubuntu-18-04-37bae511efb0

Headsup: Not recommend to install NVDIA driver with apt because we will need specific driver and CUDA versions.

1. Nvidia Driver & Compute Capability

The RTX40-series has the Ada Lovelace architecture (not in cuda-compat table), but we know it has v8.9 Compute Capability.

Compute Capability 8.9 actually supports both CUDA 12 & 11. But we can only use CUDA Toolkit up to 11.8 for deep learning libraries. The driver requirement would be 525.60.13 or newer.

It’s tempting to download 520.61.05 as it ships with CUDA11.8. But the table already told you 11–8 is NOT compatible (not sure why they do this tho…). We need 525+ but they are already in CUDA12.X.

FYI, if you were using the RTX30-series (Ampere), you might not need to upgrade, except for driver, since you should already be using the Compute Capability 8.x, so you can continue with CUDA11.x.

2. Nvidia CUDA

Referencing above screenshot we can use CUDA11.8.

After selecting the OS and other settings applicable to your system, copy & past the 2 commands from the webpage (rectangular box) into your terminal.

Download from: https://developer.nvidia.com/cuda-11-8-0-download-archive

wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
sudo sh cuda_11.8.0_520.61.05_linux.run

Un-check “Driver” before installing (if you have already installed a Nvidia driver before this step).

When you finish, it will look like this.

3. Nvidia cuDNN (CUDA Compute Capability)

Referencing above screenshot we can use cuDNN 8.9.6 with our CUDA11.8 selection as recommended by the Nvidia Note.

Download from: https://developer.nvidia.com/rdp/cudnn-archive

Install the downloaded cuDNN file afterwards.

See cuDNN 8.9 Improvements

cuDNN8.0 Improvements

3. Python

Spoiler alert: you will need to use tensorflow 2.13.

Given the spoiler, you need to use Python3.8+.

sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install python3.8

To work and code in Python3.8, it’s recommended that you create a new virtual environment (see my full installation guide, up top).

4. Tensorflow (2.13.0)

We won’t have to compiling from source like the old days. But you will still need to check the compatibility table:

In terminal, install the corresponding tensorflow with the following command:

pip install tensorflow==2.13.0

Afterwards, go into your python console, and run the follow code. You should then see True as the output at the end.

import tensorflow as tf
tf.test.is_gpu_available()

5. Pytorch

Pytorch has been the easiest library to install and enable gpu, just go to their website, and use their generated command: https://pytorch.org/get-started/locally/

In terminal, install the corresponding tensorflow with the following command:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

— END —