3 Ways To Calibrate Your Camera Using OpenCV and Python

Fix camera distortions in an easy way.

When a camera takes a photograph, we often see the image not quite the same as we see it in our brain. This is caused by the camera lens and it happens more than we think.

This alteration of the image is what we call distortion.

Generally speaking, distortion is when a straight lines appear bent or curvy in an image.

There are different types of distortion, depending essentially on the model of the camera lens you used.

Although you sometimes want to create a nice effect on your images, a distortion can be bad for computer vision systems. Since the coordinates of the image are deviated from its original position, you might create some errors or fail to detect an object.

When you calibrate a camera, you discover some specific parameters that will fix these distortions.

Camera Model

A camera model describes a relationship between a point in a 3D space and its projection into a 2D space (an image). The pinhole camera model is often used in computer vision as a reasonable approximation of a camera.

Duration the calibration process, some parameters are discovered to fix most of the distortions caused by a camera:

Intrinsic parameters: focal length, optical center, and skew coefficients. Specific to each camera.
Extrinsic parameters: rotation and translation vectors that translate a 3D scene into a 2D coordinate.
Radial distortion coefficients: it models the radial distortion, that causes straight line to appear curved. It occurs when light rays bend more near the edges of a lens than they do at its optical center.
Tangential distortion coefficients: it models the tangential distortion, that occurs when the lens and the image plane are not parallel.

To find these parameters, we need some images that contains a well defined pattern. The algorithm then find some specific points, whose coordinates we know from a real object. With the coordinates in the image, we then can solve the equations for the coefficients that we want.

There are different objects that we can use to calibrate that we are going to see bellow.

OpenCV

First of all, let’s create our environment. To calibrate our camera and get the coefficients, we are going to use the OpenCV library.

OpenCV is an open-source library that contains a lot of computer vision algorithms, from image processing to object detection. It is a powerful library that can be used on different platforms.

To install it:

pip install opencv-python
pip install opencv-python-contrib

Note: in this tutorial it was used OpenCV version 4.5.1 and Python 3.8.7, in a Windows 10 PC.

Chessboard

We can use a chessboard as the reference object. Since it is well defined and we know all the coordinates, it is a great object to use. In fact, it is the standard method to calibrate a camera.

First, let’s take some pictures of a chessboard using the camera we want to calibrate. You can use a chessboard image or use a real one. At least 10 images are necessary for better results.

Note: try to variate the position of the chessboard to get better results. Remember to put the chessboard on a flat surface.

To find the chessboard inside an image, we are going to use the function findChessboardCorners, passing the image and the pattern (number of rows and columns of the chessboard).

Once we find the corners, we improve their accuracy using the function cornerSubPix.

We do the same for every image we have, recording the corners we have found. After that, we call the function calibrateCamera that will return the camera matrix, distortion coefficients, rotation and translation vectors.

Parameters used in the function:

dir_path: path to the directory where the chessboard images are stored.
image_format: extension of the images to be used.
square_size: size, in centimeter, of each square of the real chessboard. Use a ruler and try to be as accurate as possible.
width, height: how many squares there are in the chessboard (in my case, 6 x 9.

After we get the camera matrix and the coefficients, we then save them in a file so we can use it later without having to go through this process every time.

The complete process is as follow:

Undistort

With the parameters, we then can undistort the image that we receive from the camera.

We use the function cv2.undistort that will return to us a new image.

Original (left) and corrected (right) images. By author.

ArUco boards

The ArUco fiducial marker can also be used to calibrate a camera. A fiducial marker is a landmark added to a scene to facilitate locating points.

What Is a Fiducial Marker

And how they are used on computer vision.

medium.com

An ArUco marker is a synthetic square marker composed by a wide black border and an inner binary matrix which determines its identifiier.

Calibrating using ArUco is much more versatile than using traditional chessboard patterns, since you don’t need to include the complete board view.

The first step of the ArUco process is to create the board. For that, we use the function GridBoard_create, indicating the dimensions (how many markers in horizontal and vertical), the marker length, the marker separation, and the ArUco dictionary to be used.

Note: you can save this board and print it instead of getting the image from the internet. But remember to use with the real dimensions you printed when calibrating the images.

Then, we need to create a list with all the markers and ids we have found in the calibration images we’ve taken. We use the detectMarkers function to get ids within each image.

After that, we pass the markers, ids, and the board to the function calibrateCameraAruco. This function will then return the camera matrix and the distortion coefficients so we can use them later.

And we follow the same process as with the chessboard.

ChArUco

One of the advantages of the ArUco markers and boards is that they are very fast to detect. One the contrary, the detection of the corners of a chessboard can be more accurate than the ones of the ArUco, since each corner is surrounded by two black squares.

A ChArUco board combines these two approaches:

The first step is, of course, to create the ChArUco board. For that, we will need:

Number of chessboard squares in X direction.
Number of chessboard squares in Y direction.
Length of square side.
Length of marker side.
The ArUco dictionary to be used.

When you read an image, you need to detect the corners inside the image with the function aruco.detectMarkers and interpolate the corners with the function aruco.interpolateCornersCharuco.

Do these steps with every image you have taken to calibrate the camera. With all the corners captured, we call the function aruco.calibrateCameraCharuco. This function will return to us the camera matrix parameters as well as the coefficients.

And we follow the same process as before:

Original (left) and corrected (right) images.

Final Thoughts

A camera calibration is an important process for a computer vision application. It will correct the distortions that a camera inserts into an image. If you need to know distances or orientation, it is a must-do process.

In this article, we’ve seen three ways to calibrate a camera. You can use any one of them.

If you have some problems, here are some tips that might help:

Use a flat surface to put your chessboard image. You want to get the distortions caused by the camera, not the object.
Take many pictures as necessary. I’ve found that 20 is a good number to start.
Take pictures with different angles to improve the quality of the calibration.
Use an environment with good illumination.

Hope you have enjoyed the reading.