avatarThiago Carvalho

Summarize

Getting Started with OpenCV

First steps towards computer vision with Python

A while back, I trained an object detection model for a college project, but honestly, I don’t remember much about it besides the fact it required lots of Redbulls and long nights watching my model train.

I’ve recently regained some interest in those topics, and I decided to start over and learn it again, but this time I’m taking notes and sharing my learnings.

Build with reiinakano.com/arbitrary-image-stylization-tfjs

— I wonder if someday we’ll be able to use style transfer to copy styles from one data viz to another without compromising its integrity.

OpenCV

OpenCV is an open-source library, initially developed by Intel, and it’s filled with handy methods and functions that support computer vision and machine learning.

In this article, I’ll get my feet wet learning how to read images, display them in a Jupyter Notebook, and how we can inspect and change some of its properties.

import cv2
import numpy as np
import matplotlib.pyplot as plt

Let’s start with .imread to load the picture, and then we can use .imshow for displaying it in a new window.

image = cv2.imread('img.jpg')
cv2.imshow('Some title', image)
cv2.waitKey(0)
cv2.destroyAllWindows()
Image from /u/niketagl

The methods .waitkey and .destroyAllWindows are essential to run our code without crashing. The first will tell Jupyter to keep running that block until some key is pressed, and the second will close the window at the end.

We can also try displaying the image with Matplotlib .imshow; that way, it’ll be displayed inline instead of in a new window.

image = cv2.imread('img.jpg')
plt.imshow(image)
Matplotlib showing a BGR image

Uh, ok. That looks weird. The colors are all messed up.

OpenCV loads the images as Numpy arrays, and those have three dimensions Reds, Greens, and Blues. The dimensions are often referred to as channels, and they hold values from 0 to 255 that represents the intensity of color for each pixel.

>>> print(type(image))
>>> print(image.shape)
<class 'numpy.ndarray'>
(776, 960, 3)

That means it’s RGB, right? Not really. It’s BGR, which is the same thing but in a different order.

Matplotlib uses RGB, and that’s why our pic was looking weird. That’s not an issue since OpenCV has some very convenient methods for converting colors.

image = cv2.imread('img.jpg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
plt.imshow(image)
Matplotlib showing an RGB image.

Cool, we got to read and display our image with OpenCV and got a peek at how to convert GBR colors into RGB to display them inline with Matplolib.

Other color formats can be handled with OpenCV, like HSV, CMYK, and more.

Colors

Since we’ll be repeating this a lot, let’s create a method for plotting with Matplotlib. We can set the size of the plot and remove the axis to make it even better.

def show(img):
    fig, ax = plt.subplots(1, figsize=(12,8))
    ax.axis('off')   
    plt.imshow(img, cmap='Greys')

Note that I’ve also defined the colormap in .imshow as ‘Greys’; That parameter will be ignored when we plot RGB images but will be helpful later on when we draw the individual dimensions of the arrays. For now, let’s try our method.

image = cv2.imread('img2.jpeg')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
show(image)
Picture by Pixbay

Alright, now let’s try converting it to grayscale and then to RGB.

image = cv2.imread('img2.jpeg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
image = cv2.cvtColor(gray, cv2.COLOR_GRAY2RGB)
show(image)
Grayscale

We can use .split to get individual arrays for the colors and assemble the picture back together with .merge. That’s practical for modifying, inspecting, and filtering a single dimension of our array.

For example, we can multiply the array by zero to remove it;

img = cv2.imread('img2.jpeg')
B, G, R = cv2.split(img) 
img = cv2.merge([B*0, G, R*0])
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
show(img)
Only Green.

We can increase or decrease the intensity of a color, or build a new Numpy array with the same shape to replace it, or whatever you can think.

img = cv2.merge([np.ones_like(B)*255, G, R])
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
show(img)
Increased blues.

The same concept of split and merge can be applied to other formats such as HSV and HSL.

img = cv2.imread('img2.jpeg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
H, S, V = cv2.split(img)
 
img = cv2.merge([np.ones_like(H)*30, S+10, V-20])
img = cv2.cvtColor(img, cv2.COLOR_HSV2RGB)
show(img)
Yellow hue, higher saturation, lower value.

HSV: Hue, Saturation, and Value.

That format is handy for filtering colors since it works with hue — That means, instead of having to figure out the ranges of combinations between red, green, and blue, we can use ranges of angles.

Wikipedia

We can define a lower and upper HSV boundary with Numpy. Apply the method .inRange to filter those values, and create a mask. Then we can apply this mask at the saturation with .bitwise_and, that will make everything outside the boundaries turn to zero.

That in other words: We can filter some colors and make all the rest in grayscale.

# read img and convert to HSV
img = cv2.imread('img2.jpeg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# split dimensions
H, S, V = cv2.split(img)
# upper and lower boundaries
lower = np.array([80, 0, 0]) 
upper = np.array([120, 255, 255])
# build mask
mask = cv2.inRange(img, lower, upper)
# apply mask to saturation
S = cv2.bitwise_and(S, S, mask=mask)
# assemble image
img = cv2.merge([H, S, V])
# convert to RGB and display
img = cv2.cvtColor(img, cv2.COLOR_HSV2RGB)
show(img)
Blues.

Splitting the image also allows us to inspect its composition more easily.

We can plot a color from RGB, a Saturation from HSV, or any other channel we want.

img = cv2.imread('img2.jpeg')
B, G, R = cv2.split(img) 
show(B)
img = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
H, S, V = cv2.split(img)
show(S)

With the ‘Greys’ colormap, the values go from white(low) to black(high).

We can tell by looking at the first map that the intensity of blue is higher in the ground than it is in the building, and we can see with the saturation plot that the values around the skateboard are higher than in other parts of the image.

I’ll stop here for today. We explored how to load and display our pictures, how to convert the array to different color formats, and how to access, modify, and filter the dimensions.

For the next one, I’ll try to explore transformations and how to move, resize, crop, and rotate images.

Thanks for reading my article. I hope you enjoyed it.

Resources: OpenCV Read Image; OpenCV Color Conversions; Matplotlib Display Image; OpenCV Operations on Arrays; OpenCV Basic Operations;

Data Science
Python
Opencv
Computer Vision
Programming
Recommended from ReadMedium