|ARTIFICIAL INTELLIGENCE| COMPUTER VISION| PYTHON|
Harnessing the power of colors in Python
Color images have more hidden information than you think
In this journey into image analysis, we will look at how to exploit the information in images.
In this article we will discuss about:
- What is a color
- How to identify the predominant colors in an image
- How to use color clustering to segment images
- How to recolor an image that is in black and white
- How to all do that in Python
What is a color?
Life is a painting, and you are the artist. You have on your palette all the colors in the spectrum — the same ones available to Michaelangelo and DaVinci. — Paul J. Meyer
The definition of color is not an easy task. We can say that the visible light hits something and absorbs some wavelength and then what is not absorbed is reflected. Notice, that visible light is actually a very narrow range of frequencies/wavelengths which is called the visible spectrum and it ranges from 380 to 750 nanometers. This tiny range of frequencies is actually special since it is something that a typical human eye can capture. The magic happens when our eyes collect the reflected wavelengths of an object and this light excites the human photoreceptors.
In the human eye, the cone is specialized nerve cells that act as photoreceptors. There are three types of cone cells, and each type is sensitive to a different wavelength. There are also rods, which are more sensitive to light and support vision at a low-light level. But cones are perceiving the colors and they are much better at perceiving details. The color perception changes from individual to individual, since the percentage of the cones, can change and there are different genetic mutations (thus, we all perceive color differently).
As a curiosity, human eyes perceive red as being 2.6 times as bright as blue and green, that’s why a single red dot can capture our eye at first glance (yes, like a cat when you use a laser).
All the code used for the image in this article (and in the previous one) can be found here:
Color images
In a color image, we have three values for each pixel: one for the Red, Blue, and Green (or RGB). We can represent each pixel as a 3-dimensional vector, and therefore an image as a three-different array. Normally each color is encoded with 8-bit (meaning 256 shades) and a pixel could represent more than 16 million colors. We can image a pixel as a point in a 3d space, with RGB color axes starting from black (0,0,0).
Notice, that by loading an image in Python we can easily slice it in the color components:
#slicing channels
im1 = im[:,:,0]
im2 = im[:,:,1]
im3 = im[:,:,2]
The line connecting black (0,0,0) to white (255, 255, 255) is called gray-vector. Theoretically, all the colors on the line from black (0,0,0) to red (255, 0, 0) are the same color but just different shades (different levels of illumination). This can be generalized also for the other colors, and we can define the chromaticity plane where the colors on the edge are defined as pure and moving to the center becoming gray (or polluted by the light). This concept is used to normalize the RGB color or to transform it into a large format (color plus intensity).
Dramatic black and white
Black and white creates a strange dreamscape that color never can. — Jack Antonoff
A color image can be converted in grayscale, the transformation is not invertible. The conversion is done by multiplying each channel for weight and summing up:
The weights have a value of 1/3 or in some contexts (for analysis purposes, if you are more interested in plant classification the green is more important) they can have different values. In visualization often, these weights are used:
The threshold on color images is more complex since you need to consider for each color a minimum and maximum value (a range). Since color changes with the intensity of light, this can also be problematic. Often the image is converted to zero before the threshold.
im = io.imread(url)
im1 = (0.2125 * im[:, :, 0]) + (0.7154 * im[:, :, 1]) + (0.0721* im[:, :, 2])
im2 = (0.7 * im[:, :, 0]) + (0.2 * im[:, :, 1]) + (0.1* im[:, :, 2])
im3 = (0.2 * im[:, :, 0]) + (0.2 * im[:, :, 1]) + (0.7* im[:, :, 2])
Counting color in an image
Many times, I was asked to match the color template when preparing a PowerPoint. The eternal struggle in trying to understand if the colors are similar. How we can know which are the predominant colors inside an image? It looks like a simple question, but using Pantone’s master book is not a feasible solution.
As you may know, RGB colors have assigned a hexadecimal code that is used to display on web pages. The hex color code is also recognized widely by Python libraries, so the idea is to obtain a list of the most represented hex colors and their count. The idea is to use a clustering algorithm to find color clusters and we transform the cluster center (centroid) into hexadecimal colors. In this way, since we are forming clusters of color we will extract the most prominent colors. Then we will plot using a bar plot.
def extract_colors(image, n_colors, resize = 0.5):
"""
count colors in an image
"""
def RGB2HEX(color):
"RGB color to HEX colors"
return "#{:02x}{:02x}{:02x}".format(int(color[0]), int(color[1]), int(color[2]))
#scaling if too big
if resize is not None:
"note: in the next version multichannel argument is changed"
image = rescale(image, resize, anti_aliasing = True, multichannel= True)
image = image *255
image = image.reshape(image.shape[0]*image.shape[1], 3)
#clustering step
clf = KMeans(n_clusters = n_colors)
labels = clf.fit_predict(image)
#counting step
counts = Counter(labels)
counts = dict(sorted(counts.items()))
#obtaining HEX colors
center_colors = clf.cluster_centers_
ordered_colors = [center_colors[i] for i in counts.keys()]
hex_colors = [RGB2HEX(ordered_colors[i]) for i in counts.keys()]
#save in a pdf
df = pd.DataFrame(columns= ["colors", "value_count"])
df["colors"], df["value_count"] = hex_colors, counts.values()
return dfLet’s start with this aerial picture
We run our function, which is basically returning a data frame:
Then we can use seaborn to plot the bar plot:
ax = sns.barplot(x="colors", y="value_count", data=df, palette = df["colors"])
plt.xticks(rotation = 45)
Notice what’s happening using different pictures:
Another application is the use of k-means to segment the image. In fact, using clusters we can divide the image into segments. This approach also has the advantage that we can quickly do tests by changing the number of clusters
def segment_image_kmeans(im = None, K =3):
'''
segment an image with k-means
'''
# converting to HSV space
img_hsv=cv2.cvtColor(im,cv2.COLOR_BGR2RGB)
vectorized = np.float32(img_hsv.reshape((-1,3)))
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 20, 1.0)
K = 3
attempts=10
ret,label,center=cv2.kmeans(vectorized,K,None,criteria,attempts,cv2.KMEANS_PP_CENTERS)
center = np.uint8(center)
res = center[label.flatten()]
result_image = res.reshape((img_hsv.shape))
return img_hsv, result_image
k=3
img_hsv, result_image = segment_image_kmeans(im = img, K =K)
Coloring black and white images
An interesting application is to make images that are black and white into color. Especially for old photographs, thanks to deep learning today it is possible to recolor images
In future articles, we will discuss convolutional neural networks and other models. For now, suffice it for us to know that there is an artificial intelligence model behind this process that has been trained to color images. The code is in the notebook.
Concluding remarks
Colors, like features, follow the changes of the emotions. — Pablo Picasso
There is a lot of information hidden in the colors that can be used in many downstream tasks. For instance, extracting colors can be used to search images for their similarity in the color footprint. We can also couple this information with shapes and other features we can extract from the images.
A more efficient way can be extracting features that consider colors, shape, and texture. As an example, we could use a pre-trained model to build a feature extractor and based on this calculate the similarity between images. In other words, as you can obtain text embedding you can do the same with images.
The previous article of the series is here. To be concise here I showed the essential code, but all the codes used are present here
If you have found this interesting:
You can look for my other articles, and you can also connect or reach me on LinkedIn. Check this repository containing weekly updated ML & AI news. I am open to collaborations and projects and you can reach me on LinkedIn. You can also subscribe for free to get notified when I publish a new story.
Here is the link to my GitHub repository, where I am collecting code and many resources related to machine learning, artificial intelligence, and more.
or you may be interested in one of my recent articles:
Additional resources
- If you are interested in a beautiful article about the history of color in art
- feature extraction from images: here
In Plain English 🚀
Thank you for being a part of the In Plain English community! Before you go:
- Be sure to clap and follow the writer ️👏️️
- Follow us: X | LinkedIn | YouTube | Discord | Newsletter
- Visit our other platforms: Stackademic | CoFeed | Venture | Cubed
- More content at PlainEnglish.io
