Sensors and Actuators in Machine Learning: How Do They Work?

We are taught in Data Science that analysis and learning requires we follow a process of data extraction, preparation, cleansing, modelling, and evaluation. Let’s talk about two basic entities that make machine automation of these steps possible.

Our senses are an intrinsic part of our life experience. They are essential for perception, ideation, imagination, and remembering. We need senses for learning and knowing; and without senses we wouldn’t be capable of interaction and understanding and a “sense” of self.

Our senses provide information about our environment and individual selves, and the way we respond to that information defines what and/or who we are, what we think or feel, what we decide, and how we act.

Senses are an undeniable component of learning, so it makes sense that in the data science worlds of automation, artificial intelligence, machine learning and robotics — the most successful technologies include two entities which are described as “Sensors” and “Actuators.”

Let’s discuss what these two technologies are, and how they are applied in digital products and services to improve our quality of life.

What are Sensors?

As artificial intelligence (AI) and machine learning (ML) continue to advance, there will always be a demand for efficient ways for machines to “learn” from humans and their environments.

For machines, this kind of learning is designed to improve performance, especially when it comes to how they process data. Before they can process this data, machines first need to know that the data is there, and to make that happen, machines need to be connected to what are called “Sensors.”

Sensors are hardware and/or software systems designed to capture data via their own processes of observation. Think of sensors as a type of “detector”, like how an optical smoke detector “senses” the presence of smoke, which means there is a fire.

Sensors work by capturing physical “real world” events, and through the use of firmware and/or software algorithms, and they covert the events into electronic signals. Why electronic signals? Let’s use the smoke detector analogy again.

For the detector to work, it needs to “sense” smoke; And while humans use their sense of smell to sense smoke — for a detector to work, humans designed a “sensor” process by combining inside the device — electric or battery power, an infrared light emitter directed into a chamber designed take in smoke, and a photodiode light receptor — all to create the imitation of sensing smoke. Think of the sensor as the one that says, “Is there smoke in here? I will keep checking.”

What are Actuators?

Since we know what happens when smoke enters the chamber, how does the detector know when to trigger the alarm? This is where the “Actuators” come into play.

Actuators are hardware and/or software systems designed to receive data from sensors — in order to perform specific actions. In the detector, the photodiode light receptor and integrated circuit are a part of the “actuator” process. Think of the actuator as the one that says, “Oh! There must be smoke in here. I need to sound the alarm.”

The light receptor “senses” that it received scattered infrared light, which means there is smoke inside of the detector. Once the receptor registers this, it “actuates” i.e. springs into action and sends an electronic signal to the integrated circuit, which triggers the alarm.

Keep in mind, we are talking about an artificial way to “smell” smoke and to alert people of a possible fire, which is the point.

Artificial intelligence (AI), whether it’s ChatGPT or a smoke detector, do not experience senses in the way humans do. So as humans, our best strategy is to use available technology and ingenuity to create ways to imitate i.e. create “artificially intelligent” experiences for software and devices.

Wearable Fitness Smartwatches:

In the healthcare and fitness industry there’s no shortage of products that use the power of AI and ML. With products like Peloton and Tonal, and companies like Apple and Samsung competing for pieces of the HealthTech pie—the industry is consistently evolving with innovations. Let’s take the health and fitness brand, Fitbit for example.

Fitbit is known for their wearable fitness smartwatches, combined with an ecosystem of software designed to help consumers become mindful about living better lives. Whether it’s about providing access to online medical care or virtual coaching — Fitbit is about empowering consumers to actively monitor their health; And how do they do this in regard to the smartwatch? Let’s take a look under the hood to understand how their sensor and actuator cycles actually work.

Haptics

Fitbit licenses from the California based company, Immersion to use what’s called “Haptics” within their smartwatches. Haptics use the sense of touch in experiences where consumers are required to be in prescribed positions and perform movements with parts of the body.

The earliest commercial examples came in PlayStation and others, in which game controllers vibrate in users’ hands during gameplay, which enhance the UX. Nowadays, haptics are in VR platforms such as the Metaverse, to create an even deeper immersive virtual environment.

For Fitbit smartwatches, Haptics include sensors and actuators designed to provide vibrations and other motion feedback in the user experience (UX). This generates a simulated sense of touch for users.

Sensor Patent

According to the US Patent No 8,351,299 filed by Immersion — sensor technology in Fitbit smartwatches include:

An apparatus comprising:

- A housing;

- A sensor coupled to the housing that senses motion of the housing and provides a sensor output based on if the sensed motion exceeds a predetermined threshold;

- A timer coupled to the housing that measures at least one time period and provides a timer output on expiration of the at least one time period; and

- A vibrotactile device that provides a haptic output based on the sensor output; if the vibrotactile device receives the sensor output before the timer output, and provides the haptic output based on the timer output, if the vibrotactile device receives the timer output before the sensor output.

Translation:

Inside the smartwatch, a sensor is encased in a housing which enables the sensor to detect when you move in certain ways. Every time you move, the housing itself moves, and the sensor inside it detects the housing movement. If the movement grows to a certain level of intensity, the sensor will produce a signal. In regard to the housing — it includes a countdown timer, designed for use during exercise workouts. There is also a vibrotactile device, designed to make Fitbit users feel vibrations during workouts. The device vibrates either when the sensor sends a signal, or when the countdown timer sends a signal — whichever comes first.

Actuator Patent

Also, according to the US Patent No 8,059,105 filed by Immersion — actuator technology in Fitbit smartwatches include:

A haptic feedback device, comprising:

- One or more processors configured to receive an input signal and generate a force signal based on the input signal, wherein the input signal is associated with a user-independent event, the user-independent event comprising one or more of a reminder event, an initiation of a task, a processing of the task, a conclusion of the task, a receipt of an email, or an event occurring in a game; and

- One or more actuators configured to receive the force signal and impart a haptic effect based on the force signal.

Translation:

Inside the watch, there are actuators designed to receive signals. When they receive signals, it could trigger an action for a reminder, or to begin an activity, or to continue an activity, or to end an activity, or you’ve received an email, or to signal that something has happened in a game oriented activity. When the actuators receive the signal for any of these reasons, the actuators respond by sending the appropriate signals to the vibrotactile device, which makes the watch vibrate.

Sensors and Actuators Are Everywhere

Depending on the industry and how advanced the technology, there are various types of sensors and actuators that serve various purposes. Any system that relies on some form of machine learning or “smart processing” will not work without them. Some are software-based, hardware-based, or a hybrid of both, but all have the same purpose — to complete specific tasks.

Sensors and actuators are not just good for checking for smoke or heat based sources. Others are used for detection of other entities via infrared light, photoelectric light, stereoscopic vision, radar, air or wind, water, gas, chemicals, metals, and other elements. Sensors are also designed to check how elements behave, such as the temperature, pressure, humidity, size, their motion, their speed, or their distance.

Sensors today are also capable of generating data in regard to what is there, when it’s there, and how it appears — but sensors can’t improve upon how they do this — at least not on their own.

This is where the best DS, AI, and ML specialists come in and apply advancements in what are called, “Neural Networks.”

Neural Networks: How Machines do “Deep Learning”

Data Scientists are taught that effective analysis and learning requires that they follow a process of data extraction, preparation, cleansing, modelling, and evaluation; And of course, DS specialists are expected to know how to apply these strategies in their data with their tools and techniques.

However, there are computer systems which include advanced algorithms, designed to give machines the capability to analyze and learn on it’s own — and of course, in artificial ways.

The process is called machine learning via “Neural Networks”, or as it’s also called, “Deep Networks” or “Deep Learning.”

Think of deep learning as the product of massive amounts of data, passing through a system of sophisticated algorithms that include many sensor and actuator cycles and calculations — all designed to train the machine to perform a specific task, or, a specific set of tasks.

With deep learning, the data is fed into the machine algorithms; And not only is the system sensing and actuating by performing calculations — it learns; And it performs tests on what it learns, and then it generates data based on its test results and learnings. This testing and learning process is what ML experts call, “Neural Net Training.”

The machine then takes the results and learnings and creates additional data layers within the neural network. Within each new layer, the machine stacks the data on top the previous layers; And now the top layers benefit from the historical data contained in each layer as you go down. As the algorithms dive down into previous layers, it makes improvements to the data and the network’s performance. The process of stacking and diving to improve previous layers is what experts call, “Backpropagation.”

For each layer, the machine has now optimized its data and capabilities in how it senses, actuates, performs tests, and learns. The more layers it creates, the deeper into the stacked layers the algorithms go in order to improve upon its performance capabilities, thus “Deep Learning.”

Over time, depending on how advanced the algorithm is, the system repeats the learning process. It includes the data it generated in the newer layers; And it performs the learning, layer stacking, and layer diving processes in comparatively recursive ways.

This is the core purpose of a neural network — to continuously improve upon the learning process within the machine environment so that it performs its specific task, better.

Since a neural network is about mastering a task, then if you connect many neural networks to each other and program them to collectively perform several tasks — it’s called “Artificial Intelligence.”

Neural Networks Are Still Viewed As “Black Boxes”

While what these neural networks have been able to accomplish is remarkable, they are still a very long way from being capable of “thinking” in ways that humans can — and that’s okay.

For what these networks can do, in order to perform designated tasks well, massive amounts of preferably accurate data sets still need to processed in the system. This is to ensure that the machine does not learn from flawed data and provide fatally inaccurate results.

In regard to the complexity of the results from AI and ML processes, deep neural networks are still in many ways “black boxes”. It means that while these systems may get tasks done, how it specifically gets things done is still mostly unknown.

It’s not just unknown to the designers and engineers who build neural networks. Neural networks themselves don’t have the capability to explain how it performs tasks. It’s a way of saying — while AI products like ChatGPT can generate a result, it can’t tell you how it arrived at that result, because like all AI, it’s not self aware. To make the outcomes of the ML algorithms easier to understand, more research and investigation is necessary.

Beyond understanding the ML algorithms and the stacked layer relationships they create — there is also the business and legal accountability questions that need to be addressed before these technologies go fully mainstream. After all, we can’t afford as a global society to have AI systems go fully autonomous within our critical service infrastructures such as, but are not limited to — government, banking, electricity, food, water, education, finance, healthcare, and military.

It will be a long and hard journey, but for the data to be as valid and accurate as possible, proper quality control measures need to be implemented. That’s why for systems like this, it’s best to do the difficult work upfront before releasing these systems out into the world with deeply flawed data models.

That’s why, for now, AL and ML experts are still holding on to the master keys — at least until they have a more confident understanding of how these deep learning systems truly perform their functions.

Will Machines Become Smarter Than Humans?

So many of us humans ask whether machines with AI and ML technology will become smarter than humans. At this time, it’s still too early to definitively know the answer.

However, no matter how advanced machines get at sensing, actuating and learning — we have to remember that when machines are “learning”, what they’re really doing is carrying out processes conceived by DS, AI, and ML designers — as a way for machines to simulate learning.

In reality, these machines are just running sophisticated algorithms and calculations on only the tasks that humans prescribe. This is not the same as how people learn.

What you, the human, must keep in mind is — a machine’s version of learning, is genuinely no more than an AI or ML designer’s attempt at an interpretation of how machines might learn a specific task, or as it’s aptly called — an imitation.

Once you consider this fact, it puts things in perspective. It makes sense.