Internet of Things
Impact of Massive And Hidden IoT Data On Performance of Digital Venture Production Systems
Architectural, technical, and design considerations
IoT is ubiquitous now. Our lives depend on the performance and availability of these devices, especially in the health care, life sciences, and financial sector. If IoT devices do not perform and become available in critical times, simply people may die and incur financial losses.
These details are beyond the scope of my article. This article focuses specifically on hidden data produced by IoT devices and sometimes ignored by technical solution professionals.
IoT devices generate massive amounts of data on an ongoing basis. There is nothing new about it. However, the hidden data created by monitoring IoT devices sometimes can be as much as streaming data.
From my experience, the operational and service management aspect of IoT data is sometimes overlooked. This negligence causes serious capacity, scalability and even availability issues at later phases in the lifecycle of IoT solutions.
Streaming data sets received from IoT devices and gateways go through a complete data management life cycle, such as storing, analyzing, re-building, archiving, and destruction. Therefore, the amount of data produced by IoT devices requires careful performance, scalability and availability measures.
Solution architects and designers need to simulate the actual workload models based on the functional and non-functional requirements. They also need to consider historical data and future growth as part of the requirements analysis for performance.
Data collection via IoT sensors needs to be planned carefully. First, solution architects and designers need to determine the type of physical signals to measure. Then, they can identify the number of sensors to be used and the speed of signals for these sensors in the data acquisition plan.
However, in addition to the challenges of massive streaming data, application usage patterns are also an essential factor for solution performance. In particular, the processors and memory of the servers hosting the IoT applications need to be considered carefully using industry benchmarks.
Using benchmarks for applications, data, and infrastructure, solution architects and designers can create a particular IoT performance model and a set of test strategies.
Due to its resource-hungry nature, the IoT performance model mandates more data storage capacity, faster processor, more memory, faster network infrastructure and so on.
While in the traditional performance models, we mainly consider user simulations, in the IoT performance models, we also must consider the simulation of devices, sensors, and actuators.
For these simulations, we need to keep several factors in mind. First, it is paramount to be aware of data frequency shared amongst devices from a data management perspective. This means that not only the amount of data produced and processed but also the data accessed and frequently shared by multiple entities of the IoT ecosystem must be considered and calculated.
The blind spot usually comes from operational and service management domains of IoT data. The monitoring, alerting, and event management of IoT devices creates a tremendous amount of data.
If an organization’s policy requires always adding the alerts, events, incidents, and other system management functions to keep these devices well-performing and available, solution architects and designers need to have a comprehensive performance model, including the system and service management of the complex IoT ecosystem. Failing to factor in operational and service management data can severely impact the functionality and performance of IoT systems in production.
One viable solution is to use IoT gateways in managing both streaming and operational data.
IoT gateways can act as intermediaries between Edge Devices and the Cloud. Gateways can offer additional location management services. Performance and availability also require the consideration of endpoints and gateways. Endpoints consist of physical sensors. These sensors send messages to the IoT platform via gateways, which are fundamental components of the ecosystem.
These gateways are so important because the objects can take place in multiple locations due to their nature — for example, a moving object, such as a transport vehicle with built-in IoT sensors. The messages created by these moving objects are stored by the gateway and delivered by the gateway to the IoT platforms in the ecosystem. In addition, the moving devices can connect and reconnect to the gateways using various data link protocols.
When designing gateways, solution architects and designers should configure gateways to manage operational and service management data. This proactive approach will prevent potential bottlenecks when excessive monitoring and alerting events were generated in expected situations.
To conclude, IoT solutions require special consideration for both streaming functional data and operational support data to mitigate performance, scalability, and availability risks.
Thank you for reading my perspectives.
The original version of this story was published on another platform.
Related Articles
I published several articles related to IoT on Medium.
IoT Is The Next Big Thing In Our Lives
Scalability And Capacity For IoT
Reference
A Practical Guide for IoT Solution Architects
About The Author
I am a technologist, published author, editor, blogger, digital marketing strategist, and content developer with four decades of industry experience. I write articles for Medium, News Break and Vocal Media. On Medium, I established ILLUMINATION, ILLUMINATION-Curated, ILLUMINATION’ S MIRROR, ILLUMINATION Book Chapter, Technology Hits, and SYNERGY publications supporting 9,900+ writers and serving 137,000+ readers on Medium. You can Join my publications requesting access here.
