The topic of AI has been everywhere in recent years. It is becoming increasingly difficult to find people who have not heard of artificial intelligence. New solutions are constantly appearing on the market. Admittedly, most of them work on a simple principle. “A few sensors collect data, a microprocessor processes it, and then transfers it to the cloud. This is where all the ‘magic’ happens. This operating principle is effective for many devices. But when response time is critical, the process significantly increases operating time.

That is why we are increasingly encountering AI-powered embedded systems. This is quite a revolution in the Internet of Things, because the entire process of analysing data from sensors takes place in the device. Simple information is sent to the end user in the form of an alert. The device becomes virtually autonomous. The question is, why is this new model so important and how does it really surpass the traditional approach? Let’s analyse the differences between a classic IoT device and a smart Edge AI device.

Why is "edge AI" a game-changer?

Let’s start with the fact that a traditional IoT device simply collects data and sends it in its entirety to the cloud. You could say that such a device is a ‘courier’ who picks up a package and runs with it to the base, i.e. the cloud. There, the data is unpacked and analysed. This can cause delays, because the information has to travel a long distance. In addition, there are costs associated with data transfer. This in turn creates further security and privacy risks, which is quite important in the case of MedTech devices.

However, thanks to Edge AI, problems resulting from delays or the costs of transferring large amounts of data cease to exist. This is possible by placing all the computing power in the device. This means that all data is processed locally, directly on the microcontroller or dedicated chips from Nvidia. As a result, the entire process is much faster. The device also does not have to send queries to the cloud for every little thing. Most decisions are made locally, and only relevant information is sent to the cloud.

Where are we seeing AI-powered embedded systems?

Smart City Monitoring

In one of our projects for our client, we used Edge AI to create Smart City Monitoring. The device monitors the environment and reports badly parked cars, potholes in the road, or litter to the relevant service locations. It processes video in real-time to identify issues and instantly sends an alert with a photo and precise GPS coordinates to the city’s central system. Municipal services can mount it on their vehicles, such as buses, refuse collection vehicles, or taxis. Instead of streaming video 24/7, our embedded system with AI on board analysed the video locally and sent only aggregated data to the cloud indicating where the problem was. This drastically reduced network usage and costs and sped up the entire process.

Executive Assist

Smart city devices are not the only ones that use AI-powered embedded systems. Examples can be found in virtually every industry related to IoT. A device that straddles the line between smart home and smart office is Executive Assist, developed by our team. The AI is a voice-activated personal assistant for the CEO. The CEO can request scheduling, travel, or document retrieval. The system instantly processes commands and routes them to team members. It automates routine tasks for the human assistant to focus on strategy. By automating routine tasks, Executive Assist significantly reduces the personal assistant’s workload, allowing them to provide more personalised and impactful support. This leads to higher executive satisfaction and improved staff morale, building a more efficient business model and a stronger reputation for innovation.

MedTech devices

Another good example of the use of edge AI is MedTech devices, where very often the timely transmission of information can determine a person’s survival. Imagine a situation where we have a mobile ultrasound machine used by a doctor in an ambulance. Instead of sending the entire video image to the cloud for analysis, AI integrated into the device assists in real-time diagnostics. It avoids sending the entire video image to the cloud for analysis. Instead, the system immediately marks concerning areas, like lesions or haemorrhages. This helps doctors make faster, more accurate assessments. It is especially useful in field conditions. Remember, AI supports specialists; it doesn’t intend to replace them. It simply gives them crucial time to react at key moments.

The technical reality - It's not as simple as it sounds

The use of edge AI in a microcontroller-based embedded systems can significantly improve its performance, but implementing it may not be as straightforward as it seems. Devices located at the edge of the network typically have limited computing power and memory, which means that complex AI models must be much more optimised than if they were running in the cloud. Despite optimisation, AI can drastically increase energy consumption, which in the case of battery-powered devices significantly shortens the device’s lifespan and forces frequent replacement of the power source.

Another challenge may be the lack of an operating system, as many MCUs run on bare metal or a lightweight RTOS. This results in a lack of support for dynamic model loading and virtual memory, which can help the model run with limited physical resources. With limited memory, firmware updates (OTA), which are necessary for implementing new models and introducing security patches, can also be problematic for devices with limited security mechanisms.

An additional challenge in edge AI is ensuring compatibility between models trained in new frameworks and the limited environment of microcontrollers, which often have to operate in an unchanging configuration. Model optimisation libraries (e.g. TensorFlow, CMSIS-NN) present another problem because developers constantly evolve them and may stop supporting them for older hardware. In practice, this means ensuring long-term software stability, even when the open-source community abandons its development.

The future of AI-powered embedded systems

When we look at the whole picture, the shift we’re seeing from traditional IoT to Edge AI-powered embedded systems is much more than a simple technological upgrade. It’s a fundamental change in philosophy. And IoT has changed over the years. It was about connection, now it’s about intelligence. 

We’ve seen that moving the “brain” of the device away from the cloud onto the edge on microcontrollers or dedicated chips solves some of the biggest problems, like the need for constant network connection, or delays in data transfer. Now Edge AI makes devices faster and more reliable.

We can process complex video for Smart City Monitoring device in real-time or provide life-saving, immediate diagnostic support in MedTech devices without having to stream terabytes of data. 

Of course, no one says it is easy to implement AI on the IoT device. Especially with constrained power, memory and lack of full OS. It is an engineering challenge that requires deep expertise in optimization, managing OTA updates and long-term reliability. But AI is changing the IoT world. It’s now longer only about connecting things, it’s enabling them to act smartly and autonomously. 

This is where the real competitive advantage lies, especially for IoT product companies. If your product roadmap includes adding real-time, intelligent functionality to your hardware, you need a partner who understands the difference between a cloud model and a highly optimized embedded one.