Global – October 29, 2026 -- Small language models under 10 billion parameters are starting to run on GPU/NPU-powered edge hardware with as little as 4GB of RAM, but most deployed IoT devices still depend on the lightweight compute inside the cellular modem SoC, according to Telit Cinterion.
Cellular modules host inference on the same chip that manages the network
Edge AI runs the model on the device itself. The device responds with near-zero latency, consumes far less bandwidth and keeps working if the network connection drops. Cloud-dependent products send a sensor value to a server and wait for the result, which adds latency and network dependence.
In many IoT products, the cellular module is the edge. The chip that handles the radio and network connection can also run a small model. Because the module reads its own signal quality, a model can monitor network availability and optimize reliability dynamically.
Wake-on-demand inference makes edge AI viable on battery-powered hardware
The best power-consumption results combine classic heuristic processing with pipelined AI model usage. The module wakes the processor for inference only when raw sensor data warrants it and stays idle otherwise.
Dedicated accelerators are no longer a prerequisite for edge AI
Until recently, edge AI meant adding costly dedicated microprocessors or external hardware accelerators, which raised both bill-of-materials cost and board space. Two major trends have changed that equation, according to the company.
SoC size caps model size across high-volume IoT products
The smaller the SoC, the smaller the model must be. Most high-volume IoT products run streamlined models scaled to compact cellular SoCs. Practical tasks are narrow and well-defined, such as anomaly detection, audio keyword detection and targeted vision. Single-stage object detectors are enabling localized execution of high-resolution video analytics.
Deep multi-modal reasoning and frontier large language models still require massive cloud clusters. Larger models also rely on the cloud for heavy lifting.
Telit Cinterion says the central question is which part of a product should run a model. A motor's vibration signature or a spoken keyword in a noisy room are likely candidates, but each carries its own limits on memory, power and field accuracy.