Enterprises Start to Find Uses for AI at the Edge

Data-driven experiences are rich, immersive and immediate. But they’re also delay-intolerant data hogs.

Think pizza delivery by drone, video cameras that can record traffic accidents at an intersection, freight trucks that can identify a potential system failure.

These kinds of fast-acting activities need lots of data — quickly. So they can’t sustain latency as data travels to and from the cloud. That to-and-fro takes too long; instead, many of these data-intensive processes must remain localized and processed at the edge and on or near a hardware device.

“An autonomous vehicle cannot wait even for a tenth of a second to activate emergency braking when the [artificial intelligence] algorithm predicts an imminent collision,” wrote Northwestern University professor Mohanbir Sawhney in “Why Apple and Microsoft Are Moving to the Edge.” “In these situations, AI must be located at the edge, where decisions can be made faster without relying on network connectivity and without moving massive amounts of data back and forth over a network.”

“AI edge processors allow you to do the processing on the [device] itself or feed into a server in the back room rather than having the processing being done in the cloud,” said Aditya Kaul, a research director at Omdia, a research firm.

AI at the Edge: Enterprise vs. Consumer Adoption

The ability of AI chips to perform tasks such as machine learning inferencing has expanded dramatically in recent years. Consider the graphic processing unit (GPU), which offers more than 10 teraflops of performance, equal to 10 trillion floating-point calculations per second. Modern smartphones have GPUs that can handle billion floating-point operations per second. Even a couple of years ago, this kind of on-device processing wasn’t available. But today, devices at the edge – smartphones, cameras, drones – can handle AI workloads.

Only with the emergence of deep learning chipsets – or artificial intelligence-enabled silicon including GPUs among other chips – has this been possible. And the AI chipset market has taken off like a rocket.

“From essentially zero a few years ago, [edge AI chips] will earn more than $2.5 billion in ‘new’ revenue in 2020, with a 20% growth rate for the next few years,” wrote the authors of the Deloitte report, “Bringing AI to the Device.” [See Figure 3 to the right.]

According to the Omdia report, “Deep Learning Chipsets,” the market for AI chipsets is expected to reach $72.6 billion by 2025.

Experts say the consumer market has paved the way. Today, in 2020, the consumer device market likely represents 90% of the edge AI chip market, in terms of the numbers sold and their dollar value.

“The smartphone market is at the leading edge of this,” said Aditya Kaul, senior director at Omdia, which recently released the report “Deep Learning Chipsets.”  Smartphones still represent about 40-50% of the AI chipset market.

But, Kaul said, AI-enabled processing at the edge is coming to the enterprise, in areas such as industrial IoT and retail as well as health care and manufacturing.

“You can call it ‘enterprise-grade AI edge,” Kaul said.

The impetus for enterprise adoption of AI at the edge, Kaul said, is “clarity on use cases.” Machine vision, for example, which automates product inspection and process control, can improve the quality and efficiency of …