The Move to the Edge – what drives this next leg of GenAI growth?

GenAI at the Edge has the makings for the next tech product cycle

The introduction of Chat GPT and its expansion to 1.8 billion monthly users has triggered an aggressive ramp of accelerated computing capacity in cloud data centers. However, to date that has only benefited a narrow slice of the supply chain (ie, those supplying and delivering services based on this data center compute and networking), and has not lifted the broader tech chain delivering edge devices accounting for a large portion of hardware and semiconductor demand. We think the introduction of generative AI to edge devices has the potential to stimulate positive mix changes (driving requirements for more processing, storage and upgraded peripherals), and could also pull-in replacement cycles as new devices offer increased usefulness, enabling content creation, productivity and personalization.

Supporting growth acceleration in key segments

We estimate the impact on tech could be far greater from a take-off at the edge, as AI servers represent USD 100bn potential revenue in 2024 versus a USD 450bn smartphone industry, USD180bn PC industry and USD 163bn in IoT semiconductors. Each generation of compute drove a 2x growth in the semiconductor industry including the ramp of mainframes, PC/Internet, mobility/cloud and now potentially with AI + IoT. A ramp of more intelligent end points also supports a virtuous circle of more demand for data transmission, storage and processing back in a centralized cloud, still core to service creation and delivery. As a risk, if it truly takes off and shifts compute closer to the edge, it could limit some cloud service providers' growth and investment in capex/cloud AI.

What's the benefit of generative AI models on edge devices?

Running a model on an edge device benefits the user through: faster real-time data processing without latency to the cloud, reduced bandwidth costs, enhanced privacy and security of personal data, better customization and personalization, and context awareness of the local environment. New experiences it could enable would include ondevice multimedia and content creation; real-time language translation; transcription and meeting notes; smoother video conferences by improving eye contact, background, audio, and video framing; AI-enhanced gaming; faster and more advanced camera editing; and always-available co-pilot assistants also taking advantage of on-device resources. For cloud operators, having this processing on-device offloads compute and storage costs to resources available on the user's hardware.

Enabling technologies have come together for 2024 launches and 2025- 27 ramps

1. We expect 2024 to be a year of increased promotion and initial adoption and 2025-27 to be when the markets may take off from further applications to take advantage of the features as many key elements come together:
   1) Chipsets with high performance, dedicated AI cores for PC, smartphone and IoT capable of running small language models on device hardware from industry,
   2) AI frameworks to build models for edge devices,
   3) Techniques to optimize models for mobile devices (quantization, knowledge distillation, compression, neural architecture search, conditional compute, early/frame exits),
   4) developer kits available from industry leaders to fine-tune models for edge devices and optimize applications,
   5) readily available models and a range of open-source models, and
   6) device hardware launches

Gen AI PCs can drive mix and volume as a device built for content creation

Generative AI is well suited to the PC platform as a content creation device to be embedded in the OS and applications to drive a leap forward in users' productivity and creativity with personal assistants/co-pilots, improved video conferencing, multimedia content creation and AI-enhanced gaming. The newest generation processors focus on improving AI processing with an integrated AI neural processing unit in their 4Q23 chipset launches. We project Generative AI on the PC platform will grow from 25% Gen AI ready PCs today to 60% in 2027, growing from 59mn to 166mn units through proliferation of hardware with these advanced chipsets and from user interest in upgraded features.