Malik Saadi, Vice President, Strategic Technologies, ABI Research.
getty
For almost a decade now, the smartphone and personal computer (PC) markets have been stuck in limbo. Refresh cycles for these devices have remained flat or even declined due to a lack of “must-have” features that drive sales and provide a great value proposition to end users. My company, ABI Research, believes the solution lies in artificial intelligence (AI) innovation, which provides enhanced experiences and unprecedented tools for increasing productivity.
This innovation comes in the form of generative AI, which, until recently, was constrained to cloud environments with vast compute and memory resources. However, hardware and software advancements mean these workloads can now run on devices like smartphones and PCs, addressing privacy concerns, networking costs and response latency.
To scale AI more widely across personal/work devices, our analysts have observed chipset vendors, device manufacturers and independent software vendors (ISVs) aligning to support on-device—and later hybrid—AI capabilities. As generative AI proliferates to more devices, ABI Research forecasts over 1.8 billion annual shipments of devices capable of running generative AI on devices by 2030—a far cry from the nearly 200,000 shipments in 2024.
Recent chipset innovations for PCs and various software solutions unlock novel productivity-based applications that entice consumers and enterprises to replace devices.
AI-Based Productivity Apps Boost Device Sales
Only a small minority of people replace their smartphones within a year or two, according to recent survey data, dampening potential hardware sales; the story is even bleaker with PCs and laptops, as replacement rates are now exceeding four to five years. To date, AI app developers have focused primarily on “experiences,” such as photo editing and largely unimpressive voice assistants.
Most people and organizations will not find this to be a compelling enough case to warrant device upgrades. Instead, “killer” productivity apps—using on-device generative AI—can potentially reduce device refresh cycles to less than two years and could enable buyers to justify the return on investment (ROI). The key here is that vendors showcase a clear ROI case for these apps, such as saving time or money.
How On-Device AI Transforms Consumer And Enterprise Applications
On-device AI brings ROI-driven benefits, namely increased productivity while improving security, reducing networking costs and lowering latency. These benefits fuel the adoption of next-generation productivity applications to drive on-device generative AI PCs and smartphones in consumer and enterprise spaces.
Consumers
On-device generative AI applications can help consumers manage smart home appliances, complete their tax returns, optimize energy usage, schedule holidays and other tasks. These productivity applications translate to tangible benefits, such as smaller utility bills and more daily leisure time. These applications also enable prosumers and artists to be more creative in their marketing and in generating their work art (e.g., music generation, video editing, etc.).
Advertising these unique benefits will attract more consumers and motivate them to upgrade their devices more often. As developers iterate or develop new AI-based productivity apps, this could become a long-term consumer trend that yields higher sales.
Enterprises
AI applications based on finely-tuned models can personalize note-taking, automate contract creation, schedule multi-time-zone meetings, create summaries and outcomes of meetings, craft context-based email drafts, etc. Although some of these enterprise tasks sound small, the compounded time savings translate to more cost savings than you may initially think.
For example, an employee making $20 per hour and getting back 30 minutes daily would translate to $2,210 in organizational savings per 231-day working year. Multiply this number by the number of employees using AI-based productivity apps, and you’re looking at significant profitability by adopting mobile devices with embedded AI functionality.
Besides automation and time savings, on-device AI can potentially unlock “killer” Internet of Things (IoT) and extended reality (XR) apps in healthcare, manufacturing, logistics and other verticals. For example, the high reliability of on-device AI negates patchy network connectivity, which is a game-changer for IoT and XR users in rugged work environments (e.g., mining, shipping, etc.). Like the consumer market, the benefits of on-device AI will make organizations more inclined to upgrade work devices and integrate AI into processes.
Provide Software Tools For AI App Development
Translating on-device AI hardware capabilities into commercial success is not just about hardware. Software innovation is vital, and app developers must be well-supported to build optimized and “killer” productivity-focused apps. With this in mind, there are four key software considerations that hardware vendors should account for:
• Machine Learning Operations (MLOps) Tools: Market leaders must invest more heavily in and improve compression, model imitation, pruning, distillation and other techniques that optimize AI. Optimized models, which are smaller, more power efficient and use less memory, facilitate the use of more complicated performant models on-device. Collaborating with optimization leaders and leveraging tools such as retrieval augmented generation (RAG) will be key to developing small, contextualized AI models that offer leading hardware efficiency.
• Open Source: ISVs and startups benefit from open-source models, providing easy access to forward-looking technologies and datasets. These ML tools enable vendors to develop productivity applications tailored for on-device AI.
• Consolidated Software Stacks: Streamlined app development requires a consolidated software stack and an integrated array of tools, frameworks, libraries and technologies. These software stacks promote interoperability, ensuring productivity AI applications can run across central processing units (CPUs), graphics processing units (GPUs), application-specific integrated circuits (ASICs) and other prominent architectures.
• Software Development Kits (SDKs): The steady cadence of new chip architectures should be supported by chip vendors’ SDKs to reduce optimization effort and time for new hardware architectures. This will ensure that apps are efficient and consume less energy.
Productivity AI app development will heavily influence future market demand for mobile devices. Boosted productivity with proven ROI supersedes new “experiences,” no matter how cool the experience may be. Although on-device AI capabilities will be the main catalyst for productivity apps, my team sees hybrid AI architecture that connects cloud, edge and on-device as the leading framework in the future. A hybrid architecture balances the use of cloud, edge and on-device computing based on what makes the most commercial and technical sense for the use case.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
