AI Chips in Your Devices Explained: What Every NPU Actually Does

By Chester Takau · July 2026

The AI chip in your device is almost always an NPU — a neural processing unit built into your phone or laptop's main processor specifically to run small AI tasks without draining the battery. It's not a separate part you install; it's already inside nearly every phone sold since 2023 and most laptops sold since 2024. It handles things like keyboard prediction, portrait blur, and voice transcription in the background, constantly, at a couple of watts. It is not the same chip that runs ChatGPT or Claude — that work happens on GPUs in a data center, no matter how powerful your device's NPU is.

A phone, laptop, and earbuds shown in cutaway with a small glowing AI chip highlighted inside each device, representing the AI chips in your devices explainer

What actually is an NPU, and how is it different from the CPU and GPU?

An NPU (neural processing unit) is a small, dedicated section of your device's chip built only to do the repetitive math neural networks run — multiplying and adding huge grids of numbers, over and over. Your CPU can technically do that math too, but it's a generalist, built to juggle everything from opening apps to running your browser, so it does AI math slowly and burns more power doing it. Your GPU can do it faster still, since it already has thousands of small cores built for parallel work, but running it constantly at GPU power draw would empty a phone battery in an hour. The NPU exists to sit in the middle: purpose-built for AI math, running at roughly 1-2 watts instead of the 150+ watts a discrete GPU pulls, so your device can leave it switched on all day. IBM's plain definition of the term covers the underlying hardware if you want the more technical version.

What does the NPU in my phone actually get used for day to day?

More than most people realize, and almost none of it announces itself. Portrait-mode background blur, night-mode photo processing, live captions, voice wake-word detection, keyboard next-word prediction, and background noise suppression on calls are all typically NPU jobs on a modern phone. Qualcomm's Snapdragon 8 Gen 5, which is shipping across 2026 flagship phones, upgraded its NPU specifically to push these kinds of on-device features up to 46% faster than the previous generation. None of that shows up as a chatbot you open — it's running quietly underneath features you already use, which is exactly why so many owners have no idea their phone even has a dedicated AI chip.

Why does a phone with a higher TOPS number sometimes run AI slower than one with a lower number?

Because TOPS (trillion operations per second) measures a ceiling, not real-world speed, and it's measured inconsistently between vendors. A chip's TOPS figure changes depending on what precision it's calculated at — a "50 TOPS" rating at INT4 precision is a different, less accurate kind of math than a "30 TOPS" rating at INT8, so the bigger headline number can come from cutting corners on accuracy rather than doing more real work. This gap shows up in actual usage, too: in 2026 testing, a Pixel 10 ran local LLM inference at roughly 10.4 tokens per second while an iPhone 17 hit about 52 tokens per second on a similar-generation chip — a 5x real-world gap that the two phones' advertised TOPS numbers don't explain. Apple's Neural Engine on the A19 comes in around 38 TOPS, while Qualcomm claims 100 TOPS for the Snapdragon 8 Elite Gen 5 — and Google doesn't publish a TOPS figure for Tensor G5 at all. Treat TOPS the way you'd treat a car's horsepower number on a spec sheet: a real input, but not the whole story of how the thing drives.

Does on-device AI actually keep my data private, or does it still phone home?

When a task genuinely runs on the NPU, yes — the data never leaves the chip, because there's no network round-trip involved. Voice transcription, face-unlock matching, and photo scene detection that run locally stay local by design; that's the entire reason phone makers built dedicated AI silicon instead of just routing everything to the cloud. The catch is that "AI feature" and "on-device AI feature" aren't the same claim. Plenty of marketed AI features — anything conversational, anything that needs a large model — still send data to a server, NPU or not, because the model is too big to run locally. If privacy is the reason you care, check whether a specific feature is described as on-device or offline-capable, not whether the phone has an NPU at all, since nearly every phone does now.

Can I run a local AI chatbot on my phone's NPU, and how fast is it?

You can, but expect it to feel noticeably slower than typing into ChatGPT or Claude in a browser. The token-per-second numbers above are the practical answer: even flagship phone NPUs deliver single-digit to low-double-digit tokens per second on a real local model, well below the pace most people read at comfortably on the faster end, and clearly behind it on the slower end. A laptop with a discrete GPU still beats any phone NPU by a wide margin for this specific job, because generating text is memory-bandwidth bound, and mobile NPUs simply don't have the memory bus a GPU does. A phone NPU is built for short, constant background tasks, not sustained conversation with a large model — that's a different job with different hardware requirements.

The video above walks through GPUs, TPUs, and NPUs together rather than treating phone chips in isolation, which is a useful visual companion if you want to see how the different AI chip types map onto real hardware.

Does the NPU drain battery, or does it save battery compared to the CPU or GPU?

It saves battery, and that's the entire point of the chip existing. Running the same AI task on the CPU or GPU instead of the NPU costs meaningfully more power for an identical result, because neither was built for that specific kind of math at low wattage. That's also why NPUs have spread past phones and laptops into smaller devices: the same low-power AI silicon now shows up in wearables handling gesture and heart-rate pattern detection on a coin-cell battery, which would be impossible if that processing had to route through a bigger, hungrier chip. You'll notice the NPU by your battery lasting longer during heavy camera or voice use, not by seeing it listed anywhere in your settings.

Will my phone's NPU still be useful in 2-3 years, or does it need software support to matter?

It needs ongoing software support more than it needs raw specs, and that's the part the industry is still getting wrong. Android Authority's 2026 analysis points out that phone NPUs have existed since the 2017 Huawei Mate 10 — eight years — and are still underused because the software layer keeps getting abandoned: Samsung dropped its own Neural SDK, and Android's original NNAPI framework has been deprecated. A great NPU sitting under an operating system that stops routing tasks to it is just idle silicon. Before assuming your current phone's AI chip will "still work" in a few years, the more useful question is whether the manufacturer has a track record of shipping features that actually target the NPU — Apple and Google have been more consistent here than the broader Android ecosystem.

Which chip has the best NPU right now — Apple, Snapdragon, Tensor, or Dimensity?

It depends whether you're judging by raw spec or by what you'll actually feel. On paper, Qualcomm's Snapdragon 8 Elite Gen 5 claims the biggest number at around 100 TOPS. In real-world large-model throughput, Apple's tighter hardware-software integration on the A19 currently wins by a wide margin, despite a lower published TOPS figure. Google publishes no TOPS number for Tensor G5 at all, leaning instead on custom software features layered on top of it. On the PC side, Qualcomm's Snapdragon X2 Plus launched at CES 2026 with an 80 TOPS NPU aimed at agentic tasks, while Intel's Panther Lake chips ship a 50 TOPS NPU across more than 200 laptop designs — and Gartner projects that by the end of 2026, effectively every new enterprise PC sold will include a dedicated AI chip. There's no single winner across brands right now; there's a winner for whichever specific task you're asking about.

So which AI chip should you actually pay attention to?

If you're buying a new phone or laptop, don't shop the TOPS number — shop whether the specific features you want (offline translation, local transcription, on-device photo editing) are ones the manufacturer actually ships and keeps supporting. That's the same logic that applies to AI security cameras that process footage on-chip instead of streaming it to the cloud — the chip only matters if the software routes real work to it. It's the same story with AI smart speakers that handle voice commands locally versus ones that quietly send every request to a server. And if a term in this piece — NPU, TOPS, inference — didn't fully land, the AI and gadget terms glossary breaks each one down on its own. Every device you own with a screen or a microphone probably has one of these chips in it already. The interesting question was never whether you have one — it's whether anything on your device is actually using it.

A phone, a laptop, and a pair of earbuds shown in cutaway with a small glowing chip highlighted inside each one, dark bl

Transparency note: This article was researched and written by Chester Takau with AI assistance for research gathering and drafting. All recommendations reflect the author's own editorial judgment.