What is an AI PC and what can it do for you?
You can’t get away from talk about AI - or artificial intelligence - right now. And if you’re thinking about buying new PCs or laptops for your business, you’ve probably heard talk about AI PCs. But what is an AI PC, and is it right for you?
Well, an AI PC is any device, desktop or laptop, that comes pre-configured with a combination of hardware and software so it can run AI applications natively, rather than in the cloud. It helps make devices more efficient and enhances everyday tasks in the software you use all the time.
Let’s find out exactly what makes an AI PC, and how it can help you and your business.
About the hardware
Up until recently, computers weren’t really set up to run AI models locally. They used a combination of the CPU (central processing unit) and GPU (graphic processing unit) if one was available. But this isn’t an efficient way to power AI applications.
Modern CPUs don’t process data in a way that suits AI. They’re designed with a few cores, which can process very complex instructions. AI models work better with many smaller cores.
GPUs feature many individual cores, which suit AI processing well. In fact, LLMs (large language models) like ChatGPT run on huge batteries of GPUs. But powerful models are expensive, and they’re only really found in specialist gaming PCs. They also need a lot of energy to run, and using them for AI processes can cause a reduction in graphics power.
The neural processing unit
To get AI applications working efficiently on local machines, a new solution was needed. So chip makers like Intel, AMD and Apple have developed AI-focused neural processing units (NPUs). They’re built on the same physical chip as the CPU, but they’re a separate unit specifically designed to process AI calculations quickly and efficiently.
They’re not super powerful - a modern GPU will almost certainly outperform an NPU in sheer processing power - but they’re far more efficient. Plus, the NPU can draw on the CPU and GPU for extra processing power where it’s needed.
What is a neural processing unit?
NPUs are designed to mirror the neural networks in a human brain. They consist of thousands of individual processing units arranged in a network that allows them to process trillions of computations every second.
They’re built for low-power consumption and are perfect for the kind of mathematical operations needed in AI workloads.
NPU performance
When it comes to performance, NPUs are still a new area. That means there’s no standard benchmark, like you might find for a CPU or a GPU. But the overall performance isn’t really the point of an NPU. It’s designed to perform AI operations efficiently in the background without taxing other system resources.
That said, manufacturers do measure NPU performance, using a metric called TOPS (trillions of operations per second). Current generation Intel Ultra NPUs run at around 50 TOPS. Early models were significantly slower, around 17-18 TOPS.
It’s important to note that TOPS performance isn’t standardised or testable, and the rest of the system can limit NPU performance.
About the software
It’s great having a PC with the hardware to run AI calculations, but without the software to make use of it, that hardware’s useless. Luckily, there’s a whole raft of software options that take advantage of an NPU. And, as AI enabled PCs become more popular, we’ll see more software optimised to use it.
So, let’s explore some of the software that can take advantage of an AI PC.
Generative AI
When most people think of generative AI, they think of LLMs like ChatGPT and Claude, that run on massive server farms and are accessed through the cloud. But, running much smaller, more efficient LLMs locally can be useful for a whole range of applications.
Real-time captioning and translation
Video conferencing software like Microsoft Teams or Zoom can use local AI processing to provide captioning and translation of speakers in real time. It can even differentiate between speakers and translate into different languages in real-time.
Natural language search
New apps like Microsoft’s Recall, built into operating systems like Windows, can remember everything you’ve looked at or done on your computer. Then, when you can’t find something, like a file you can’t remember the location of, or a website you’ve forgotten the name of, you can just ask your computer in natural language. ‘Hey Windows, what was that website I was looking at a few days ago with the kittens on it?’
While this is a powerful tool, it doesn’t come without concerns. There are potential security risks around your historical data being stored, and you’ll have to make sure that it complies with your organisation’s cybersecurity protocols.
Natural language communication
Sometimes, getting a PC to do what you want can be a challenge in itself. You know what you want it to do, but you don’t know the exact combination of menus and button presses you need to wade through to make it happen.
That’s where natural language control comes in. You can just bring up your AI assistant and ask it how to do whatever it is you want. It can’t take control of your PC - yet - but it can explain how to do what you want.
Help with writing
AI writing assistance is already extremely popular. Tools like Office 365 and Google Docs have writing assistance built in, and third-party apps like Grammarly take it to the next level. But all of these tools operate in the cloud, which means you have to be online, and your data has to be shared to use them.
AI PCs can process writing assistance locally, so you don’t need to be online or share your data. This is more efficient, and it removes the security and privacy concerns around sharing your data online.
Image generation
AI image - and even video - generation is everywhere. Online tools like Midjourney and Dall-E can create incredible quality visuals from simple, natural language prompts. But the images you create are publicly available for anyone to use, and it’s another subscription you need to pay every month.
But with an AI PC, you can generate your own images locally. The quality may not be as high as the best online alternatives, and they might take a little longer to generate but you’re saving the subscription cost, and there’s no limit to the number of images you can create.
AI Assistants
Assistive AI technology is still new and developing, but it’s maturing fast. Essentially, AI agents can perform tasks on your behalf to save you time. For example, if you’re planning a large meeting your AI agent could book out a room, send out invitations and share the agenda with attendees. And, as the agent gets to know you better, its suggestions will improve.
Enhancing existing apps
As NPUs in PCs get more common, so many software developers are integrating AI tools and processes into their apps. Sometimes these are locally-run versions of processes that can use cloud AI processing, and sometimes they’re unique to AI PCs.
Let’s have a look at some popular app types, and how they’re using AI to improve your experience.
Image processing
Adobe - who make Photoshop and Lightroom - are big on AI. Current versions of their apps are packed with AI functionality, using their proprietary Firefly and Sensei models.
You can get easy access to incredible tools like generative fill and expand, intelligent object removal, neural filters, automated selection and masking tools, content aware manipulation tools, font recognition and many more.
And they’ll all perform better on an AI PC.
Video processing
All of the big video processing and editing tools, like Premier Pro and Davinci Resolve, have their own AI tools built in.
Things like generative clip extend, AI search to find locations or people, auto caption translation, speech and audio enhancement, colour management, AI masking, reframing, auto scene cuts, face recognition and resolution upscaling are all at your fingertips with an AI PC.
3D modelling and animation
Industry-standard 3D tools like Autodesk Maya and Blender are also starting to implement AI tools in creative ways.
Some tools work in the background to streamline settings, textures and materials. But you can also control and animate characters through AI posing technology using natural language.
Coding and development
There’s a lot of talk about how AI will replace coders, but if you’ve tried it out you’ll know it’s not quite there yet. Models like Claude with its built-in artefacts window are clever, but they don’t understand code like a human developer does.
But as a coding assistant, AI can be an incredible timesaver. Which is why coding systems like Visual Studio now have AI tools baked in.
Visual Studio is integrated with GitHub Copilot to give you things like code improvement suggestions and natural language code conversion. It also uses IntelliCode for code autocompletion, refactoring suggestions and quick actions in the code window, and it can explain code to you, answer questions, optimise code, identify bugs and more.
The benefits of an AI PC
So we’ve talked about some great functionality you can access with an AI PC above. But actually, a lot of what we’ve mentioned is also available on cloud-based online AI systems in some form. So, what are the main benefits to you of local AI processing?
Security
One of the biggest benefits of on-device AI is security. If all you’re using ChatGPT for is personal questions, then where your data goes probably isn’t a major concern. But if you’re using it as part of your business, then security is critical.
When you upload something to ChatGPT or any other online AI system, you have no idea what happens to it once it’s in there. And the connection you’re sending it over isn’t secure.
Using a local AI system removes these problems. Because everything happens on your local machine, there’s no data transfer or file sharing. In fact, if you’re working with really sensitive data, there’s no reason you couldn’t work completely offline, disconnected from the internet, for even more security.
Convenience
Cloud-based AI is great, but if you want to get the most out of everything available, you’ll often find yourself jumping between browser tabs and different apps. Plus, in busy periods things can slow down or even stop.
Local AI is integrated seamlessly with your operating system, and the apps you use every day already. And while your NPU might give you the same raw processing power as ChatGPT or Claude, there’s only ever you accessing it. So the speed is the same all the time.
Deeper access to your machine
Because it’s a physical component of your computer, the NPU is far more integrated than a remote AI can ever be. So that means you get secure AI optimisation of your machine, and a natural language interface to your OS and apps.
What does the future hold?
Realistically, we’re probably not actually going to be talking about AI PCs for long at all. Not because they’re not going to take off, but because soon all new PCs will be AI PCs. It will just become the norm.
Then, after a while, pretty much every computer still in use (not to mention other devices like smartphones and tablets) will have some degree of AI functionality. And over time, it will change the way we interact with technology. Natural language commands will become the norm, and searching through menus to find the one thing you really need will be a thing of the past.
So, getting on board with AI PC technology early means you’re getting ahead of the game, and you can start taking advantage of the benefits offered right away.
