Local AI testing reveals promise, friction and limits
Early testing of local language models shows that keeping sensitive data on personal hardware can unlock useful automation, but setup and reliability remain challenges.
Radwan AbdulhadiEditor and technology writer
Privacy is driving renewed interest
Local artificial intelligence is moving from an enthusiast experiment toward a practical option for people who want to use powerful models without sending personal information to cloud services. Early testing described by The Verge shows both the appeal and the complications: models running on a user’s own computer can work with sensitive material, but getting them to perform reliably still takes time and technical patience.
The attraction is not simply avoiding a subscription or token charges. A local agent can operate inside a computer and access information that a user would not want to upload, including financial records and documents covered by an embargo. That privacy difference can determine whether someone is willing to use an AI tool at all, even for relatively small tasks.
Powerful hardware can add to the confusion
Apple is promoting its newer Mac desktops for local AI, while a line of Windows systems built around Nvidia’s RTX Spark platform is aimed at agentic workloads. The Windows machines are described as offering up to 128GB of RAM, highlighting how much memory can matter when running larger models locally.
The testing involved an M5 Ultra Mac Studio with 256GB of unified memory, allowing a 125-billion-parameter Qwen 3.8 Flash Next model to run locally. The model occupies about 105GB, illustrating why hardware requirements can quickly become a barrier. Running a local model does not require paying for tokens, but the computer needed to run a large one can be costly; the test describes a $12,000 computer as unnecessary for many of the simpler jobs.
Hermes Agent turns experiments into tasks
The open-source Hermes Agent was installed as the main interface for the experiments. It is a self-hosted desktop application available for macOS, Windows and Linux, and it can be used without charge when connected to local language models. Its model picker made it possible to select Qwen 3.8 Flash Next without manually managing every step of the installation.
The choice also exposed one of local AI’s less obvious difficulties: there are a huge number of models, often optimized for different tasks and hardware configurations. Larger models generally offer more capabilities than smaller ones, but selecting among the available options can be overwhelming for newcomers. Hermes could also be controlled through a Telegram bot, adding convenience while leaving the underlying setup and model decisions to the user.
Small automations show the practical upside
One early project was a daily morning briefing that scanned email and a calendar, highlighted items requiring attention and added a short weather report. The briefing initially failed because the Mac could not be asleep when the scheduled job ran at 7:30am. After that problem was identified, the task worked consistently, although it still needed more useful information to make the result worthwhile.
A more successful experiment involved reorganizing a Steam library containing more than 400 games. Hermes inspected the installed Steam client, suggested organizational approaches and then sorted the games by genre while preserving custom categories such as favorites, co-op titles and party games. To complete the job, it required a Steam web API key, which was revoked once the task was finished.
The same setup was used to analyze financial records and create a specification comparison spreadsheet for a new laptop. These tasks were limited in scope, but local processing made them possible without uploading sensitive information or embargoed material to an external service. A larger project is also underway to automate repeated laptop benchmark tests through Python scripts, though building that workflow remains unfinished.
Local agents still need supervision
The early results do not suggest that local AI has become a hands-off digital assistant. The morning briefing broke multiple times, and even explaining the procedures required for benchmark automation proved difficult. The agent can perform useful heavy lifting, but it still needs clear instructions, access permissions and troubleshooting when its assumptions do not match the user’s environment.
That limitation changes the way local AI is best understood. Rather than a digital companion or an autonomous replacement for professional work, the experiments treat Hermes as a computer program that can handle repetitive operations under supervision. The privacy benefits make some uses more acceptable, but they do not remove the need for caution, testing or human review.
For now, the technology’s strongest case is in narrowly defined tasks involving information that users prefer to keep on their own machines. Local models can save manual effort and protect sensitive data, yet the learning curve, hardware demands and occasional failures remain central parts of the experience.
Sources
- The VergeLearning to use local AI is exciting, overwhelming, and frustrating
Continue chronologically
Related entity coverage
Related stories
Jailbroken PS5s Can Now Run Steam PC Games Locally
Jailbroken PlayStation 5 consoles can now run Steam PC games using a newly adapted launcher that creates individual home screen tiles.
Steam Games Earn $560K, But Co-Creator Nets $15K Annually
A game developer breaks down how high Chinese refund rates, platform fees, and publisher cuts reduce $560,000 in gross revenue to modest earnings.
Gears of War: E-Day Launches on GeForce NOW
Gears of War: E-Day leads the GeForce NOW lineup this week, bringing cloud-based RTX performance, new game releases, and upcoming Fire TV integration.
Moonshot AI Launches Kimi K3 on Amazon Bedrock
Kimi K3 from Moonshot AI is now on Amazon Bedrock, offering 2.8T parameters, native vision, and explicit prompt caching for coding workflows.
