Testing Major AI Agents: Performance, Pricing, and Features
A recent software evaluation put six major always-on AI agents through a series of tests to examine their proactivity, instruction-following capabilities, and browser control.
Radwan AbdulhadiEditor and technology writer
The Rise of Always-On AI Agents
While early iterations of automated assistants faced significant criticism for reliability issues, the current landscape features always-on tools designed to handle personal workflows. As [AI agents] are booming across the technology sector, major software developers are introducing products aimed at capturing early adopters.
Unlike standard chatbots that operate on a turn-based model, these newer proactive utilities can monitor user data, track flight deals, and manage daily information streams. However, utilizing these programs requires connecting them to personal services such as email and calendars, introducing considerations around security and data privacy.
Overview of Tested AI Agents and Pricing
A comprehensive 24-hour trial evaluated six prominent tools running on an iPhone. The evaluated options varied significantly in both accessibility and cost structure.
Meta’s Muse is available for free and gained traction following its September release. Instinct is also free but requires an invite code and operates via text messaging rather than a dedicated application. OpenAI’s Dots operates within ChatGPT and represents the highest-priced option at $100 per month. Meanwhile, Google’s Gemini Spark is offered as a beta feature inside the Gemini app for $20 monthly, and xAI’s Grok Bot operates in its own app at a cost of $30 per month. Anthropic’s Claude Duet rounds out the group at $20 a month as a wait-listed research preview.
Instruction Following and Research Depth
To test how well these tools follow complex guidelines, the evaluation included generating a tech news briefing and formulating headlines based on specific reporting areas. Instructions specified taking 30 minutes to conduct thorough research before generating output.
Among the tested participants, Meta’s Muse and OpenAI’s Dots continued researching for the majority of the allotted time, whereas the other evaluated tools produced pitches almost instantaneously or after only a brief search period. This difference highlighted variations in instruction-following precision across the platforms.
Editorial Pitch Results and Output Quality
The resulting content from the tools generated mixed reactions when reviewed for editorial use. Among the submissions, Google’s Gemini Spark produced a notable angle contrasting retail policies regarding third-party shopping agents.
Other generated concepts explored topics such as secure encrypted chats, proactive assistant usage limits, and focus applications designed to mitigate digital doomscrolling. While these tools demonstrate expanded capabilities in browser control and task execution, observers note that the software remains nascent and subject to ongoing development.
Sources
- WIREDI Tested Every Major AI Agent for a Day. Here’s the Best One
Continue chronologically
Related entity coverage
Related stories
Google and Meta Challenge OpenAI and Anthropic in AI Race
Google and Meta have unveiled powerful new frontier models, challenging Anthropic and OpenAI and completely reshaping the competitive AI landscape.
AI Executives Sign Pledge to Self-Police Development
Top artificial intelligence executives from leading tech labs have gathered in Washington to sign a new joint commitment focused on safe development.
Google's Gemini AI Hacked Three Companies During Security Tests
Google's Gemini AI model hacked three real companies during security testing due to a misconfiguration, but the company claims the AI stopped the breaches immediately.
AI Models Rapidly Stripped of Guardrails to Aid Extremism
A new Counter-Terrorism AI Benchmark report reveals how open-weight AI models are quickly stripped of guardrails to assist terrorist activity.
