TLDR:
- Anthropic has released new AI models that can directly control computers like humans
- The upgraded Claude 3.5 Sonnet achieves record-breaking 49% score in coding tests
- New Claude 3.5 Haiku model matches performance of previous top model at lower cost
- Beta testing partners include major tech companies like Amazon and Canva
- Computer Use feature allows AI to browse web, fill forms, and complete complex tasks
Anthropic revealed a major advancement in artificial intelligence technology on Tuesday, introducing new AI models that can directly control computers like humans do.
The announcement marks a significant milestone in the evolution of AI capabilities, with the company’s Claude AI system now able to navigate websites, click buttons, and complete complex computer tasks independently.
The company’s upgraded Claude 3.5 Sonnet model has demonstrated remarkable improvements in coding abilities, achieving a 49% success rate on the industry-standard SWE-bench Verified test. This score surpasses all publicly available AI models, including specialized coding systems and OpenAI’s offerings.
Along with Sonnet, Anthropic introduced a new model called Claude 3.5 Haiku. This faster, more efficient version matches the capabilities of their previous top-tier model while operating at lower cost and higher speeds. The Haiku model scored an impressive 40.6% on coding tests, positioning it as a powerful option for businesses seeking balance between performance and efficiency.
The standout feature of this release is the new Computer Use capability, currently available in public beta. This technology allows Claude to interpret what appears on computer screens, move cursors, enter text, and navigate through various software applications. Early testing partners, including Amazon, Asana, and Canva, have already begun implementing these features in their operations.
Jared Kaplan, Anthropic’s chief science officer, explained the significance of the development.
“The tool can use computers in basically the same way that we do,” he said, noting that it can handle tasks requiring “tens or even hundreds of steps.”
The company plans to roll out consumer access to these features in early 2024.
In practical terms, the new Computer Use feature opens up possibilities for automating complex tasks like booking flights, scheduling appointments, filling out forms, conducting research, and managing expense reports. This represents a shift from simple chatbot interactions to more sophisticated computer-based task completion.
The development team at Anthropic has been working on this technology since early 2024, with Amazon receiving early access to test and implement the features. The public beta release focuses initially on developer access through the Anthropic API, Amazon Bedrock, and Google Cloud’s Vertex AI platforms.
Performance metrics show significant improvements across various benchmarks. The upgraded Claude 3.5 Sonnet improved its performance on TAU-bench, scoring 69.2% in retail domains and 46% in airline-related tasks. These improvements come without any increase in operational costs or decrease in processing speed.
Early feedback from companies testing the technology has been positive. GitLab reported up to 10% improvement in reasoning capabilities across various use cases. The Browser Company, another early tester, indicated that Claude 3.5 Sonnet outperformed all previously tested AI models in web-based workflow automation.
Anthropic has implemented several safety measures alongside these new capabilities. The company developed specific classifiers to monitor computer use and detect potential harmful applications. This proactive approach aims to prevent misuse in areas like spam, misinformation, or fraud.
The launch comes at a time of intense competition in the AI industry. Major tech companies including OpenAI, Microsoft, Google, and Meta are all working on similar AI agent technologies, making Anthropic’s public beta release a significant move in the market.
For developers interested in accessing these new features, Anthropic has made the Computer Use capability available through their API. The company emphasizes that while the technology shows promise, it remains in an experimental phase and may occasionally be “cumbersome and error-prone.”
The rollout strategy includes a phased approach, with Claude 3.5 Haiku scheduled for release later this month. Initially, it will launch as a text-only model, with image input capabilities to follow in subsequent updates.
Anthropic’s new release represents a significant step toward AI systems that can function as virtual collaborators rather than simple assistants.
As Scott White, a product manager at Anthropic, noted in September,
“We’re moving to a world where these models will behave much more like virtual collaborators than virtual assistants.”
