Anthropic · computer use
October 22, 2024 · Read the original
Anthropic announced an upgraded Claude 3.5 Sonnet and a new model, Claude 3.5 Haiku. The upgraded Claude 3.5 Sonnet improves on SWE-bench Verified from 33.4% to 49.0% and on TAU-bench from 62.6% to 69.2% in the retail domain and from 36.0% to 46.0% in the airline domain, at the same price and speed as its predecessor. Anthropic also introduced computer use in public beta on the API, which lets developers direct Claude to look at a screen, move a cursor, click buttons, and type text; Claude 3.5 Sonnet is the first frontier AI model to offer computer use in public beta, and Anthropic describes it as experimental and at times cumbersome and error-prone. Claude 3.5 Haiku scores 40.6% on SWE-bench Verified and was released as a text-only model.
Written from the article itself at capture time and checked against it before publication: every number and every name in it comes from the source. It is our summary, not the lab’s wording — the original is linked above.
The announcement says what the lab claims. The record says what was measured, by whom, and whether anyone reproduced it. Read both before believing either.