TL; DR. FrontierCUA is a benchmark of 40 held-out enterprise workflows on functional clones of real SaaS, spanning domains like support, commerce, CRM, ITSM and site reliability. The best model, GPT-6.1 Sol, passes only 25%.
1 · Where FrontierCUA sits
Frontier labs have made incredible strides in building powerful computer-use agents (CUAs) in the past couple years. Just 2 years ago, early CUAs struggled with concepts as simple as grounding (localizing the target interface element an instruction refers to and predicting its on-screen coordinates), and completing multi-step workflows (clicking more than once). Benchmarks, like ScreenSpot, OSWorld, and WindowsAgentArena, that measured these abilities primarily targeted personal assistant — everyday general computer work — short-horizon tasks.
Now, most models can complete longer-horizon (hundreds of steps) multi-app tasks with ease. Newer benchmarks like OSWorld 2.0, MyPCBench, focused on 1) expanding the horizon length 2) personal assistant tasks 3) multi-app workflows. Agent’s Last Exam (ALE) is a CUA (and coding/tool-use) benchmark covering expert workflows in specialized software applications like CAD modeling and weather forecasting.
Each of these targets everyday, personal-assistant, or specialist desktop work, leaving enterprise operations largely uncovered. Yet enterprise operations are exactly the work most worth handing to an agent: the arduous, high-volume jobs that keep a business running. FrontierCUA targets enterprise operations, the operational core of a business in domains like CRM and sales, support, ITSM, SRE, finance and more, across eleven domains and growing! And where OSWorld, MyPCBench and ALE lean on open-source, simulated or specialist software, FrontierCUA runs on high fidelity functional clones of the commercial SaaS like Zendesk, Shopify, HubSpot, GitHub, Jira, Cloudflare, Datadog, Notion and Google Workspace.
Apps/task is a mean; horizon is the mean executed actions per task as reported by their papers (Opus 4.8 for OSWorld 2.0, Opus 4.6 for MyPCBench, GPT-6.1 Sol for FrontierCUA). A dash marks an unreported metric.
2 · FrontierCUA
FrontierCUA is a private set of 40 enterprise workflows, each run on functional clones of the apps it uses. The agent works only from the screen: it reads a screenshot and acts with a mouse or keyboard action.
Provider Interface Anthropic models (Claude Opus 5 / 5.5, Fable 5 / 5.1, Sonnet 5) use the native computer-use tool. OpenAI models (GPT-5.6 / 6 / 6.1 Sol, GPT-6.1 Terra, GPT-6 Astra) use the Codex agent, reaching the GUI through a desktop CLI. Open models (DeepSeek, Qwen, Kimi, GLM, MiMo, Grok, Muse) use the OSWorld pyautogui agent, via OpenRouter or each model’s native harness.
Coverage
Short instructions, long-horizon tasks. The instructions are around 70 words, placing it lower than ALE and OSWorld 2.0.
How it’s built
Every task is built and tested in three standard stages: task design, rolling out, and quality checks. In particular, we’ve put emphasis on the realism, complexity, and diversity of the task design.
Task design
Seed world. We start from a specific enterprise workflow (e.g. a support team reconciling refunds across a help desk and a storefront) and build the world around it: simulated users, their work streams, and the records they generate across the clone apps, so every task sits in an account that reads as lived-in. We build the clones ourselves, reproducing the real products’ interfaces in full, so an agent works the same surface an operator would. It has to stay inside that interface, with no dev tools, export or API URLs, or code.
Diversity. We author across a set of axes: domain, app coverage, record type (Zendesk tickets, Weights & Biases runs), how the deciding detail is represented, the business process, and the output shape. To keep the set diverse, we cap how often any one axis repeats and check each new instruction for word overlap against all the others.
Instruction. Each task plays out over a long, multi-app workflow where the deciding detail is easy to miss, and difficulty comes from how these steps stack and interact. The instruction is generated by an LLM from the seeded world, the diversity criteria, and the enterprise workflows, and we stress it to be fair but vague: it states clearly what a correct result is, while leaving how to reach it to the agent.
Verification. We grade with an agentic judge against atomic rubric checks, each a single yes/no on the final state, weighted by importance. We opted for an agentic judge over a programmatic verifier because task deliverables may be open-ended and live in application state, a list in a sheet, a summary in Slack, an issue in Jira, and the same correct answer can be written many valid ways. It reads the final-state diff and scores each criterion independently.
Run infrastructure
The agent interacts with the seeded world and acts through the GUI until the task is done or a time budget runs out (a two-to-three-hour wall-clock cap); the median run is about 374 steps and 36 minutes. The agent has no access to the source code for the clone apps and no API or database for them. When the run ends, the environment records each app’s final state.
The agentic judge then reads that captured state, the records, messages and documents the agent left behind, as the source of truth. The judge is also given access to the rollout to resolve intent (to avoid the agent gaming task completion evidence).
Quality assurance
Before a task ships, we check each of its four parts, the instruction, the rubric, the seeded world, and the rollout.
Instruction. An expert should be able to finish the task from the instruction and the environment alone. A reviewer tries exactly that; if they fail, the problem is clarity, and the instruction goes back for sharpening.
Rubric. The rubric has to be calibrated and self-contained: a no-op scores 0, a reference solution scores 1, and every criterion is atomic and checkable from the saved state alone.
Seed world. The world must hide no shortcut to the answer: we sweep every record, document and metadata field for leaks, and we close anything that would trivialize the task, a sortable column, an export, a dashboard total.
Rollout. The task has to be both solvable and robust to gaming. One hinted run has to solve it in full, which proves it is feasible, and a reward-hacking pass then scans the rollouts for edge cases: degenerate or empty outputs that still satisfy a criterion, unintended side effects, and GUI bypasses (typed API or export URLs, dev tools, raw file access). An independent reviewer then signs off that the task is feasible and fair.
Failure modes
No model comes close to solving the set. The grid below shows where each one passes, and the case study that follows looks at one failure up close.
Where the best model loses points
Every task is solvable as written: each ships with a reference solution that scores 1.0 in the same apps. GPT-6.1 Sol passes 10 of 40 tasks, and its failure modes fell into five groups:
Evidence gathering (39%). The agent acts before gathering all the facts the decision rests on, stopping its search early or never opening a source it needed.
Reasoning over evidence (30%). The agent gathers the right information but reasons over it wrongly, misreading a rule or drawing a conclusion the evidence does not support.
Stopping before finishing (24%). The agent declares itself finished with the task only partly done and most of its budget unused, often after handling the first few items and treating the rest as done.
Interface errors (5%). The agent chooses the right action but misses on execution, mostly grounding: it clicks the wrong spot or selects the wrong element, so its correct intent never reaches the app.
Arithmetic slips (2%). The agent has the right inputs but makes a mechanical error in the calculation, a wrong sum, a mis-sort, or an off-by-one.
The same groups lead across all 20 models; on the 10 newest tasks, reasoning over evidence is the largest (45%).
Case study
Routing sales inquiries (CRM). In one task, Opus 5.5 has to route forty-four inbound sales leads in HubSpot to the right salesperson, using rules posted in Slack and tracing customers through Zendesk tickets and Notion contracts. Opus opens every page it needs, then misses that two of the leads are actually from the same customer.
The rule is simple: just route the inbound lead to the right salesperson. The catch is that two of the leads look like different companies. Leah’s form gives one business name and Gwen’s gives another, but both list the same USDOT number. A USDOT number belongs to the trucking company itself and stays fixed even when the carrier rebrands or picks up a new trade name. Here it is one company trading under two names, so both leads are really a single account and have to be handled as one.
The failure is a reasoning-over-evidence miss. The matching USDOT number is the overriding rule here: because Leah and Gwen are the same company, both leads go to the same salesperson. Opus never connects that the two forms share that number, so it skips that rule and just follows each ticket on its own, routing Leah to Marcus Chen because that is who her ticket points to.
3 · What’s next
FrontierCUA is a growing benchmark, and we will keep building it out on a few fronts:
More tasks across the domains it already covers.
Longer-horizon workflows across more domains and applications: MLOps, data engineering, IT and security operations, and finance operations.
So expect many more versions to come! Explore the live benchmark and leaderboard at frontiercua.com.
4 · About Collinear
Collinear AI builds the RL environments and evaluations that show where frontier models break, and studies how they learn. Our research team studies how to design the most realistic, complex, and diverse worlds and how to verify these tasks.
If you’re interested in learning more, check out our Research and other blogs, or our other benchmarks CWE-bench, MatFlowBench and YC-Bench!