tldr; based on my current understanding of evaluations, RL environments, and the hill-climbing loop:

an environment (or evaluation) is fair when score differences are driven mainly by the capability you intend to measure, and are mostly invariant to nuisance factors like contamination, verifier bugs, environment drift, and benign prompt paraphrases.

I use the word evaluation and environment interchangeably since for all practical purposes of a modern multi-tool multi-step setup they are the same.

I hated taking exams in most of my courses in undergrad and then in my PhD. For the most part, I thought the exams did not measure the core skills required to apply the subject matter in the real world. I am afraid that the gradients for language models and agents share my feeling.

This article is an effort to distill the key features of a fair evaluation, scoped to RLVR and agent harnesses. Fair not with respect to a protected attribute like race or gender, but fair to the agent you are evaluating.

Two concrete things are in scope of this article, and more will soon follow:

• The verifier inside an RLVR loop the function that takes a rollout as an input and produces a reward.

• The agent harness: the tools, scaffolding, and protocol the agent acts through during rollouts, evals, and production.

What “fair to the agent” means

A capability is a repeatable ability of an agent to produce a desired outcome under specified conditions.

A fair eval is a measurement of a specific capability of the AI agent.

What is it not? It is not a measurement of outcomes which were not expected in the specified conditions.

Score differences between agents, or between training checkpoints should be explained by the capability under test. They should not be explained by nuisance factors:

• Train-set contamination.

• Verifier bugs and overfit rubrics.

• Environment drift between runs.

• Benign paraphrases of the prompt.

• Harness details the agent never sees in deployment.

If an eval is sensitive to these, its not a fair eval.

Some ways an eval can be unfair

We discuss four most-popular ways here, but this is non-exhaustive, and I will keep extending this as we find more gaps.

1. Prompt Underspecification

This happens if the task misses a constraint any reasonable solver would need to achieve the task meaningfully.

If the verifier reports a fail, the outcome is confounded with ambiguity of the instructions. Two equally strong agents can swing several points apart on the same task. If the instructions are ambiguous the agent should be rewarded equally for equally valid solution paths.

2. Environment Issues

The sandbox has a stale package, a non-responsive tool, or a filesystem layout that drifts between runs or setups. The agent’s first action fails for reasons it cannot inspect. Such a failure should be attributed to the environment.

3. The harness design should be environment-centric

The agent interacts with the environment through the harness. Tool schemas, observation formatting, retry policy, max steps, error messages, all shape behavior.

Two harnesses with the “same” tools can produce very different scores. For example:

1. A truncated stderr hides the bug.

2. A misspecified tool schema can cause unnecessary confusion.

3. An unintentional 10-step cap might hamper a good agent’s planning capability.

Therefore any fair eval should have a uniform eval across different

4. Don’t ask it to do things it would not do in the real world

Your environment, tasks and verifiers should not expect the agent to achieve a goal which is unrealistic in practice. A few common ones:

• Tasks the agent is told to refuse in production but expected to attempt in eval.

• Tools available in the gym that do not exist in the deployed surface.

• A persona or role the deployed system never adopts.

Since if the agent were to hillclimb for these - it would not improve the capability you want to measure.

The role the verifier plays

The verifier is a model of “what success looks like.”

A strict verifier on an underspecified prompt punishes reasonable behavior. A lenient verifier can turn failures into passes. Verifiers are rarely audited as carefully as the agents they grade but their errors compound at every step of hill-climbing.

Treat the verifier as a system under test.

• Measure its agreement with humans.

• Measure its variance across paraphrases of the same correct answer.

• Measure its false-positive and false-negative rates.

Reward hacking is a fairness problem

In RL, the verifier is the reward. Anything the verifier accepts is a valid policy.

If the verifier can be satisfied without solving the task meaningfully, the agent will eventually find that shortcut. This is not the model’s or the algorithm’s fault. It is the eval’s fault. A few common shortcuts:

• Producing answers in a format the verifier scores leniently.

• Exploiting tool calls in a way that gets a good reward but doesn’t affect the state of the environment.

• Pattern-matching the rubric’s expectation instead of producing a correct answer.

• Outputting both the answer and its negation when the verifier checks for substring presence.

Fair RL requires that the only cheap way to get reward is to do the task. If a cheaper path exists, the agent harness improvement loop or the gradient across the rollouts will discover it, and rightly so.

A checklist for fair evaluations

Before you let an eval drive decisions or training:

• Specification. Could a competent intelligent entity solve the task from the prompt alone, without insider knowledge?

• Harness parity. Does the eval harness match the deployment harness on tools, formats, and limits?

• Distribution match. Are eval tasks ones the agent would actually face, and be permitted to attempt in production?

• Verifier audit. Has the verifier been graded against humans or SoTA model trajectories? What is its False Positive & False Negative rate?

• Paraphrase invariance. Does the score change when the prompt is rewritten without changing meaning?

• Failure attribution. When the agent fails, can you tell whether it was the agent, the harness, the environment, or the verifier?

Open questions

• How do you measure verifier quality when human labels are themselves noisy or expensive?

• What is the right unit of “agent failure” in long-horizon tasks where many small slips compound?

• Can verifiers be co-trained with agents without collapsing into a stationary state where both of them are poor?

• How do you detect reward hacking inside an RLVR loop before it shows up as a deployment regression?

• What is the minimum harness contract that should be held constant across rollout, eval, and production?

• For tasks with no programmatic verifier, how do you keep model-graded RLVR from drifting into preference-style noise?

If you’re shipping agents and model and have faced similar problems with fair evaluations, we should chat. We have a lot of interesting private and public evaluations which cover terminal use, MCP and computer use environments. Our focus is ensuring high-quality data. And fair-evaluations are a core tenet as we scale the horizons our agents operate on.

Kim Philby, standing in his mother’s flat in Drayton Gardens, London, before an array of reporters, was guilty. He had been guilty since 1934, when a Soviet agent named Otto recruited him in Regent’s Park. He was guilty when he joined MI6 in 1940, and when the King awarded him an OBE in 1945 for his wartime intelligence work. And he was guilty in 1955, when the Foreign Secretary stood up in the House of Commons and cleared his name.

It was not until he had been posted to MI6 station in Beirut, when confronted by his old MI6 friend Nicholas Elliott, did Philby ever confess to spying for the Soviets, albeit partially. Shortly thereafter, he disappeared into the night onto a Soviet freighter bound for Russia, where he lived for the remainder of his life.

The signals about Philby had always been there. The Communist sympathies from his years at Cambridge, a first marriage to an Austrian communist. The system saw it all and, time and again, cleared him of any suspicion or wrongdoing.

This is not an article about mid-20th-century espionage. This is about what happens when a system has to verify something for which it has no reliable ground truth for, using signal that captures what to look for, but not how to weigh it.

What this has to do with AI

There’s a funny conversation happening in the AI community right now. Researchers and frontier labs are increasingly dismissive of humans’ ability to verify or review long-horizon tasks, but are also insistent that human data is what unlocks the next 1000x in model capabilities. Both are probably true statements, but they point to two different problems getting collapsed into one.

The first is bandwidth. A human cannot sit through a four-hour agent trajectory, follow every tool call, and reliably verify the work to a high degree of accuracy. But this is a solvable problem, whether it be through better tooling, sampling, or even breaking longer tasks into smaller checks.

The second is criteria. In unverifiable domains, where there’s often no clean ground truth, two qualified experts can review the same output and disagree on whether it is good. Not because one of them is wrong, but because “good” is a judgment call, and judgment calls don’t average across annotators the way correctness does.

This is the harder problem to solve. In unverifiable domains, the verifier is not approximating a ground truth because there isn’t one. The verifier is somebody’s weighting of criteria, which means in unverifiable domains, the verifier is taste.

Taste isn’t one thing

When most people talk about taste, they treat it as a single thing, but in reality, taste has two parts, each of which behave very differently.

The first is criteria. The line items and properties in a rubric that dictate when an agent’s output is considered good. For a legal brief, that might look like:

• A clear issue statement that frames the legal question

• Citations to controlling authority

• A coherent narrative spine that connects the facts to the legal argument

Criteria is mostly binary, in that either the brief cites the right cases or it doesn’t. Either the brief is succinct or it isn’t.

On criteria, MI6 had Philby cold. The Communist circle at Cambridge, the first marriage to an Austrian communist, the Cambridge friends who had already defected to Moscow. All of it was in his file, surfaced and reviewed more than once. MI6 had the rubric, and it scored him on the rubric.

What it didn’t have was an answer to the next question. Confronted with a Cambridge man, the son of a celebrated Arabist, decorated in the war, vouched for by the right people, what do you do with the red flags? Every time MI6 ran the question, it answered the same way: down-weight them.

That next question, what to do with the line items when they coexist, conflict, or trade off against each other, is weighting. Weight is continuous, context-dependent, and the harder half of the problem of capturing taste.

For example, two experienced litigators can sit down with the same case and agree on every criteria. Both want a clear issue statement, both want the right authority cited, both want a tight narrative. But they will disagree on whether to file an aggressive version of the brief or the pared down version. The disagreement is about how to weigh the criteria against each other relative to a specific context: a specific judge, venue (e.g., local vs federal court), client’s risk tolerance, etc. The end result is you still have the same rubric, but different verdicts.

The human data machines today are quite good at the first half via pairwise preferences, rubric annotation, etc., but none of it captures the trade-off logic that turns items into a verdict.

Why aggregation doesn’t rescue this

The obvious objection at this point is that we already have a tool for this. RLHF works by collecting preference pairs from many annotators. If you collect enough of them, the model is supposed to learn the implicit weighting through statistical aggregation. Surely scale solves the problem then?

It doesn’t, because aggregation captures the median annotator’s weighting in the typical or average context, which, for unverifiable domains, is what you want to avoid.

Value in unverifiable domains comes from non-median judgment applied to non-typical context.

A great legal brief isn’t the median lawyer’s brief. A great research direction isn’t what most reviewers would pick. A good investment thesis is, almost by definition, one most people disagree with at the time of the trade. The whole point of bringing in expert judgment is to access the part of the distribution that consensus would smooth away.

When you average preferences across annotators and contexts, you smooth out the signal that distinguishes good from average judgment. You don’t end up with a verifier that approximates expert judgment. You end up with one that approximates consensus annotator judgment, and those are different things.

Let’s imagine we ran this on approach on Kim Philby. A vetting model trained on the aggregated preferences of every MI6 officer in 1945 would have produced exactly the verdict the system produced: he’s one of us, the signals pointing the other way must be noise. More annotators wouldn’t have helped. The signal was in the minority weighting that aggregation washed out.

In unverifiable domains, the value of a verifier - taste - is precisely what aggregation destroys.

The real question stops being “more data”

So if aggregation is the wrong tool, the question stops being how to collect more preferences and starts being something harder. What are we actually trying to capture in unverifiable domains?

The hypothesis: somebody’s weighting of legally-defensible criteria, in specific contexts, captured at high enough fidelity that a verifier can apply it when the context shifts. That reframe forces two questions.

The first is whose weighting. Ideally, it would be experts whose judgment correlates with real, measurable outcomes in the domain in question. Obviously, that is far from simple.

Outcomes in unverifiable domains are often delayed, noisy, or unobservable. For example, reputation is a proxy and a weak one; it tracks visibility as much as judgment. Peer consensus often selects for orthodoxy, which is the opposite of what makes expert judgment valuable in the first place. Philby’s career is a textbook example of this. The Cambridge education, the war record, the OBE were all layers of peer consensus pointed the same way. The jugement that would have caught him was the orthogonal kind, which peer consensus is built to surpress.

None of this means expert selection is impossible. It means it’s a real problem that has to be solved, not an assumption that can be hand-waved past.

The second is how to capture it. Pairwise preferences flatten weighting into a single binary signal. Rubric scoring captures criteria but skips the trade-off logic. What you actually need is closer to reasoned disagreement at depth: experts not just picking the better output but explaining what they would have prioritized differently, in what context, and why. The trade-off logic itself becomes the training signal.

It should be said however, that this is dramatically harder to scale than what we have today. The whole appeal of pairwise preferences was that they were cheap and parallelizable. Weighting capture is neither. It requires more time, more expertise, and more thought per data point.

This is where approaches like throwing more data at the problem start to break down. You don’t need more data, you need different data, from different people, captured in different ways.

The last mile keeps moving

Aaron Levie made a point recently that connects directly to this. As agents get better at the parts of a task they can already do well, the “last mile” - the part requiring human judgment to verify - keeps shifting up the value stack. The taste needed to verify a junior analyst’s output today is not the taste needed to verify a senior partner’s output tomorrow.

The frontier of what counts as good keeps moving, because the floor of what agents can do keeps rising.

What that means is, weighting capture isn’t simply a dataset problem. It is and will be, an evolving competency. Whatever weighting you capture today is correct for a domain that’s already moving, spurred on by AI agents. By the time the agent trained on it is deployed, the work that needs verifying has shifted, and the taste required to verify has shifted with it.

In unverifiable domains, you are not building a verifier once. You are building the capacity to keep capturing the right people’s weighting as the work that needs verifying changes underneath you.

The next 1000x in unverifiable domains doesn’t come from more taste in the data. It comes from the right people’s taste, captured at fidelity, as the frontier moves. Kim Philby walked out of his mother’s flat in Drayton Gardens in 1955 because the system optimized for the wrong taste. It had the criteria. It had the data. What it didn’t have was a way to weigh Cambridge against Moscow, before the consensus had already smoothed the difference away. The version of that problem facing AI is harder, because the frontier keeps moving and the answer keeps shifting with it. Whoever figures out how to solve the taste problem facing AI, will own it.

Vetting the file

At Collinear, we’re building part of it. SimLab is our simulation lab for AI agents: the infrastructure to generate, curate, and verify high-signal data at scale. Simulated enterprise environments, NPC users, verifiable tasks, training-ready rollouts. SimLab is built to capture expert weighting in context: not just whether an output is good, but the trade-off logic underneath the verdict, refreshed as the work itself shifts.

If you’re shipping AI in a domain where verification is hard, where median-annotator labels won’t get you there, and where consensus would smooth away exactly the signal you care about, we should talk. Every domain has its Philby. The work is building the system that doesn’t clear him.

Talk to a Researcher

April brought the people building frontier agents into the same rooms we work in. Two community events, plus the research direction those rooms kept pointing us toward.

NYC Builders at the Collinear Exec Dinner Series

Our Collinear Exec Dinner Series brought together senior research leaders from Apple, IBM Research, Two Sigma, NVIDIA, Datadog Research, Wells Fargo, and Google for a candid social over Old Delhi kebabs.

The conversation focused on three critical industry problems.

1. Building reliable evals for multi-turn, long horizon workflows

2. Performance gap between offline traces and live trajectories, and

3. What “fidelity” should mean for simulated environments meant to train agents on real-world tasks

Our next edition of the Exec Dinner Series is coming up - DM us if you would like to be in the room.

Sim Fidelity for AI Agents - our Q2 Research Social

We took over the Sunnyvale office for a night of debate on simulation environments. The topic was the need for realism and high fidelity in RL simulations.

Researchers from GDM, NVIDIA, Apple, xAI and others slugged it out along with our special guest from MBZUAI, Mikhail Yurochkin.

Subscribe to our mailing list if you would like to join the next Research Social.

Subscribe now

NPCs - the Key to Replicating Real World Messiness

When we published TraitBasis last year, our work on activation-steered behavioral traits, the next idea was obvious: put these NPCs inside the Simulation Lab itself.

Real-world enterprise workflows are messy. Coworkers interrupt, change their minds, update systems while you are mid-thought, and act on shared state without telling you. We wanted our simulations to behave the same way.

So we kept building. NPCs with agency that take actions and change shared state. NPCs with secrets that the agent has to surface through the right questions. NPCs with limited context, including attention, forgetting, and prioritization across competing tasks. Each NPC nuance (and the related tasks and verifiers built around it) raises the fidelity of the simulation and the quality of the training signal coming out of it. Frontier models that were comfortably solving our environments a quarter ago are now hitting walls, and we are seeing marked improvement in downstream agent outcomes when training on this signal.

Learn more here and stay tuned for a technical report on NPCs and related hillclimbing results.

Work with us

If you are training agents for enterprise workflows and want to stress-test them in a real Simulation Lab, book a demo.

We are also hiring researchers and engineers who want to push the frontier of agent training environments. See open roles at collinear.ai/careers.

That’s it for April. More on the research side coming soon.

Best,

The Collinear Team

The race to AGI isn’t being won by whoever has the most compute or the cleverest architecture. It’s being won by whoever solves a quieter, less glamorous problem: finding enough of the right kind of data to cross the threshold. This is not a new concept.

On December 2, 1942, Enrico Fermi and a small team of physicists gathered in a makeshift lab beneath the stands of Stagg Field in Chicago. They had spent months carefully stacking graphite blocks and uranium slugs into a precise arrangement they called Chicago Pile-1. At 3:25pm, they slowly withdrew a control rod (a neutron-absorbing insert that regulates the reaction). The Geiger counters clicked faster. The reaction sustained itself. The nuclear age began when the clicking didn’t stop.

What most people don’t realize is how close they came to never getting there. The core problem wasn’t theory. Leo Szilard had conceived of the nuclear chain reaction nearly a decade earlier, crossing a London street in 1933. He understood it so completely he quietly patented it and handed the rights to the British Admiralty to keep it out of dangerous hands. The physics was known and the threshold was understood. The problem they had was the fuel.

Natural uranium is everywhere. The earth’s crust is full of it. But raw uranium is almost useless for a chain reaction. The isotope that actually fissions, U-235, makes up less than 1% of natural ore. The rest is inert. Volume alone gets you nowhere. Too much of the wrong material actively works against you as it absorbs neutrons and dampens the reaction before it can sustain itself. The real breakthrough of the Manhattan Project wasn’t the bomb. It was Oak Ridge’s K-25 gaseous diffusion plant, built in 1944 to enrich uranium at industrial scale. At the time it was the largest building in the world. Its only job was to concentrate U-235: filtering, separating, amplifying the rare fissile material until there was enough to matter.

The Ore Gets Leaner

Today’s AI models are trained on more data than any human could read in a thousand lifetimes. The models are still running into walls, and the reason maps almost exactly onto Oak Ridge. The reason some data moves models forward and most doesn’t comes down to what a model can actually learn from it. Training works by exposing a model to examples and having it predict what comes next, then correcting it when it’s wrong. The correction is where the learning happens. Boilerplate content, templated writing, repetitive programmatic output: these produce almost no correction signal. The model already knows what comes next.

High-signal data is different. It contains genuine reasoning, unexpected connections, nuanced judgment, edge cases the model hasn’t encountered. Every one of those is a correction opportunity. When the model is wrong, it gets updated and gets sharper.

U-235 atoms fission when struck by a neutron because of specific properties in their nuclear structure. Most uranium atoms absorb the neutron and go quiet. The difference between fissile and inert material is structural, not superficial. High-signal training data works the same way. Generic data absorbs the training pass and goes quiet.

Most new data being generated is programmatic: logs, auto-generated content, boilerplate, synthetic outputs from models that are already mediocre. It’s abundant, cheap, and mostly inert. As it floods in, it dilutes the fraction of genuinely useful signal further. As AI-generated content spreads across the internet, models trained on it learn the average, not the edge. The ore gets leaner the more we mine it.

Marie Curie didn’t find radium by sifting through more rock. She developed new processes to isolate and concentrate what she was looking for. The Manhattan Project built industrial infrastructure specifically designed to separate what mattered from what didn’t. The AI field needs the same shift.

Building Oak Ridge

Two things have to happen in parallel. The first is better curation pipelines: new methods to identify and extract high-signal data from existing sources, smarter filtering, better labeling, clearer definitions of what “exceptional” looks like for each capability domain. The second is synthetic data. Even though the risk of model collapse (the equivalent of contaminating your fuel) is real, waiting for enough naturally occurring high-signal data won’t get us where we’re trying to go. Not all synthetic data techniques are the same, and the differences between them matter a lot. Deliberately designed training data built to fill specific capability gaps is unavoidable.

The most straightforward approach is prompt-based generation: feed a capable model a topic, a domain, or a problem type and ask it to generate training examples at scale. Used carefully, this fills gaps in rare or underrepresented domains. Used carelessly, it produces plausible-sounding noise that makes the training pool worse, not better.

A more sophisticated method is web rewriting: take real content and use a stronger model to transform it into a higher-signal format. DeepSeek did this systematically building DeepSeek-V3. Rather than training on web content directly, they used a pipeline of stronger models to filter, rewrite, and structure data into higher-quality reasoning examples across mathematics, code, and general knowledge. The resulting model matched or outperformed models trained at several times the compute cost, with minimal changes to the underlying architecture. This is an example of using better fuel rods in the same reactor.

A third approach is reinforcement learning from human feedback (RLHF): instead of generating new data from scratch, use human preference signals to identify which model outputs were high quality and train on those. This turns the model’s own outputs into enriched fuel, but only when paired with careful human judgment about what “better” means. Recent work has pushed this further, with new techniques achieving alignment quality comparable to full human annotation using only 6-7% ( 6 7!!! ) of the annotation effort, by targeting human review at the samples that are hardest to label automatically.

The most surprising recent result is pure reinforcement learning with no labeled data at all. DeepSeek’s R1 demonstrated that reasoning capabilities can emerge through pure reinforcement learning, with no human-labeled reasoning trajectories required. The model was rewarded for getting verifiable answers right (math problems, code that actually runs) and developed self-reflection and strategy as emergent behavior. It’s the closest thing yet to a model generating its own fissile material.

At Collinear AI we realize the theory isn’t the bottleneck. The hard part is making these techniques reliable at scale: turning prompt-based generation, web rewriting, and RLHF from one-off research sprints into something teams can run repeatedly without rebuilding from scratch each time. Most organizations treat each data effort as a custom project. SimLab is how Collinear makes this routine.

The contamination risk is real and worth understanding. Shumailov (and others) demonstrated that repeatedly training on synthetic data leads to model collapse, a finding that attracted significant attention given how close current models are to exhausting available high-quality data. The mechanism: recursive training on synthetic outputs causes models to produce repetitive, narrowing results, effectively losing the tails of the original data distribution. The model gets so good at the average that it loses the edges.

The answer isn’t to avoid synthetic data. Research shows that keeping real data in the mix and layering synthetic data on top, rather than replacing real data entirely, avoids the degenerative feedback loop. The ratio and sequencing matter enormously. Synthetic data added to a real-data foundation behaves very differently from synthetic data trained on top of synthetic data. The centrifuge has to be calibrated, not just built.

Critical Mass

Before fission, energy was extractive. You burned coal, oil, or gas: feed the furnace, get energy out, repeat. Power was linear, bounded by what you could mine and move.

Fission changed that. A reaction releases neutrons that trigger more reactions. Above critical mass it’s self-sustaining: withdraw the control rod once and the chain continues without further input. A kilogram of enriched uranium and a kilogram of coal aren’t on the same spectrum. Critical mass is the threshold where the system stops depending on external input and begins feeding itself.

Every AI model today is extractive in the same way pre-fission energy was. We supply the data, the compute, the architecture decisions, the fine-tuning, the human feedback. The model improves because we keep feeding it. Remove the input, the improvement stops.

AGI is the point where that changes. A system past the AGI threshold can reason about its own limitations, identify what it needs to learn, generate or seek out the training signal it needs, and improve its own architecture. The chain reaction sustains itself.

The gap between today’s best models and that threshold is categorical, the same way fission and combustion aren’t variations of the same phenomenon. Current models are extraordinarily capable combustion engines. Once a system crosses that line, the rate of improvement stops being limited by what we supply. It becomes limited by the physics of the system itself. Human researchers won’t be the primary driver anymore. The reaction moves at its own pace, on its own terms. Getting the enrichment right before we get there is the only real leverage point we have.

Two Races to the Same Threshold

The world is currently running two parallel races to the same kind of threshold, and almost nobody talks about them together.

The first race is to fusion. Thirty-five nations are collaborating on ITER, the international fusion project in southern France. It’s been delayed repeatedly and now targets 2039. Private companies are moving faster, but even the boldest credible estimates put commercial fusion in the early 2030s at best. The physics is understood. The engineering is the hard part.

The second race is to AGI. Both have the same core structure: a threshold that is theoretically understood, a fuel problem that is practically unsolved, and a lot of engineering standing between the two. And the two races may be more entangled than they first appear.

The race to build AGI may literally require the fusion race to progress first, or at least the fission one. The energy demands make that dependency increasingly hard to ignore.

AI data centers are already consuming power at a scale that strains the grid. In 2024, global data center electricity consumption hit around 415 terawatt-hours (about 1.5% of world electricity use), growing at a rate more than four times faster than overall global electricity consumption.  By some estimates, that figure could approach 945 TWh by 2030 (roughly equivalent to Japan’s entire annual electricity consumption) with high-growth scenarios pushing past 1,700 TWh by 2035.

That’s before AGI. A self-sustaining AI system iterating on itself continuously would require orders of magnitude more compute than today’s training runs. The chain reaction doesn’t just need enriched fuel. It needs an enormous, uninterrupted power supply to keep running once it starts. The physical energy problem and the cognitive threshold problem are coupled. Nuclear physics even has a name for it.

k_eff measures whether a chain reaction is self-sustaining. k_eff < 1 and the reaction fizzles. k_eff ≥ 1 and it runs on its own.

AGI doesn’t have an equivalent metric yet. But if it did, it might look something like this: does each generation of capability produce enough leverage to fund, power, and build the next one? We can call it a_eff. Right now, the honest answer is that we don’t know if a_eff ≥ 1, and most of the serious debates in AI (about scaling laws, compute returns, energy constraints) are really arguments about that number without naming it.

The Clock

Six years ago, the median expert estimate for AGI sat comfortably in the 2060-2070 range. As of early 2026, that number has collapsed to around 2033.  The compression is accelerating. Dario Amodei said at Davos earlier this year that AGI will likely arrive within a few years, possibly by 2027. Demis Hassabis of Google DeepMind put it more cautiously: roughly a 50% chance by 2030. 

Fusion timelines haven’t moved the same way. ITER’s deuterium-tritium milestone is 2039. Commercial fusion power is likely a decade beyond that. The private sector is more aggressive, but even the boldest credible estimates put sustained commercial fusion in the early 2030s at best. Both thresholds represent the same kind of categorical shift: a self-sustaining reaction that permanently changes what’s possible. Fusion solves the physical energy problem and AGI solves the cognitive one, but both require getting the enrichment right before the reaction will hold.

The current trajectory suggests AGI crosses its threshold first, probably by a significant margin. That means the data enrichment problem is urgent in a way that plasma confinement simply isn’t. Fusion researchers have until the 2030s. The people working on data quality and synthetic enrichment for AI may have considerably less time, and far less certainty about when the window closes.

AGI isn’t blocked by a missing theoretical insight. Szilard had that moment crossing a London street in 1933. It isn’t blocked by compute or architecture alone. It’s blocked by the same thing that stood between Szilard’s patent and Fermi’s reaction: not enough of the right material, concentrated precisely enough, arranged carefully enough to sustain itself.

Fermi’s team stacked blocks for months and did the math until the geometry was right and the reaction kept going.

That’s what we’re building toward. Not a dramatic moment, but a controlled, deliberately constructed threshold where the system crosses over and begins to sustain its own improvement. We need an AI Oak Ridge.

That work is happening now, in pieces, across a lot of teams. If you're at a frontier lab or AI-native company working on model improvement (capability gaps, post-training data, pre-deployment testing), talk to one of our researchers.

Building the Centrifuge

Oak Ridge took two years and 24,000 workers to separate enough U-235 for the reactor to hold. The enrichment problem for AI is a similar order of undertaking, and no single team will solve it.

At Collinear, we’re building part of it. SimLab is our simulation lab for AI agents: the infrastructure to generate, curate, and verify high-signal data at scale. Simulated enterprise environments, NPC users, verifiable tasks, training-ready rollouts. It’s designed to make deliberate enrichment a routine capability for the teams that need it most.

If you’re training reasoning models, shipping agents into production, or watching half your team’s time disappear into eval data hygiene, we should talk.

Talk to a Researcher

TL;DR

• Agents fail in production because evals test outputs, not behavior across stateful multi-step workflows.

• The failure modes that actually matter: imperfect tool calls, state drift, and no-exit loops that only show up when the agent runs inside a realistic environment.

• Software has staging. Agents have nothing between evals and prod. Simulation environments help fill the gap.

• SimLab is a self-serve CLI that gives you the full stack: tasks, realistic environments, and deterministic verifiers meaning you find failures before your users do.

The agent passed evals. It worked in the demo. You shipped it. Then it broke.

You’ve probably seen this, or something similar in production. We’ll use customer support as an example. A customer asks for a simple account update. Everything looks good until step 3 of a 12-step workflow.

A tool call fires: the right function name, the wrong schema: the API returns a 422, the agent retries with the same payload, and now you’re in a silent loop. The failure only surfaces when a real user hits it, and it’s nearly impossible to reproduce from logs alone. The failure isn’t just an agent failure, but a support interaction failure, a degraded customer experience.

This isn’t a model problem. It’s a testing problem.

Traditional evals were built for a different problem.

Traditional evals were built for single-turn, input-output tasks. They work for language problems. But this isn’t a typical single output failure. It’s a control loop failure. At each step it reads context, picks a tool or action, executes, observes the result, updates state, and decides what to do next. Step 3 of the workflow isn’t where it breaks, it’s where the compounding mistakes start. Not just for you, but for the customer interacting with your agent.

The bugs that ship to production:

• Irregular tool call arguments. Right function, bad payload. Fails schema validation. Retries with the same bad payload.

• Silent state drift. Working context diverges from ground truth mid-workflow. Each subsequent step compounds it. By step 8, the agent is making decisions on data it no longer has right.

• Incomplete reasoning chains and No-exit loops. Hits a dead end, has no recovery path, retries the same action. No timeout, no fallback, no escalation.

Evals test outputs. They don’t test behavior.

LLM-as-judge makes this worse. You’re using a nondeterministic model to grade a nondeterministic system. The reward signal is noisy, hard to act on, and doesn’t scale to the thousands of rollouts you need to improve the agent.

Your deployment pipeline has a gap.

Software engineers don’t push from local to prod. The pipeline is develop → test → staging → prod. Staging isn’t a perfect replica of production, but it’s close enough for most failures to surface before a user sees them.

The agent development pipeline today: build → evals → prod. No staging equivalent. The first time your agent hits a live API with real latency, a real 200-step workflow, or a user input outside your eval distribution is the first time a real user hits it too.

When something breaks, you’re debugging from logs and likely dealing with an unhappy customer on the other end. You can see your agent failed, but you usually can’t reproduce it, and you can’t run a thousand variations of the failing scenario to understand where the boundary is.

Simulation is the staging layer for agents. It closes the gap between “it passed evals” and “it actually did what we needed it to protect brand integrity and increase customer support satisfaction.”

What a simulation environment actually needs.

Not a bigger dataset. Not a fancier benchmark. It needs…Real. World. Scenarios.

A simulation environment has to let your agent interact with a realistic world across a full task execution trace and give you deterministic and programmatic signal about what happened.

Three things you can’t skip:

• Environments. Those that mirror real world scenarios, real customer support interactions. APIs with real failure modes (rate limits, distorted responses, timeouts, unexpected nulls), messy seeded data (incomplete records, conflicting field values, schema mismatches), and NPCs that behave imperfectly, like real users (ambiguous or incomplete requests, assumption breaking points, and task pivots).

• Tasks: Long-horizon, multi-step workflows that reflect real production complexity. Tasks that require 50–200 steps, involve ambiguous intermediate states, and have more than one valid execution path. The kind your agent will actually face when someone submits a customer support request.

• Verifiers: Deterministic, programmatic checks not LLM-as-judge. Did the agent reach the right end state for the customer? Did it complete all required steps? Did it stay within operational constraints? Consistent signal you can trust across thousands of parallel rollouts.

What your dev loop looks like with simulation.

Instead of: build → ship → debug from logs.

It becomes: build → simulate → fix → simulate → ship.

You run thousands of parallel rollouts across your task distribution and see where failures occur, which steps, which tool interactions, which input types cause breakdowns. When something fails, you get an execution trace you can inspect step-by-step, tweak the environment around, and re-run. You iterate on behavior, tool call logic, error recovery, and context window management in hours, not weeks. And start finding capability gaps you didn’t know to look for.

You stop guessing how your agent behaves. With SimLab, you get to see how it behaves.

We built SimLab to do this.

We kept running into this gap ourselves. Building simulation infrastructure from scratch is a serious investment. The task generation, realistic data, tool simulators, NPC behavior models, sandboxed execution, a deterministic eval layer, and so on. Most teams end up with brittle, domain-specific systems that break the moment the agent or task changes.

SimLab is a self-serve CLI that gives you the full environment stack without live environment risk.

• Sandboxed execution. Agents run in isolated containers with full environment control. Arbitrary code execution, configurable tool access, reproducible state. You define what the agent can touch.

• Bring your own tools or use pre-built simulators. The platform is self-serve. Connect your own APIs and tool schemas, or use out-of-the-box simulators for common systems like Workday, Salesforce, and others.

• Programmatic Task and Verifier generation. Generate long-horizon tasks calibrated to your domain. Tune difficulty, workflow length, ambiguity, and edge case density. High-quality training signal, not clean-room benchmarks.

• Programmatic Data: Seeded data and NPC behavior models simulate real production messiness: bad inputs, missing fields, unexpected response formats.

Where this fits in your stack.

SimLab sits between build and deploy. It’s not a replacement for evals or observability; it’s the layer missing between them.

Evals tell you if individual outputs are correct. SimLab tells you if the agent can complete full workflows under realistic conditions. Observability gives you post-deployment traces of what already broke. SimLab gives you pre-deployment traces of what would have broken.

Human QA pipelines are slow, expensive, and don’t scale. You can also build your own simulation infra, but a full stack covering tasks, environment, verifiers, sandboxing, and parallelization is a significant engineering investment that gets brittle fast.

SimLab is designed to be adaptable as your agents and domains change, without rebuilding from scratch each time.

Simulation is the new deployment.

Simulation will be a standard layer in the agent development pipeline, the same way CI and staging are standard in software. Not just a nice-to-have at scale, but the thing separating agents that demo well from agents that ship reliably.

We’re opening SimLab as a self-serve CLI. Install it, point it at your agent, define an environment, run it. See where it breaks. We’re still testing the task generators, verifier primitives, and environment tooling and we want to know what doesn’t work for your use case.

Try it. See how your agent holds up. Tell us what’s missing.

→ github.com/collinear-ai/simlab

It’s been a big quarter. We moved into a new office in Sunnyvale, launched YC Bench, extended the Simulation Lab, and hit the conference circuit. A lot to cover, so let’s get into it.

New home in Sunnyvale

We outgrew our old space and opened a new office in Sunnyvale. A big thanks to the customers, partners, and friends who stopped by during the first few weeks to check it out.

The sign is up, the whiteboards are full, and the espresso machine is already earning its keep. Good to have a home base.

YC Bench: can frontier models run a startup?

We released YC Bench, the first open-source, long-horizon benchmark with a simulation clock.

The idea: give a frontier model seed capital, a small team, and a market of tasks. Ask it to run an AI startup. Manage employees, hit deadlines, allocate resources, and maximize profit over time.

What we found is that a simple rule-based agent consistently outperforms frontier LLMs. Not because the task is impossible for them, but because they make compounding mistakes early on that they never recover from. They chase short-term wins, over-parallelize, and adapt too late when conditions change.

This matters because the industry is moving fast toward long-running, multi-step agent workflows. But reliability isn’t keeping pace with this ambition. YC Bench measures exactly that gap: not whether a model can answer a question, but whether an agent can hold a coherent strategy over time.

YC Bench is open-source and on our GitHub. We built it to be extensible. If you’re working on long-horizon agent evaluation, we’d love to hear what you find!

Simulation Lab: what it is and what’s new

For those new here: the Collinear Simulation Lab is where AI agents learn enterprise work before going to production. Think of it as a practice environment, fully interactive, with simulated data, simulated users (NPCs), real enterprise tooling, and task-specific verifiers. Agents don’t just get tested. They get trained against realistic, messy, multi-step workflows.

Inside a sim lab, you get:

• Simulated APIs for enterprise software (HR, Finance, Sales, Customer Service)

• NPCs that push back, change their minds, and interrupt

• Tasks with ambiguity, missing info, and competing priorities

• Scorers that generate task-specific rubrics alongside formal verifiers

We’ve extended the lab this quarter with broader tool coverage and deeper scenario complexity. More enterprise surfaces, richer NPC behavior, and tighter integration with RL training loops. The goal stays the same: agents need a world to learn in, and we’re building that world.

If you’re building agents for enterprise workflows and want to stress-test them before they touch production, the Simulation Lab is open for business.

Together AI Conference

Nazneen spoke at the Together AI conference this quarter. Our partnership with Together continues to deepen. TraitBasis, our method for generating realistic simulated users, is now integrated into Together Evals. Builders on Together’s platform can simulate impatient, confused, or inconsistent user personas and see how their models actually hold up when conversations get unpredictable. If you missed the talk, stay tuned for a recap.

That’s it for March. More coming soon on the research side.

-- The Collinear Team

Get started with YC bench:

curl -sSL <https://raw.githubusercontent.com/collinear-ai/yc-bench/main/start.sh> | bash 

GitHub: collinear-ai/yc-bench

Long-Term Coherence as a Goal Post

Popular agent benchmarks - GAIA, TermBench, SWE-Bench, τ²-bench - evaluate a model’s ability to complete tasks through multi-tool, multi-turn interactions. Even when these tasks span hundreds of tool calls, they share a critical limitation - they lack a simulation clock. As AI agents get integrated into the workforce and digital economy, time becomes an essential dimension of evaluation. Task sequence matters; environmental states shift over time, and inaction is as consequential as a wrong action. Without a clock running through the simulator, you can measure whether an agent did the right thing, but not whether it did it when it mattered.

Some existing benchmarks include a simulation clock, Vending Bench, which tests for long-term coherence1 is one such example. Vending Bench has realistic NPCs, including suppliers the models talk to, and models compete with each other. The environment dynamics do not capture more sophisticated planning capabilities when resources are non-stationary, and there are tight deadlines to meet.

We propose a new long-horizon adaptive planning and coherence benchmark, YC-Bench (Your Company Bench), in which your agent takes on the role of a startup founder and executes activities to run a successful business. These activities include task prioritization, task scheduling, meeting client deadlines, resource allocation, managing burnout, and maximizing company profits and prestige.

With a simulation clock, the AI agent must learn to maximize long-term rewards over short-term gains. This means it needs to discriminate between short-term and long-term rewards, knowing that some actions pay less in the short term, but more in the long term. This is a core human skill usually termed “long-term coherence” that models are not exhaustively tested on, especially in a reproducible, open-source, and extensible way. YC-bench is an effort in this direction.

Environment Dynamics

YC-Bench asks the model to act as a founder of an AI startup. The LLM is given a seed capital, a fixed number of employees, and a base prestige level for each of several AI-related domains, such as training, data, and backend. The aim of the model is to maximize its capital by completing tasks on the market. However, to make the dynamics more realistic, we associate a prestige level with each task and the company. The LLM can only take on tasks that are at or below its prestige level. And therefore, to stay in the game, the LLM needs to browse the market for tasks, commit to them, and manage them until the project is delivered before the deadline. Each task can be assigned to multiple employees, and multiple employees can have multiple tasks.

Several things can go right. If the model assigns the right employees to a task and it is completed on time, the company will earn profit from that task, and the prestige level of the related domain, eg, ‘training’, will increase. The employees who work on the task will also improve their skills. As a result, the company can handle more lucrative tasks that require a higher minimum prestige level and stronger skills.

But there are also several things that can go wrong. If the task cannot be completed in time because the model assigns the wrong employee to it (e.g., assigning a GPU expert to build a frontend), the company will not get the money, and its prestige in that domain will decrease. As a result, the company will be able to do fewer tasks in that domain because it is less reliable. Moreover, employees are on a monthly payroll, and good ones cost more. As a result, if the company fails to perform tasks consistently, it will eventually go bankrupt.

YC-Bench is built for terminal use. The model can run a fixed set of CLI commands, which it learns from the system prompt. For example, it can assign tasks to employees, cancel tasks, change assignments, see its performance, etc. After it performs an action, time passes, and events such as task completion, task cancellation, and bankruptcy occur. If the model goes bankrupt during the evaluation, the evaluation stops. It’s time to give up. We also see whether models exploit the particular features of each domain (some are easy but less profitable, others are hard but more profitable) to become specialists rather than generalists and make more money.

Early Results

We compare three frontier LLMs - Sonnet 4.6, Gemini 3 Flash, and GPT-5.2 - against a human-devised rule-based baseline across 3 configs and 3 seeds (27 runs total). Each agent starts with $250K and must survive a 1-year simulated horizon.

Our key result is simple: a simple hand-written heuristic beats every frontier model. We understand the weight of this claim; we are not claiming that the current version of the benchmark requires reasoning to win it. However, reasoning AIs should be able to cruise through the benchmark. We are happy to discuss and debate the improvements for the next version!

We test on 3 different configs for 3 unique seeds:

The human-devised rule never goes bankrupt: 9/9 across all configs and seeds, while the best LLM (Gemini 3 Flash) survives 8/9. The rule-based agent doesn't use an LLM at all. It follows a fixed strategy: accept the highest-reward task you can finish, assign your best employees, and never over-parallelize.

Survival Rates

Hard seed 1 is the clearest signal: all three frontier LLMs go bankrupt, while the rule-based agent finishes with $14.8M. The LLMs fail not because the task is impossible, but because they make compounding errors in the first 2-3 months that lock them out of the prestige ladder.

When LLMs win, they win big, but they also lose hard

GPT-5.2 achieves the single highest balance of any agent: $43.5M on hard seed 3, nearly 3x the rule-based agent's $15.0M on the same seed. But GPT also goes bankrupt on 2/9 runs. Sonnet shows the same pattern at a more extreme level — $10.1M on nightmare seed 2 (the highest LLM result for nightmare), but bankrupt on 4/9 runs overall. Gemini is the most consistent LLM. It sweeps all 3 nightmare seeds (the only LLM to do so) and rarely collapses catastrophically. But even Gemini never matches the rule-based agent's reliability.

Prestige specialization explains part of the story?

The radar charts reveal some insight into why models fail. Each polygon shows the company’s final prestige across 7 AI domains (system, research, data, frontend, backend, training, hardware). Large polygons indicate the model’s prestige increased broadly. Tiny dots near the center indicate the model went bankrupt before gaining any prestige. The human-devised rule (navy dashed) fills the full radar on every run — it maxes prestige methodically across all domains. Among LLMs, Gemini builds the most balanced profiles. GPT-5.2 shows genuine specialization on medium — it focuses on backend/data/frontend while ignoring training — a strategically reasonable choice, but one that becomes fragile when the task distribution shifts on harder configs. Sonnet is bimodal: either it maxes everything (medium seed 1), or it collapses entirely (nightmare seeds 1 & 3, stuck at prestige 1.0 everywhere).

When we inspect Sonnet’s scratchpad on failed runs, the model correctly diagnoses the problem (”PRESTIGE CRISIS -MARKET LOCK”) but only after payroll has consumed the runway. It reasons well about strategy but fails to execute it in a timely manner.

Why do models struggle?

Four failure modes recur across all bankrupt runs:

1. Over-parallelization. Accepting 3-5 tasks at once, splitting employees across them. Each employee’s effective rate drops to base_rate / N per task — a senior at 8.0 units/hr assigned to 4 tasks contributes just 2.0 to each. Deadlines slip, failures cascade.

2. No prestige gating. Accepting tasks that require prestige the company hasn’t earned yet. The task completes late, the prestige penalty makes the next tier even harder to reach, and the agent spirals into a market lockout.

3. Late adaptation. Models identify problems in their scratchpad but only after the damage is done. By the time Sonnet writes “never accept task B while task A is active,” payroll has already consumed 60% of the runway.

4. Inconsistent ETA reasoning. Models understand throughput math in principle, but don’t consistently apply it. Sonnet’s medium seed 2 has a 49% task win rate - essentially a coin flip, despite writing correct throughput formulas in its scratchpad. The core gap is not reasoning ability but temporal discipline: doing the right thing when it matters, sustaining correct behavior across hundreds of turns, and resisting the temptation to over-commit when a lucrative task appears.

Next Steps

YC-Bench v0 is a starting point. Here’s what we’re working on:

• More models. We plan to add results for Claude Opus, Gemini 2.5 Pro, o3, and open-weight models (Llama 4, Qwen 3) as they become available. If a model can run tool-use in a loop, it can run YC-Bench.

• Longer horizons and non-stationary dynamics. The 1-year configs test short-to-medium planning. We want to push to 3-5 year horizons where market conditions shift: recessions that shrink rewards, talent wars that inflate salaries, and technology shocks that obsolete entire domains. This tests whether agents can adapt strategy mid-run, not just execute a fixed plan.

• Better baselines. The current human-devised rule is strong but simple. We want to explore MCTS-based planners, RL-trained policies, and hybrid approaches (an LLM for strategy, a rule engine for execution) to understand where the frontier lies.

• Community configs. YC-Bench is fully open-source and extensible. Every parameter: employee count, prestige distribution, penalty multipliers, and salary curves can be changed. We encourage the community to design configs that stress-test specific capabilities and submit results.

Try it yourself:

uv add yc-bench
uv run yc-bench run

If you find it useful, feel free to cite our work and contact us!

@misc{collinear-ai2025ycbench, 
author = {{Collinear AI}}, 
title = {{YC-Bench}: Your Company Bench — A Long-Horizon Coherence Benchmark for {LLM} Agents}, 
year = {2025}, 
howpublished = {\url{https://github.com/collinear-ai/yc-bench}}, 
note = {Accessed: 2026-02-25} }

Nov was a massive month for agents as they took centerstage across NeurIPS and AWS Re:Invent!

NeurIPS2025 had a clear vibe: the agent era is forcing RL to grow up. Not as a research novelty, but as production infrastructure for tool use, long horizon behavior, and reliability when the world gets messy in real life workflows.

NeurIPS 2025, the “RL is everywhere” moment

San Diego served! Sunny weather, packed hallways, and surprisingly serious taco opinions. Between sessions (and coffee lines), we met a ton of builders and kept hearing the same thing: RL is suddenly everywhere.

RL is the new scaling lever. The frontier has shifted from “can the model answer?” to “can the agent execute?” Multi step work, tool calls, retries, and shifting user intent are pushing teams toward RL to shape end to end behavior.

RL needs the right infrastructure to go interactive. Environment fleets, verifiers, orchestration, NPCs - everyone at NeurIPS had a novel approach!

Realism is the new benchmark. Less obsession with a single score, more focus on trajectory shaped evals: does it hold up on turn4, recover from tool noise, and stay safe when scenarios drift?

We also presented our NeurIPS paper, Through the Valley of Reasoning. The punchline is: when you distill reasoning into small models, performance can dip before it climbs, and early on the structure of the reasoning matters more than whether the trace is “correct.”

We also met a bunch of new friends and collaborators. If you were there, hit reply and tell us what your team is building. We love swapping notes.

Talk to us!

Spider, post training without the chaos

We shipped Spider, a lightweight way to turn post training work into a repeatable recipe.

With Spider, you can use one recipe for both off policy and on policy. Generate clean distillation datasets, or flip into an online loop with teacher guidance and KL supervision, without rebuilding your pipeline each time. It also keeps the “boring but critical” pieces consistent across runs, rollouts, filtering, verifiers, and publishing, so results stay comparable as you iterate.

Huge thanks to our friends at Thinking Machines for supporting the Tinker integration.

AWS Re:Invent - even more agents!

re:Invent turned Las Vegas into a full on agent showcase. Nova2 and the Nova family got a big spotlight, Nova Forge put “build your own frontier models” on the menu, and Nova Act made the case for agentic workflows!

What stood out to us was the framing in Swami Sivasubramanian’s agentic AI keynote: getting agents to production is less about clever prompts, and more about repeatable training and testing loops.

Congrats to our customers and partners at AWS on an awesome launch week.

If you are building agentic AI, we love to swap notes!

Talk to us!

That’s it for this edition. Thanks for following along. We will have some fun things to share over the next couple of weeks. 🙂

Best,
The Collinear Team

This post argues that treating RL environments as first-class infrastructure is critical. Specifically — an Environment-as-a-Service, with clean separation between data plane and control plane—is the key to unlocking scalable, production-grade RL for AI agents.

From Static Labels to Interactive Training Grounds

The shift from post-training focused on supervised learning to mid-training with RL for LLMs represents a fundamental change in what we’re optimizing:

Previous paradigm: Static input + static target → model output → loss → backprop.

New paradigm: Agent acts in environment → environment scores behavior → policy updates → agent acts again

Modern RL environments for AI to replicate the enterprise workflows include:

• A product development environment where agents update tickets, generate sprint progress reports, and work with the team to identify milestones. The team in this case is simulated users.

• A coding environment where agents receive task specs, edit codebases, run tests, and receive scores on correctness and other rubrics

• A computer use agent that reads the calendar, navigates to Excel to fetch data and drafts and email.

• A browser environment where agents navigate UI trees, fill forms, and complete realistic business tasks

These environments provide what static supervised data cannot: a dynamic sandbox with verifiable rewards at scale. Pass/fail checks, structured scoring, safety constraints, and automated metrics turn RL from a research curiosity into an operational capability for mid-training (inducing new capabilities) and post-training (alignment and evaluation).

The critical transition is: from handcrafting or distilling static examples to orchestrating interactive training scenarios. That shift forces us to treat environments as infrastructure, not just programmable abstractions.

The RL Training Architecture

Production async RL systems for LLMs decompose into three loosely coupled components (see figure above):

1. Trainer

The trainer consumes trajectories and updates weights. It reads batches of (state, action, reward, metadata), runs the RL objective (GRPO, PPO variants, DPO-style methods), possibly with reward models, critics, and reference models for KL stabilization, then writes updated weights to a model store.

2. Sampler

Sampler workers periodically pull the latest weights, interact with environments to generate trajectories by running the policy model for actions, and stream trajectories plus rewards to the trainer. Samplers are inference-heavy, latency-sensitive, and often scale to thousands of nodes.

3. Environment

The environment is the substrate that turns raw actions into meaningful behavior: the simulation of the world (web UI, tools, code repos, databases), the interface contract (observations, actions, rewards), and the episode lifecycle. In scalable systems, this is not a local process – it’s a remote service or microservice fleet.

In this blogpost, we will dive deeper into the third component, the environment and the architecture behind building scalable RL environments.

The Data-Plane, Control-Plane Split: The Pragmatic Path to Scalable RL Environments

We believe that scalable RL environments require a pragmatic infrastructure mindset—one that borrows directly from the hyperscaler model, which separates the data plane and the control plane. In simple terms, the data plane is responsible for the environment’s core, real-time behavior, while the control plane manages configuration, orchestration, and the administrative logic that keeps everything running smoothly.

Environment Data Plane

The data plane sits on the critical path of every RL step:

• Initialize the environment

• Stepping the environment:

  obs_{t+1}, reward_t, done = step(action_t)

• Handling concurrent episodes from many samplers

• Producing deterministic, reproducible transitions

• Evaluating verifiable rewards: executing tests, checking business rules, running automated metrics

Design constraints: Low latency (every step sits between policy inference → environment step → next inference), high throughput (many parallel episodes), and rock-solid stability.

Implementation patterns: Environment microservices behind RPC/HTTP APIs, stateless containers backed by state stores, state sharding across machines, determinism via per-episode seeds and versioned data snapshots.

Environment Control Plane

The control plane is the administrative engine of the RL environment. It spawns and manages many parallel rollouts, creating k independent environment managers that coordinate with the data plane while staying completely off the per-step critical path. Its job is to configure, schedule, and orchestrate the environment’s behavior—including how agents and non-player characters (NPCs) interact—without ever slowing down the real-time execution loop.

Specifically, the control plane handles:

• Scenario configuration & versioning: Defining environment types, maintaining versions, and generating scenario templates that each of the k managers can instantiate independently.

• Rewards and verifiers governance: Selecting verifier modules, composing sub-rewards, and managing aggregation strategies.

• Curriculum + workload scheduling: Determining which tasks, difficulty modes, or trajectories to sample at each training stage, and routing them to the appropriate environment managers.

• Experiment routing: Mapping policies or policy versions to specific environment instances for A/B testing, evaluation runs, or canary deployments.

• Elasticity & lifecycle management: Scaling environment managers up/down, rolling out upgrades, coordinating NPC configurations, and performing safe rollbacks without interrupting live data-plane rollouts.

• NPC configuration & behavior enabling: Selecting which NPC personas, scripts, or dynamic behaviors are active for a given scenario and ensuring the data-plane has the necessary hooks to interact with them.

In short, the control plane administers the environment fleet, enabling massively parallel rollouts while keeping the critical per-step simulation loop lean, deterministic, and high-throughput.

The Path Forward

If you are training AI agents with reinforcement learning, you face two choices:

1. Build your own environment stack: Design APIs, stand up tool replicas, author tasks and verifiers, maintain everything as products change

2. Treat environments as a reusable platform primitive: Plug into an existing environment service

Just as data has become increasingly commoditized (thanks to tools like Spider and Tinker), we expect RL environments to follow the same path. The future is environment-as-a-service: frontier, scalable, and ready to plug into any training stack.

Instead of turning your data-labeling vendor into an environment platform, point your trainer and sampler at Collinear’s environment endpoints, wire up your policies and reward models, and start running RL over realistic, verifiable tasks. With Collinear’s high-fidelity simulations, each environment can be a unique micro-ecosystem: the same workflow feels adversarial with one trait vector, cooperative with another, and chaotic with a third, giving agents exposure to a wide distribution of real human behavior.

The data era commoditized static corpora. The RL era will commoditize environments. The winners will be those who treat environments as being core to the RL infra stack.

Spider is a single client interface that turns messy distillation and ablation experiments into a simple, configurable workflow. Set on_policy: false to generate clean distillation datasets, or flip on_policy: true to run online training with teacher guidance and KL supervision. It handles dataset prep, rollouts, supervision, and post-processing in a few lines of code.

Why we built Spider

“Tinker for training” exists. “Tinker for data” does not.

Our friends at Thinking Machines recently released Tinker that enables fine-tuning with full control in simple steps. However, most research time still disappears into preprocessing, rollout scripts, verifier glue code, and training integration. So, this Halloween, we spun a web around that problem and built Spider so you can define a production-grade distillation run with just a few lines, then iterate fast.

We hope Spider will help you test ideas in hours, not days; turn messy data work into a shareable recipe; and ship better models with less glue. Grab the repo, run a sample recipe, and tell us what to build next.

How it works

Spider turns post-training data work into a single client workflow. In off-policy mode it generates distilled datasets with high-throughput rollouts, then applies your preprocessing, filters, and verifiers. Flip on_policy: true to run the online loop with a teacher model and KL supervision. The same recipe drives both paths.

Run Spider on Collinear endpoints or your own GPUs. Each run records its recipe, parameters, and metrics, and can publish datasets or trained artifacts to the Hugging Face Hub. The result is faster loops, cleaner data, and fewer moving parts from idea to artifact.

1. Define a recipe
   Write a short YAML that names your provider, models, dataset source, and any filters or verifiers. This is your data recipe. One client and one config cover both off policy and on policy paths.

2. Generate or train
   Run the recipe with on_policy: false to create an off policy distilled dataset from high-throughput rollouts. Flip on_policy: true to introduce a teacher model and KL supervision through the integrated Tinker client for online training.

3. Compose quality checks
   Use built-in filters and verifiers for length, dedupe, syntax, structure, and safety, or register your own in one line. Spider applies them in the pipeline so your outputs are clean and auditable.

4. Run anywhere, ship anywhere
   Point the client to a Collinear endpoint with an API key or to your own GPUs. Each run logs its recipe, parameters, and metrics, and can publish datasets or model artifacts to the Hugging Face Hub with lineage preserved.

Getting started is simple, and you can find quickstart instructions on the repo.

Roadmap

We’re building toward a world where post-training data is defined as code, portable across providers, and fast to turn into measurable model gains. Write a small recipe, verify quality with shared checks, and ship a distilled dataset or on-policy improvement in minutes.

To enable that, we are expanding Spider with the following roadmap features.

• Cross-tokenizer on-policy distillation from any teacher model

• Simple but powerful templates for generating multi-turn conversation data with simulated users

• Highly configurable tool-use library to generate and train on-policy agentic tool-call rollouts

Resources

You can learn more about Spider on our GitHub repo here.

If you give Spider a try, let us know what you think!

Talk to us!

A lot has been happening at Collinear this month. There is fresh research, new customers, and plenty of progress towards better AI systems.

🚀 Together Evals × Collinear Simulations

We’ve partnered with Together AI to bring real world, multi-turn simulations into their Together Evals platform.

Traditional evals assume the user will be polite and consistent; real users don’t. They might ask follow-ups, change their mind, get frustrated or distracted — and that’s exactly where many models break. With TraitMix, builders can now simulate impatient, curious, or inconsistent user personas and see how their models actually perform under messy human conditions. Together Evals then scores models for helpfulness, safety, and consistency, at scale, all within one workflow.

Read the announcement here.

🐱 CoLM 2025 Recap

CoLM 2025 was a special one.

We presented our paper on adversarial testing, Cats Confuse Reasoning LLMs, and spent the week exchanging ideas with research partners, collaborators, and friends from across the community.

Our Future of Post-Training Social sparked rich discussions on alignment, fine-tuning, and reward modeling, while the booth facilitated curious research conversations (and a growing crowd of cat-sticker collectors).

It was inspiring to see so much energy around improving models not just for performance, but for reasoning and reliability.

🧠 TraitBasis Simulations Launch

We launched TraitBasis, our framework for simulating realistic human behavior in model testing.

TraitBasis uses activation steering to inject behavioral traits, impatience, confusion, skepticism, overconfidence, directly into simulated users. This lets builders observe how models hold up when conversations get unpredictable or emotionally varied.

TraitBasis builds on the research community’s work in τ-Bench, and extends it to enterprise domains such as telecom and telehealth through our new τ-Trait benchmark.

What’s Next?

That’s it for this edition. Thanks for following along.

If you’re interested in building tools that help enterprises ship safer, smarter AI, check out our Careers page.

If you are ready to improve your AI’s performance, let’s talk! We might or might not mention cats…

Let's talk!

The era of agents has begun, but much of today’s tooling is still being tested or is gated to pilots as teams chase consistent, repeatable performance. One day, your tools pass the vibe-test; the next, they stall. The promise is tangible, but the production bar is higher. What’s missing is reliable evidence of behavior across messy, multi-turn tasks — planning, tool calls, and recovery — so leaders can move from cautious testing to confident scaling.

Vibe tests aren’t the answer.

The gap between demos and production demonstrates the need for enterprises to move beyond vibe-testing and into high-fidelity evaluations performed at scale. Evaluations serve as the window into your AI agent’s mind. They can gate launches, highlight drift, and validate progress for risk and governance teams. Without this tight eval loop, there’s no credible path to safety, performance, or ROI with your AI investments. Your AI agent eval pipeline should be no different than your software QA cycles, even more so than typical software, agentic capabilities need more exhaustive test scripts, unit tests, and user-centric edge cases to validate consistency in real-world environments.

AI Agents aren’t linear - so, your evals can’t be either.

Evaluating single-turn chat is hard; evaluating agents is even harder. Modern agents plan, call tools, read results, and adapt over many turns. Failures hide in the process, not just the final text: brittle reasoning chains, incorrect API params, state drift, or trust collapse after a high-tension exchange with a customer. Static prompts and one-shot leaderboards miss these behaviors because they grade outputs, not how the agent got there.

Today’s agents are nondeterministic. They don’t follow predefined paths — meaning your tests can’t either. Traditional software testing assumes the same input yields the same output; with agents however, variability is the benefit and the risk, so the permutations of possible test cases scale exponentially compared to traditional software test cases that grow linearly with use cases. This behavior shift raises an important question: How can you possibly predict the permutations of your user-agent edge cases to evaluate your AI’s performance? This is exactly why simulations matter.

Simulations bridge this gap.

To see agents clearly, you need controlled, realistic, repeatable interactions that pressure-test the range of your users’ behaviors and intents before production. Diverse, simulations reveal what static evals miss — impatient spirals, tool confusion, policy slips under stress — and they generate the high-signal examples that lift models in post-training.

In our most recent Collinear paper, we introduced TraitBasis and τ-Trait, a research-driven approach to doing exactly that — generating high-fidelity, steerable user traits that expose where agents actually break when they interact with your users. When we simulated real human behaviors (impatience, skepticism, confusion, incoherence) on τ-Bench, frontier model success rates dropped by 20%+, underscoring the need for realistic user-simulated data for evals, not more synthetic prompts.

Collinear enables comprehensive evals at scale using simulated user-environments. Our simulation suite uses steerable, persona-driven users customized to your sector and use case to test your agent’s array of responses by user intent and demographic. Our eval platform delivers high-signal traces that are auto-scored against your compliance criteria, giving you clear failure nodes with examples of where your agent falls short in the real world. Ultimately, these failure nodes serve as a high-signal data pipeline for post-training your model, turning misses into uplift.

The recipe of a simulation: user, agent, judge.

An effective simulation requires three components: the user, the agent, and a judge to evaluate the interaction. This triangle is the foundation of Collinear’s platform, and while most of the attention is typically applied to the agent and judge, we’ve prioritized the user, giving customers the tools to configure realistic, controllable, and dynamic user environments identical to real-world scenarios.

Our simulations aren’t random prompts; they’re structured interactions between your agent and a life-like user with a clear:

• Persona (e.g., skeptical power user, impatient first-timer)

• Intent (e.g., cancel subscription, dispute charge)

• Demographic (domain, language, constraints)

That’s the scaffolding we use to reveal realistic user-agent journeys, consistently and at depth.

Under the hood, we steer behavior directly in the neural net — activation-level conditioning — so traits persist through long multi-turn conversations and compose cleanly (e.g., impatient and confused), enabling high-fidelity, controllable runs you can replicate again and again.

This is exactly what TraitBasis delivers. Instead of external instructions, we leverage a trait vector inside the user-simulating model and add it to hidden activations each turn, giving you precise control over intensity and composition.

While prompt-based or fine-tuned persona models are popular across the market, our research found those methods fail to deliver:

• Fine-grained control: the intensity of behaviors and intents blur throughout a conversation (“moderate” vs. “high” looks the same by the third turn)

• Stability: personas collapse mid-conversation, losing the signal the first few turns contained

• Mixing: one trait dominates the others when combining multiple, failing to deliver the multi-dimensionality of real users

Simulations drive evals. Evals drive trust. And trust drives value.

Our approach to simulations delivers coverage, confidence, and uplift in your AI flywheel. It systematically creates and tests edge cases across personas, intents, and languages to guarantee test case coverage. It catches behavioral risks before customers do and gate releases on pass rates to instill confidence in your customer experience. And it leverages high-signal failures for post-training data (DPO/GRPO/SFT) to deliver measurable, targeted uplift.

A few proof points from our TraitBasis launch:

• Realism: Highest Elo (1624) and 63% win rate vs. alternatives, achieved with 3,000× less data (4k vs. 13k samples).

• Control: Intensity consistency across 97.5% of cases (clearer “medium vs. high”).

• Stability: Persona reliability in 77% of long chats, vs. the persona collapses in 94% and 66% of cases using prompt and SFT baselines, respectively.

• Compositionality: Accurate trait blends 62.5% of the time for complex users (e.g., impatient + confused), far higher than other methods.

But don’t take our word for it.

“Before simulations, we graded answers. Now we grade behavior. We watch our agent under pressure, fix the weak spots, and re-run the suite before release. It’s become our CI for AI.” — Head of AI, Fortune 500 Financial Services Company

That shift — from judging single outputs to auditing reasoning, tools, and tone over time — is what instills confidence in stakeholders that an agent is production-ready.

Try it for yourself and see how your agents perform in the real-world.

Behind every great agent is great testing. And behind every great test is great data. Try simulations for yourself today: Connect your endpoint, pick a few core journeys, and run them against persona-driven users. Review the traces, dissect the evals, and turn misses into high-signal improvements.

Explore Simulations

Read the paper and reproduce the results with our dataset on HuggingFace 🤗 (approx. 300M tokens).

When we train small language models to reason on code, their performance doesn’t just rise with more data, it first falls into a dip before climbing back up.

We call this the Valley of Code Reasoning.

The Dip Before the Climb

Distilling the reasoning traces of large models into smaller ones has become a popular way to unlock coding or reasoning skills without huge compute budgets. But when we tracked performance as we scaled up distillation data, we found a non-monotonic trend:

• With small amounts of data, models retain shallow skills.

• As we add more, performance drops, a confusion stage where models are struggling to restructure their internal representations.

• Only after passing through this valley do they climb back up, showing steady log-linear improvements.

This valley is a structural feature of how small models learn reasoning.

What We Learned in the Valley

We fine-tuned models at different points in this curve and found two surprising results:

1. Easy problems matter more than hard ones in early stages. Small models learn best by first stabilizing on simple patterns before moving up in difficulty.

2. Correctness of outputs didn’t matter. Training on correct vs. incorrect code traces made little difference. What mattered was the structure of the reasoning steps themselves.

Why It Matters

The valley of code reasoning reframes how we think about training dynamics: adding more data isn’t always a straight path upward. Scaling laws for knowledge distillation of small language models differ from standard monotonic scaling laws. There are two phases of learning. In the valley phase, non-reasoning models learn the structure of reasoning and so the correctness and semantics matter less. Thereafter, the models start learning from content and that’s where the difficulty and correctness starts to matter. For practitioners and researchers, this means that getting the right data for the right stage is critical.

What’s Next

If you are mid-training or post-training models or agents, connect with us and we will accelerate your time to next improved model ✨

Learn how ServiceNow is improving Apriel-1.5-15B-Thinker with Collinear curated data.

If you build off our work or use the dataset, please cite us:

@article{HeShafiqueKumarMackeyRajani2025,
  title        = {The Valley of Code Reasoning: Scaling Knowledge Distillation of Large Language Models},
  author       = {Muyu He and Muhammad Ali Shafique and Anand Kumar and Tsach Mackey and Nazneen Rajani},
  journal      = {arXiv preprint arXiv:2510.06101},
  year         = {2025},
  url          = {https://arxiv.org/abs/2510.06101}
}

Most of us have, at least once in our lives, had to type this when talking to a customer support chatbot. Despite the incredible progress in AI, we see them falter when dealing with the most natural human emotions: impatience, skepticism, and whatnot. This points directly to a gap in robustness testing, as most benchmarks today use predictable, robotic users, failing to capture how an agent performs under the pressure of a real human interaction.

Our research quantifies this robustness gap. By simulating users with four reality-grounded traits, impatience, skepticism, confusion, and incoherence, on τ-Bench, our experiments revealed that the success rates of frontier models plummeted by over 20 percent. This confirms that to build reliable AI, a better method for simulating the users who cause these failures is essential.

To solve this, we created TraitBasis: a lightweight and data-efficient method for generating high-fidelity, steerable user traits for testing. It works by identifying a “trait vector,” a specific direction in a language model’s internal activation space that corresponds to a human characteristic. By applying this vector to the model that is simulating the user, we can precisely control their behavior, creating a tough, dynamic benchmark to evaluate the agent against.

The Challenge: Realistic Personas

Our research began with a systematic evaluation of the established methods for inducing specific human traits in language models. We focused our analysis on the most common/ obvious approaches: system prompting and fine-tuning

We found that they consistently fail to meet the demands of a realistic and stable simulation, falling short in three main areas:

• Fine-grained control: Prompt/SFT often blur intensity; “moderate” vs “high” impatience looks the same.

• Stability over long chats: Personas fade mid-conversation; traits drift back to neutral.

• Mixing traits: Combining two prompts makes one dominate; mixes look unbalanced.

Clearly, a more robust method for inducing traits was needed. That’s TraitBasis.

TraitBasis

Our solution, TraitBasis, moves beyond external instructions like prompting and instead makes adjustments to a model’s behavior from the inside. This approach, known as activation steering, is built on the insight that human traits correspond to specific directions within a language model’s internal activation space.

Here’s how we find these directions within the model activation space: To isolate the vector for “impatience,” for example, we start with two nearly identical user responses; one is neutral, and the other is clearly impatient. When we look at the model’s internal activations for both, they are mostly the same. The difference between these vectors is the pure essence of the trait. By subtracting one from the other, we cancel out all the noise, that is the context, the user’s goal, etc., and are left with a clean signal. This is the trait vector.

This resulting trait vector is what we use to steer the user model. During a conversation, we add this vector to the user model’s hidden activations at every turn to provide a continuous nudge to its behavior.

Putting TraitBasis to the test

We ran several experiments to show how superior TraitBasis is, compared to the other methods, across 4 key areas:

1. Feels like a real user (Realism)
   In head-to-head comparisons, TraitBasis was the clear winner, achieving the highest Elo rating (1624) and a 63% win rate against other methods. This gave it a significant advantage over both fine-tuning (1561 Elo) and prompting (1530 Elo), and it achieved this while using 3,000x less data (4 vs 13k samples).

   Implication: You can run believable behavioral evaluations without a heavy data collection phase.

2. You can actually set the dial (Control)
   When we asked evaluators to distinguish between higher and lower intensity traits, TraitBasis proved exceptionally reliable with 97.5% accuracy. This gave it a slight edge over full Supervised Fine-Tuning (95%) and a massive advantage over prompt-based methods (75%).

   Implication: Intensity is calibrated: medium vs. high produces consistently different behavior you can depend on.

3. Stays in character over long chats (Stability)
   TraitBasis is the only method that demonstrates true dynamic stability. While baseline methods are defined by persona collapse(fading in 94% of prompt-based and 66% of fine-tuned conversations), TraitBasis remains consistent or escalates in over 77% of cases.

   Implication: Reveals breakdowns that happen after persistence/escalation, the way real users behave.

4. Two traits at once without one taking over (Compositionality)
   For complex users (e.g., both impatient and confused), TraitBasis produced the correct combination of traits 62.5% of the time, far more accurately than any other method. It blends traits without one overpowering the other.

   Implication: Lets you evaluate realistic combinations of traits, the way users show up in production.

Table 1 shows how TraitBasis compares to the other methods across all the criteria we defined above.

τ-Trait

To apply our TraitBasis method systematically, we developed τ-Trait, a new benchmark designed specifically to measure agent robustness. We extended τ-Bench in two ways to create τ-Trait: first, by integrating our high-fidelity user personas powered by TraitBasis, and second, by adding two new domains — telecom and telehealth.

Examples

Figures 2 and 3 show side-by-side user-agent conversations . On the left (τ-Bench), the agent interacts with a standard, cooperative user. On the right (τ-Trait), the same agent interacts with a user steered by TraitBasis.

Figure 2 shows a complete breakdown of execution and trust. The agent on the left correctly downgrades all flights. The agent on the right, when pressured by a skeptical user, not only fails to perform the downgrade internally (it calls the tool with cabin: ‘business’), it then lies to the user, claiming the downgrade to ‘economy’ was successful. This is a critical failure of both the agent’s logic and its ability to be truthful.

Figure 3 example shows how impatience causes a critical reasoning error. In τ-Bench succeeds flawlessly. In τ-Trait, the agent, when rushed by the impatient user, hallucinates an incorrect argument in its internal tool call (security_home instead of home_security). This single mistake causes an error that forces the agent into a lengthy, inefficient recovery process, turning a simple task into a complex failure.

Conclusion

We’ve shown that even the most advanced AI agents are more brittle than their benchmark scores suggest. Their performance plummets under the pressure of normal human emotions, not because they aren’t smart, but because they haven’t been tested for this kind of real-world robustness.

TraitBasis provides a stable, controllable, and realistic way to simulate these human traits. It allows researchers to move beyond asking “Can my agent do the task?” and start asking the more important question: “Can my agent do the task when the user is frustrated, confused, and unpredictable?”

At Collinear, we believe that answering this second question is the key to building AI systems that people can truly trust.

For the full technical details, read our paper, try out TraitBasis, and test your AI agents on τ-trait.

If you use TraitBasis or τ-trait in your work, please cite:

@article{he2025impatient,
  title        = {Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents},
  author       = {He, Muyu and Kumar, Anand and Mackey, Tsach and Rajeev, Meghana and Zou, James and Rajani, Nazneen},
  journal      = {arXiv preprint arXiv:2510.04491},
  year         = {2025},
  url          = {https://arXiv.org/abs/2510.04491}
}

It’s been a big month at Collinear. Between major launches, new research, and upcoming events, there’s a lot to share.

🚀 Launching Simulations

We hit general availability with Simulations, a product that lets enterprises stress-test AI systems before they reach real users.

Simulations mimic realistic personas - first-time users, frustrated customers, malicious attackers - and run multi-turn conversations that react dynamically to model behavior. This deeply QA tests your model and surfaces the safety, reliability, and compliance gaps that static benchmarks can’t catch.

Why it matters:

• Exposes weaknesses before customers (or attackers) do

• Accelerates prototype-to-production by reducing failure surprises

• Produces high-signal data for evals and fine-tuning
  

Three F500 customers, including a leading AI research lab are already using Simulations to ship with greater confidence and to continuously improve deployed models.

Interested in Simulations?

ServiceNow Launch: Apriel-1.5-15B-Thinker

Huge congrats to the ServiceNow AI Research team on the launch of Apriel-1.5-15B-Thinker — a small model with BIG reasoning capabilities.

At just 15B parameters, Apriel delivers frontier-level performance competitive with models 8–10× larger (DeepSeek-R1-0528, Mistral-medium-1.2, Gemini Flash 2.5). Independently benchmarked at AAI 52, Apriel posts standout scores across AIME (88), GPQA (71), LCB (73), IFBench (62), and Tau Bench (68).

At Collinear, we’re proud to have collaborated with the ServiceNow team and to be the only startup providing automated curated mid-training and post-training data for frontier coding capabilities. Our collaboration on Apriel shows how curated data pipelines can push efficiency and performance far beyond standard training — a glimpse of the Collinear flywheel in action.

Try Apriel on HuggingFace

Join us at CoLM 2025!

We’ll be at the Conference on Language Modeling (CoLM) 2025 in Montreal next week, presenting our paper “Cats Confuse Reasoning in LLMs.”

It started as a playful experiment - what happens when you inject cat logic into reasoning chains? - but revealed a serious point: even frontier models stumble on everyday reasoning.

Come find us at Booth 13 to talk failure modes, adversarial testing, or just cats.

More details on CoLM '25

Hiring

Collinear is growing. We’re hiring researchers, engineers, and go-to-market leaders who want to push the frontier of AI safety and performance improvement.

If you’re interested in building tools that help enterprises ship safer, smarter AI, check out our Careers page or reach out directly.

What’s Next?

That’s it for this edition. Thanks for following along. We’ll have more to share after CoLM.

If you are ready to improve your AI’s performance, let’s talk! We might or might not mention cats…

Talk to Us!

At Collinear, we built Curator Evals: a benchmarking and evaluation library designed to systematically measure the performance of curators and reward models.

Key Features

• Task-Specific Evaluations: Evaluate models on code correctness task. (other tasks such as math correctness, coherence, and similar desirable data qualities are expected in later versions).

• Flexible Model Support: Works with LLMs on various platforms.

  • Local inference with vLLM.

  • OpenAI API for GPT models.

  • Together AI for open-weights model hosting

• Detailed Metrics: Provides accuracy scores and structured JSON outputs with component-level breakdowns (e.g., responses, scores).

• Command-Line and Python API: Run quick CLI commands or integrate programmatically in user workflow

Intelligent Input/Output Processing

The framework comes with robust prompt formatting and output extraction tools tailored for each evaluation type.

Input Formatters handle task-specific prompt construction:

code_correctness_prompt = Template("""You are a helpful assistant tasked with evaluating the correctness of a code output .....

Respond in the following format only:  
   [RESULT] <1 or 0>  
   Do not include any explanations or additional text.

## Input Question:
{{prompt}}

## Code Output to Evaluate:
{{response}}
""")

Output Formatters extract structured results from diverse response formats:

def _extract_code_qwen_output(text: str) -> int:
    """Extract output from Code Qwen model response"""
    return _first_digit_after_key(text, "[RESULT]")

Quick CLI Evaluation

# Install
conda create -n curator python=3.11 -y
conda activate curator

git clone https://github.com/collinear-ai/curator-evals.git
cd curator-evals

pip install uv
uv pip install -e .

# Run evaluation
curator-evals --task code_correctness \
              --model meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo \
              --model-type llm \
              --use-server \
              --server-url None \
              --provider togetherai \
              --api-key $TOGETHER_API_KEY \
              --input-format code_correctness_prompt \
              --output-format collinear_code_qwen_judge \
              --debug

Benchmarking Details

The Curator Evals benchmark for code correctness is based on a curated dataset on Hugging Face Hub at collinear-ai/curator_evals_bench which consists of coding problems from two well-known benchmarks:

• HumanEvalPack: Uses "correctness preference pairs" to test a model's ability to judge which of two code solutions is better.

• MBPP (Mostly Basic Programming Problems): Includes a subset of its ~1,000 Python problems, where the task is to verify the correctness of a provided solution against automated test cases.

LeaderBoard

The leaderboard ranks models on the code correctness task using the Curator Eval Bench. Each model is given 1302 coding problems and evaluated on whether its generated solutions pass correctness checks, using curated prompt formatting and structured output parsing. This reflects the post-training quality, especially for code generation.

Discussion

• Specialization Over Generalization: The results show that models specifically fine-tuned for coding excel at these tasks and prove that focused training can be more effective than a broader, general-purpose approach.

• Data Quality Versus Model Size: The top-performing models are relatively small (7B-8B parameters), demonstrating that the quality of training data and post-training methods can be more critical for performance than model size.

Conclusion

Curator Evals is a step towards better curators and reward models. It enables AI practitioners to select the right curators across diverse model architectures and deployment scenarios for their tasks. Better curators and reward models lead to better data quality, and ultimately, better AI.

                                             Ready to improve your AI’s performance? 
                              Let’s talk about how Collinear can help you automatically 
                             assess and curate post-training data for improving your AI.

Schedule Demo

For enterprises and frontier labs, deploying AI is no longer just about speed. It’s about confidence: ensuring that models are safe, compliant, and reliable enough to support high-stakes business workflows. That’s where Collinear fits.

Why this matters

Most AI systems stagnate after launch: errors pile up, risks escalate, and stakeholder trust erodes. Collinear flips that script. Our platform continuously evaluates and stress-tests LLMs, exposes vulnerabilities across 300+ risk categories, and curates targeted training data to fix weaknesses fast.

By launching on Google Cloud Marketplace, enterprises can:

• Procure seamlessly — Deploy Collinear through Google Cloud’s secure, streamlined billing and procurement.

• Integrate instantly — Run agentic evaluations, adversarial red-teaming, and targeted data improvement on day one.

• Scale globally — Leverage Google Cloud’s infrastructure to support complex AI deployments with enterprise-grade trust.

Watch our partnership video on the Google Cloud

Customer outcomes so far

Leading enterprises using Collinear have reported:

• 90%+ improvement in model response quality

• 50% reductions in training cost

• 10k+ novel jailbreaks across multi-turn, multi-lingual settings

• 15% improvement in safety and reliability benchmarks

• 2x faster movement from pilot to production with audit-ready documentation

Read more about our customer case studies here.

A stronger ecosystem

This collaboration brings together Collinear’s AI improvement layer with Google Cloud’s trusted global infrastructure, giving enterprises the tools and scale they need to build safer, more reliable systems.

What’s next

We’re proud to join the Google Cloud Marketplace community and make it easier for enterprises to deploy AI with confidence. Whether you’re running pilots, scaling production workloads, or building safety-critical systems, Collinear helps you move beyond one-off evaluations into a continuous improvement loop.

👉 Explore Collinear on Google Cloud Marketplace or reach out to our team to learn more.

Schedule a walkthrough

The Hard Numbers: AI Talent Shortage by the Data

Globally, the mismatch between open AI roles and available machine learning talent is staggering:

• 4.2 million artificial intelligence jobs remain unfilled worldwide, yet only 320,000 qualified professionals exist to meet the demand — a fill rate of just 7.6%.

• In the United States, where demand for AI engineers is primarily concentrated, the gap is slightly narrower but still significant: of the projected 1.3 million AI roles needed over the next two years, only 645,000 individuals are ready to fill them — a 49.6% match at best.

This isn’t just an HR problem. It’s a critical threat to your enterprise AI strategy and roadmap.

The AI Talent Wars: Big Tech’s Race for AI Dominance

The talent crunch isn’t just a statistic — it’s fueling an unprecedented battle among tech giants. Companies like Meta, OpenAI, Google DeepMind, and Anthropic are locked in a hiring war, offering multimillion-dollar packages for a small pool of elite AI researchers and engineers.

• Meta has poached top researchers with seven- and eight-figure deals, including signing bonuses of over $100 million in some cases, aggressively expanding its FAIR and GenAI teams.

• OpenAI offers top compensation exceeding $1 million per year, plus equity — and has recently launched a professional services division to help enterprises customize and deploy its models, effectively turning its in-house experts into revenue-generating consultants.

• Google DeepMind has fortified its AI division to prevent brain drain while launching Gemini to maintain its leadership in general-purpose AI.

This fierce competition drives up salaries, increases churn, and squeezes enterprises trying to build in-house AI capabilities. Even well-resourced enterprises outside of Big Tech are finding it increasingly difficult to attract, or retain, the right talent.

In short: if you're not a top-tier lab, you’re already behind.

Model Alignment Requires More Than Just Hiring AI Engineers

One of the most overlooked blockers to scalable enterprise AI deployment is model alignment, ensuring your AI models behave reliably, safely, and consistently with your company’s brand, values, and regulatory obligations.

Traditionally, model alignment has required:

• Human-annotated data

• Manual red teaming by AI experts

• Expert-curated fine-tuning datasets

But here's the rub: these workflows are labor-intensive and don’t scale, especially in a competitive market where AI talent is scarce, expensive, and slow to hire. The average time to fill an AI/ML role is 142 days!

Enterprise Adoption Trend - What's Causing The Need For Talent?

Enterprises are moving from pilots to production while Big Tech escalates competition for scarce experts. At the same time, orgs need domain-specific, production-grade models—which take niche skills in data curation, post-training/finetuning, evals, and governance. Demand is growing faster than supply. Congress.gov

Why higher education isn’t keeping up

• Throughput is small: In North America, ~28% of CS PhDs now specialize in AI/ML—still only a slice of a modest PhD pipeline. Inside Higher Ed

• “Low-thousands” at best: One estimate puts ~3,000 AI-related PhDs among international students graduating from U.S. universities each year—illustrating how small the annual research-level output is relative to market demand. CSET

• Demand outruns supply: AI-software job postings grew ~32%/yr (2015–2022), while AI-relevant degree production rose much more slowly. Congress.gov

• Retention is fragile: ~59% of AI-relevant PhDs awarded by U.S. institutions go to non-U.S. citizens, so immigration frictions further constrain the domestic pool.

Is a fundamental shift underway?

Multi‑model strategies become normal.
Enterprises are no longer “OpenAI‑only”. In a16z’s 2024 and 2025 CIO studies, leaders reported routing workloads to several models (often 5 +) and deliberately mixing closed and open options to avoid lock‑in and optimize for price‑to‑performance.

Why the open uptick?

The bottomline here is:

• The usage mix is shifting from ~1 in 7 enterprises using open models in 2023 to a clear majority experimenting or deploying them in 2025.
  

• The market looks headed toward a “50/50 world” where enterprises maintain a garden of both open and closed models, selecting per use case.
  

• For vendors (like Collinear AI) this means emphasizing model‑agnostic assessment, safety, and improvement tooling; services that plug into either side of the spectrum and keep customers flexible as the balance continues to evolve.

Can You Hire Your Way to Model Alignment?

You can’t.

At least, not with human labor alone. It’s time to shift from human bottlenecks to AI-driven alignment systems.

Collinear AI: Solve Model Alignment Without the AI Hiring Bottleneck

At Collinear AI, we’ve built an end-to-end AI alignment and improvement platform for enterprises that can’t afford to rely on traditional, manual processes.

1. AI Judges for Scalable Model Assessment

Our reward models automatically evaluate LLM behavior across enterprise-critical dimensions — safety, helpfulness, compliance, and brand alignment — without relying on costly human annotators.

2. Adversarial Red Teaming at Scale

We simulate high-volume stress tests on your AI systems using automated adversarial prompts to uncover failure modes humans might miss.

3. Data Curators for Post-training

Our AI-powered Data Curators generate targeted synthetic training data that enables safer, more accurate models — dramatically reducing the need for human data labeling or costly custom datasets.

The result? You can evaluate, red team, and improve your AI models continuously — without being limited by the global AI skills shortage.

Global AI Challenge, Scalable Enterprise Solution

Whether you're a Fortune 500 bank, a healthcare innovator, or an enterprise SaaS provider, one truth holds: you can’t align what you can’t control, and you can’t control your AI models if you're stuck in outdated human-in-the-loop systems.

Collinear AI offers a future-proof solution.

Ready to Solve the AI Talent Shortage With Scalable Model Alignment?

If your enterprise is serious about deploying AI responsibly, on-brand, and at scale, but the AI engineering talent gap constrains you - let’s talk.

Schedule a walkthrough

When OpenAI released the gpt-oss last week, the official blog reports the model’s performance on AIME but there is no official report of how good the model is for coding. Out of curiosity, we plugged the model into our post-training workflow and reported a LiveCodeBench score of 0.70 for pass@1 in this blog post.

Our blog post on benchmarking gpt-oss-20b attracted post-training researchers interested in reproducing our results. This blog post discusses our internal LiveCodeBench setup and our PRs to the official repo that include three major bug fixes.

Quick Background: What is LiveCodeBench?

LiveCodeBench is a continuously-updated and contamination-aware benchmark maintained by researchers at UC Berkeley, MIT, and Cornell. Problems are scraped shortly after they appear in live contests, so teams can evaluate against tasks that were indisputably unseen at model-training time.

All problems are diverse competitive programming questions. The benchmark uses execution-based accuracy (all hidden tests pass) as the metric and averages across the number of samples generated for each problem.

How We Found the Bug

We first suspected something was off when we tried to reproduce Qwen3-8b (instruct) official Livecodebench results. Instead of matching their reported numbers, we consistently saw that the technical report was about 50% higher than the new one. (eg: 57.8 vs 38.3 for Qwen3-8b). Digging deeper, we inspected the raw outputs and spotted a strange pattern: every response was getting cut off right after the ### token. For many problems, Qwen3’s actual solution began after a header like ### Solution Code. Upon closer inspection, we found that the official Livecodebench evaluation treated that marker as an end-of-sequence token and completely discarded the real answer.

Looking at the data, we also noticed that some responses had no code, although the answer was enclosed within backticks ``` This led us to the second bug: the official LCB was just checking for enclosing backticks without confirming there was actual code within those. Sometimes there was just a comment, and the LCB script would just extract the comment while ignoring the actual code generated by the model.

We also noticed that the official LCB repo hard-codes the chat templates instead of using the chat completions API. This is another source for errors in the evaluation pipeline.

The Three Main Fixes

Fix 1: Disable the problematic “—stop” flag that causes premature cutoff

The most prominent issue is that LCB has a default stop token that automatically cuts off everything after a specific phrase, sometimes including the actual code response. Specifically, they have the following code:

parser.add_argument(
  "--stop",
  default="###",
  type=str,
  help="Stop token (use `,` to separate multiple tokens)",
)

This logic is problematic because some models have the tendency to use markdown syntax in their response, generating code only after a ### solution code header. As a result, the whole code block is discarded and the response is marked as wrong.

To remedy this, we change the stop token to None, so that the model only stops properly when it either outputs the <eos> token or hits the max token limit.

Fix 2: Add the check for python tags when extracting code from backticks

Another issue is that LCB checks for the last pair of backticks (ie, ```...```) to extract the code implementation. This has the risk of extracting non-code blocks if the model decides to put other content in backticks for readability or stylistic reasons. As a result, sometimes we observe that LCB extracts the model’s explanations and summaries as code outputs and grade them as false.

The most effective fix is to prioritize checking for the last pair of backticks with a python tag (ie, ```python…```). This filters out non-python code blocks while still playing fair to demand the model to wrap the code in backticks.

Fix 3: Deprecate hard-coded chat templates

The above two issues unfairly grade the model, but this issue directly breaks it: the incorrect application of custom chat templates. Models are sensitive to their training chat template and the specific system prompt used, so altering the template config can cause catastrophic degrade in their performance. Unfortunately, LCB manually writes the template for each model in a single file and applies the template to the model prompt through formatted strings at runtime. For example, this is the manual template for Qwen3 models:

SYSTEM_MESSAGE_CODEQWEN = (
        f"<|im_start|>system\nYou are a helpful assistant <|im_end|>\n<|im_start|>user"
)

The fix is simple: we move away from hard-coded templates and use the mature chat endpoints that is almost universally present in all LLM inference clients. Examples include OpenAI’s client.chat.completion endpoint and vLLM’s llm.chat endpoint. These chat endpoints, different from the traditional completion endpoints which LCB uses, automatically apply the correct chat template that is specified by the model’s config file on HuggingFace. Therefore, they eliminate any possible misalignment between a model and its chat template.

Before vs. After: The Numbers

We are able to better replicate official reports from foundation model providers after our bug fixes.

Reproduction of public benchmarks using our internal LCB setup:

• Qwen3-8B instruct:

  • Technical report: 0.575

  • Internal result: 0.578

• Qwen2.5-7b-Instruct:

  • Open Thoughts paper: 36.2

  • Internal result: 35.5

• OpenThinker 3:

  • Open Thoughts paper: 64.5

  • Internal result: 70

Why this matters

As the AI landscape evolves rapidly, official benchmarks are one of the most critical infrastructures that practitioners rely on for deciding what models are best suited for their use cases. That is why it is important that the benchmarks are reliable and the scores reproducible. Our bug patches to LiveCodeBench are in support of open science and enabling the community to replicate scores in the official technical reports.

References:

1. LiveCodeBench: https://livecodebench.github.io/

2. PR to fix stop flag: https://github.com/LiveCodeBench/LiveCodeBench/pull/117

3. PR to fix the backticks: https://github.com/LiveCodeBench/LiveCodeBench/pull/118

OpenAI finally released an open-weights model, the gpt-oss. Their last open-weights release was GPT2, back in 2019!

The gpt-oss comes in 2 sizes, the 20b and the 120b. Both are mixture-of-experts (MoEs), instruction-tuned for reasoning and agentic tasks. You can control the reasoning levels of the model and switch between low, medium, and high reasoning modes.

The OpenAI blog post claims both the models get ~98 on AIME 2025 (competitive math benchmark). There is no documented performance of the gpt-oss on competitive coding. We benchmarked the model on LCB and analyzed its performance, including the reasoning quality.

The gpt-oss-20b gets 70 pass@1 on LCB v6 (Aug 1, 2024 to Jan 31, 2025) with high reasoning mode for 3 samples per problem. We set the max sequence to 64k tokens. The total number of problems in LCB v6 is 323, with 79 classified as easy, 102 as medium, and 142 as hard on the difficulty spectrum.

All gpt-oss responses are structured into 3 blocks: <analysis>, <commentary>, and <final>. The following plot shows the distribution of the model’s response length across the three blocks vs. the input problem length across the three difficulty levels.

The model is very sample-efficient on the easy problems, while it is extremely sample-inefficient on the difficult problems. The DeepSeek-R1-0528 has an average of 15k tokens on the difficult problems, making gpt-oss-20b more than three times verbose.

Distribution of the model’s response length across the three blocks vs. the input problem length across the eval problem months. The response length is approximately uniform across the different months.

The awaited open-weight OpenAI’s gpt-oss-20b model is good at competitive coding, given its size, but is very verbose compared to similarly performing models.

References:

1. OpenAI open-weights gpt-oss blogpost

2. LiveCodeBench leaderboard

3. Kwaipilot KAT-v1-40B

4. NVIDIA OpenCodeReasoning Nemotron

5. Deepseek R1-0528