We load a benchmark into the room: its data into the warehouse, its context spread across Slack and Linear, the way it ends up in a real company.
An analytics agent is only as good as the context it can find, and the context is never in one place. The definition it needs was settled in a thread, the caveat lives in a ticket, and the table it is querying says nothing about either.
Testing that honestly means standing up a company: a warehouse, the tools around it, and knowledge scattered across them the way people actually leave it. That is weeks of work before you learn anything about your agent.
That is what the data room is: an off-the-shelf virtual company, already filled with a benchmark's data and its context. One URL, one token, running in an afternoon.
One company, filled differently for each benchmark we run in it.
We do not take a benchmark as it comes. Its data goes into the warehouse, but the knowledge its questions turn on — a definition, a rule, a threshold — we write ourselves, and then split deliberately across the tools. No single place holds the whole picture, so an agent has to reconstruct it the way it would at work.
These are the tools in the virtual company today.
More are on the way — a dbt project, a Git host, a wiki — each added the same way: the real API shape, so your agent reaches it with the client it already has.
One host, one bearer token, plain HTTP. Your agent runs on your side and never leaves your infrastructure.
The service is stateless. Every call that crosses the boundary is appended to a log as one JSON line, which is how we can tell you which documents your agent opened and which it walked past. Nothing is stored in a database, and we never see inside your product.
We started with DABstep.
Multi-step questions that cannot be answered from the tables alone: they need a fee rule, a field definition, or a threshold that only exists in prose. That dependency on outside knowledge is why it was the right one to disperse first. The demo runs five questions, in both arms.
More are coming. The next public set is queued, and we are writing our own from interviews with senior analysts, so that the hardest questions in the room are ones no model has been trained on.
/run/tasks.
Answers are not included.Access is by invitation during the beta. You get a private page with your token, your endpoints, and an instruction file to hand to your agent.