How a session runs
Five steps, one of which is the interview. The first four exist to make the fifth specific.
Session lifecycle
Invite
You pick an assignment and we mint a single-use link. Nothing is provisioned and nothing is billed until the candidate opens it. An invite can only be created against an assignment that has built successfully — nobody gets sent a link into an environment that will not boot.Open the brief
The link lands on the assignment README, the rules and the time limit, with a Start button. That page load is what fires provisioning: a machine is created, the repo is cloned, the IDE boots, and a session-scoped gateway key is minted with a spend cap and your allowed model set. The candidate reads for twenty to forty seconds, which is exactly the window the environment needs — cold start hides behind something they wanted to do anyway. The clock starts on Start, not on the page load.Work
The candidate edits, runs tests in a real terminal, and prompts or interrupts the agent. Both halves stream into the event log as they happen, with an on-disk outbox behind them so a dropped connection becomes a reconnect rather than a hole in the recording. The gateway enforces the spend cap independently of our application logic, and the candidate is warned before they reach it.Submit
The candidate submits, or the clock runs out. We snapshot the tree, revoke the session’s model key, seal the event log against further writes so the record cannot be edited after the fact, and destroy the machine.Debrief — where the grading actually happens
You scrub the replay and flag moments; the output is a set of timestamped “explain this” questions, which is the interview. You pick the moments. Export the whole thing as an immutable report for the archive. Everything before this step exists to make this step specific.
Reading a session
Tool approvals are auto-allowed, because confirmation modals every ninety seconds are not how anyone actually works. That costs us the explicit accept/reject record, so review behaviour has to be followed through the timeline instead. Four proxies make it followable, and each one doubles as a debrief prompt.
| Proxy | Derived from | The question it raises |
|---|---|---|
| Post-write read latency | Gap between an agent write and the next editor.open on that path | Did they open what the agent wrote, and how long did they stay? |
| Hand-edit ratio | editor.edit lines ÷ agent-written lines | How much of the agent’s output did they change themselves? |
| Interrupt latency | agent.interrupt minus the start of the turn | Did they stop a turn, and how far into it? |
| Verification independence | terminal.run commands the agent did not suggest | Did they run anything the agent had not proposed? |
None of these is a score. There is no composite number, no pass mark and no ranking. They are a navigation layer: they point at the six minutes of a ninety-minute session worth watching, and the timeline shows exactly what happened there.
The model is selectable. The harness is not.
You choose the allowed model set; the candidate chooses within it, and the choice is recorded alongside what it cost. Which model they reached for, and for which task, is there in the timeline — as an opening question for the debrief if it matters to you.
The agent loop, though, is the same for everybody. If one company’s candidates ran their own tooling and another’s ran ours, the sessions would not be comparable — and comparability across candidates is the entire product. A trace has to read as evidence about the person, not about which tool they happened to be handed.
Try it on one real loop
Ctillion is early. It is a working system rather than a finished product: one assignment shipped as a built-in, customer-authored assignments built from an uploaded Dockerfile, and no pricing page — because there is no pricing yet.
Accounts are issued by hand, one at a time, so every one comes with a conversation. Tell us what you hire for and what your current take-home fails to tell you.