Experiments¶
An experiment is the durable record created when a plan launches. It preserves the difference between what was intended and what actually happened: what ran, what changed, which decisions were made, and what evidence the work produced.
You do not need to watch an experiment continuously for that record to remain useful. Experiments can test a hypothesis, build data, analyze evidence, produce code, or create a report. They all appear as experiments in the thread Timeline because the useful distinction is their purpose, not an internal execution type.
One experiment, one interpretable condition¶
An experiment should represent a piece of work you may later want to compare, stop, rerun, or cite on its own. If a changed checkpoint, dataset, split, control, or intervention would alter the meaning of the result, a new experiment usually gives you a cleaner record.
That does not mean every small implementation retry needs a new experiment. Fixing a transient job failure or correcting execution while preserving the same plan belongs in the current run. Changing what the evidence will support does not.
What an experiment contains¶
An experiment can contain:
- the approved plan
- chronological progress, decisions, and deviations
- code and files created or changed during the work
- compute jobs and usage
- connected training or evaluation runs, such as Weights & Biases runs
- one or more reports
- critic review and the planner's interpretation in the parent thread
Views appear only when relevant: Reports after one exists, Compute after usage, and Metrics after a compatible run is linked.
Lifecycle and status¶
The experiment's worker executes the plan and records meaningful progress in Timeline. It can continue independently and pauses when it needs a decision or encounters a blocker it cannot safely resolve.
Queued and Running are active. Needs decision and Blocked need attention. Completed and Stopped are inactive. Read the latest Timeline entry for detail.
Stopping does not erase the work. The plan, code, decisions, and partial evidence remain available. A stopped experiment can still contain a valid baseline, a useful diagnostic, or evidence about why the original method failed.
Experiment views¶
The views are ordered around the questions most researchers ask:
| View | Use it for |
|---|---|
| Reports | The durable answers and outputs. This view appears once the worker publishes a result. |
| Timeline | The live landing view while work is running and the chronological record afterward. |
| Plan | The approved intent, inputs, controls, outputs, and resource assumptions. |
| Worker | Detailed execution logs and direct steering when ordinary thread-level guidance is not enough. |
| Compute | Jobs, accelerators, runtime, and compute consumption. |
| Metrics | Linked training or evaluation measurements when available. |
| Code | Files added, changed, or deleted by the experiment. |
You will rarely need every view at once. Start with Timeline while work is running and Reports when it is complete. Open the others to answer a specific question about intent, implementation, resources, or measurement. Some views appear only when the required data exists.
For most follow-up questions, return to the thread and talk with its planner. Use Worker when active execution needs an immediate correction.
Evidence and provenance¶
A report is designed to communicate the result, not replace the full evidentiary record. Important claims should still trace back to the plan, Timeline, code, data, metrics, and decisions.
When something looks surprising, confirm the model checkpoint, data identity and split, metric definition, sample size, controls, and source file before treating the report as current evidence. The most expensive errors are often plausible results from the wrong input rather than obvious crashes.
Experiments within a thread¶
One thread can contain many experiments. Keeping the runs separate preserves their individual conditions, while Summary and Timeline let you interpret them together.
This is where synthesis happens. Experiment 1 may establish the baseline, Experiment 2 may reveal that it depends on a confound, and Experiment 3 may test the corrected method. None of those records needs to be rewritten for the thread to arrive at a better conclusion.