Research stages and evidence

A research result has a question, a protocol and an evidence boundary. The active Plan preserves that question, a version, named targets, an operation and an evidence state. These fields make work inspectable; they do not confer scientific acceptance.

conformance -> calibration -> exploration -> tuning/training -> variance pilot
    -> frozen protocol -> sealed confirmation -> robustness -> promoted evidence

Use each stage for its own decision

Conformance checks whether the equations, state transitions and records implement their declared contracts. Backend qualification checks a specific arithmetic/device path. Calibration measures the declared task null under a matched protocol.

Exploration and tuning identify promising development settings. A variance pilot estimates the independent sample needed for a later comparison. Freezing fixes the question, conditions, settings, exclusions and seed policy before confirmation.

The Python development runner accepts planned, exploratory, tuned, frozen and retired states. It rejects confirmed and promoted requests. A program cannot grant those states merely by completing a run.

Validate without examining sealed outcomes

uv run brainlesslab check python/plans/delayed-cue.toml

This command checks the requested protocol without simulation. Use source, tests and development worlds for implementation and debugging. Do not inspect sealed evaluation data to choose parameters, diagnose a surprising result or repair a comparison.

Keep calibration, development, training/selection, variance-pilot, confirmation and robustness streams disjoint. Pair declared worlds within a causal comparison, and retain distinct mechanism streams where its design requires them.

Match the claim to the control

A reference random-action policy estimates one task floor. Blind input asks about access to observations. Shuffled input changes the correspondence between channels and wiring. Weight and full-plasticity freezes distinguish declared adaptation paths. None is a universal null for every claim.

A native score describes one task opportunity. An offline decoder describes recoverable labelled information under a particular estimator and whole-trial split. Neither alone establishes cognition, biological fidelity, a task-controlled learning mechanism or external validity.

Use independent randomised blocks or trials as inferential units. Ticks, nodes, event points and agents within a world do not add replicates. Retain unsuccessful and unavailable results rather than selecting only representative runs.

Preserve historical evidence

Accepted historical records, benchmark tables and experiment bundles keep their original source revision and status. Query research/catalogue.json before examining accepted public contributions. A maintainer replay is a linked reproduction check, not another independent sample.

The rewrite does not rerun or promote these studies. Historical ExperimentSpec tools belong to the earlier implementation; see version history. They are not a second active Python plan schema.

Continue with comparison controls, records and limits.