Skip to content
BrainlessLab BrainlessLab BrainlessLab

Create a reproducible profile

A profile describes one composition across declared trials. Use it when one in-memory run is not enough and you need a portable result.

Validate the plan

The repository includes a small executable example:

Terminal window
julia --project=. bin/brainlesslab.jl check \
  plans/examples/profile_tracking.toml

check parses, validates, and resolves the plan without simulation. It rejects a scored target when horizon - warmup is below the task’s minimum_scored_ticks. It should report a valid profile plan and its resolved target.

Run the profile

Terminal window
julia -t auto --project=. bin/brainlesslab.jl run \
  plans/examples/profile_tracking.toml --root records

The command prints the new record path. Open report/index.html for a readable summary. Use the CSV files for analysis.

See the output shape

The two generated fixtures below use the same null_random Wall target, two blocks, and root seed. The profile and benchmark therefore differ by operation, not by input data.

Example profile record

This dummy fixture illustrates the record format. It establishes nothing about performance. The values below are read from its machine-readable summary during the site build.

Condition Blocks Trials Raw mean Normalised mean Anchor censoring
example_wall 2 2 0.361 0.000 2/2 at floor; 0/2 at ceiling

summary/statistics.csv contains 0 rows because statistics and contrasts are benchmark artifacts. Profile results remain visible in data/trials.csv and summary/summary.json.

Example benchmark record

This dummy fixture illustrates the record format. It establishes nothing about performance. The values below are read from its machine-readable summary during the site build.

Task Condition Blocks Raw mean [95% CI] Normalised mean [95% CI] Anchor censoring
wall example_wall 2 0.361 [0.219, 0.503] 0.000 [0.000, 0.000] 2/2 at floor; 0/2 at ceiling

summary/statistics.csv contains 1 row. The record contains 0 contrasts because this anchor-only benchmark declares no comparator.

The profile leaves summary/statistics.csv empty. The benchmark fills it because statistics are a benchmark artifact. Both keep the floor-censored null result visible so the example also shows how task anchors appear in a record.

Check the record

A successful record contains DONE, not FAILED. Begin with:

  • request.toml for the submitted plan;
  • resolved.toml for the values that ran;
  • seeds.csv for realised random streams;
  • data/trials.csv for independent trial rows;
  • record.toml for provenance and checksums.

The example uses one trial and Tracking’s minimum scored interval to keep the run small. Its output is exploratory software evidence, not a confirmed scientific result.

Next, compare conditions under paired blocks.

Source: plans/examples/profile_tracking.toml, tools/docs/example_profile.toml, tools/docs/example_benchmark.toml, src/operations/Profile.jl, src/records/Records.jl.