bijou eval report
generated: 2026-08-16T09:15:41+00:00
data: /home/ubuntu/datasets/fontaine/grasp_sft_demos_v0
selection: 1 datasets, 313 episodes, 54101 frames (0 dropped)
samples: 256 frames, seed 0
episodes: all
checkpoint: /home/ubuntu/checkpoints/finetune/fontaine_grasp_sft_joint_corrected/step_002000
smolvla: -
sampler: euler-10
molmo-norm: checkpoint
target time: t (standard)
noise key: stable
state: intact
fps filter: all
camera-count filter: all
generate: (fast path)
subgoal mode: none (planner-less)
sample plan: none (uniform frame sampling)
Chunk metrics (raw action units, pad-masked)
| policy | chunk_mae | p50 | p90 | first_mae | chunk_mse | ms/frame |
|---|
| state-copy | 9.356 | 4.279 | 23.273 | 6.610 | 557.3 | 0 |
| state-copy-norm | 9.868 | 6.154 | 18.722 | 7.399 | 517.1 | 0 |
| bijou@2000 | 3.241 | 2.112 | 4.729 | 3.138 | 131.6 | 95 |
Paired comparisons
| policy | vs | mean_delta_mae | delta_p50 | win_rate |
|---|
| state-copy-norm | state-copy | +0.751 | +2.004 | 32% |
| bijou@2000 | state-copy | -5.593 | -1.956 | 86% |
Per-motor chunk MAE
| policy | shoulder_pan.pos | shoulder_lift.pos | elbow_flex.pos | wrist_flex.pos | wrist_roll.pos | gripper.pos |
|---|
| state-copy | 4.35 | 17.57 | 3.46 | 7.10 | 13.90 | 9.76 |
| state-copy-norm | 4.32 | 20.94 | 3.69 | 6.02 | 16.42 | 7.81 |
| bijou@2000 | 1.57 | 5.32 | 1.10 | 1.49 | 9.07 | 0.89 |
Per-dataset chunk MAE (1 datasets; sorted by bijou@2000 MAE, worst first; rows with few frames are noise-dominated)
| dataset | frames | state-copy | state-copy-norm | bijou@2000 |
|---|
| fontaine/grasp_sft_demos_v0 | 256 | 9.356 | 9.868 | 3.241 |
Sample predictions (12 of 256 scored frames)
fontaine/grasp_sft_demos_v0 — episode 0, frame 74 (global index 74)
task: Pick up the toy boat and place it on the wooden disk.
state: [-2.8, -7.6, 24.0, 41.5, 79.2, 6.9]
chunk MAE — state-copy: 1.95 | state-copy-norm: 3.95 | bijou@2000: 2.65
fontaine/grasp_sft_demos_v0 — episode 29, frame 230 (global index 4716)
task: Pick up the toy boat and place it on the wooden disk.
state: [-31.1, -0.8, 26.4, 45.3, -157.2, 7.1]
chunk MAE — state-copy: 2.03 | state-copy-norm: 5.60 | bijou@2000: 1.75
fontaine/grasp_sft_demos_v0 — episode 47, frame 100 (global index 7681)
task: Pick up the toy boat and place it on the wooden disk.
state: [-8.5, -8.8, 27.4, 43.7, 67.6, 7.2]
chunk MAE — state-copy: 3.35 | state-copy-norm: 5.66 | bijou@2000: 1.13
fontaine/grasp_sft_demos_v0 — episode 76, frame 98 (global index 12354)
task: Pick up the toy boat and place it on the wooden disk.
state: [-4.9, 5.2, 21.8, 49.7, 56.4, 41.7]
chunk MAE — state-copy: 1.98 | state-copy-norm: 2.07 | bijou@2000: 3.78
fontaine/grasp_sft_demos_v0 — episode 99, frame 38 (global index 16417)
task: Pick up the toy boat and place it on the wooden disk.
state: [-6.2, 3.2, 26.4, 57.2, 49.7, 41.7]
chunk MAE — state-copy: 11.63 | state-copy-norm: 8.01 | bijou@2000: 3.21
fontaine/grasp_sft_demos_v0 — episode 125, frame 115 (global index 21254)
task: Pick up the toy boat and place it on the wooden disk.
state: [-17.0, -6.6, 27.2, 41.6, 79.3, 3.9]
chunk MAE — state-copy: 2.22 | state-copy-norm: 5.10 | bijou@2000: 2.30
fontaine/grasp_sft_demos_v0 — episode 162, frame 55 (global index 26537)
task: Pick up the toy boat and place it on the wooden disk.
state: [-4.2, -0.1, 29.5, 56.3, 59.0, 5.9]
chunk MAE — state-copy: 4.84 | state-copy-norm: 5.75 | bijou@2000: 3.42
fontaine/grasp_sft_demos_v0 — episode 192, frame 113 (global index 31879)
task: Pick up the toy boat and place it on the wooden disk.
state: [-12.1, -3.8, 21.5, 35.4, 130.0, 4.8]
chunk MAE — state-copy: 3.01 | state-copy-norm: 5.11 | bijou@2000: 1.24
fontaine/grasp_sft_demos_v0 — episode 214, frame 73 (global index 36127)
task: Pick up the toy boat and place it on the wooden disk.
state: [1.3, -4.2, 18.7, 35.4, 80.4, 3.8]
chunk MAE — state-copy: 1.27 | state-copy-norm: 4.32 | bijou@2000: 2.33
fontaine/grasp_sft_demos_v0 — episode 236, frame 65 (global index 40035)
task: Pick up the toy boat and place it on the wooden disk.
state: [-10.6, -8.0, 27.1, 45.8, 157.2, 7.7]
chunk MAE — state-copy: 2.26 | state-copy-norm: 4.44 | bijou@2000: 1.89
fontaine/grasp_sft_demos_v0 — episode 259, frame 59 (global index 44530)
task: Pick up the toy boat and place it on the wooden disk.
state: [-10.2, 0.7, 20.7, 53.5, 141.4, 5.8]
chunk MAE — state-copy: 5.77 | state-copy-norm: 6.86 | bijou@2000: 2.15
fontaine/grasp_sft_demos_v0 — episode 288, frame 146 (global index 49532)
task: Pick up the toy boat and place it on the wooden disk.
state: [-9.5, 2.0, 22.3, 60.6, -157.2, 9.0]
chunk MAE — state-copy: 7.32 | state-copy-norm: 9.20 | bijou@2000: 2.80
