How faithfully do open-weight models stay to a machine-checked decision policy as they reason?
Every score is from the fail-closed AmberTrace
verifier — not an LLM judge — over the 1,350-item decision_eval_v1 corpus (single sample, temperature 0).
The headline isn't accuracy but the direction of error: fail-open (under-restriction) on the
safety-critical band is the failure a plain accuracy number hides.
The leaderboard scores live against AmberTrace, so a row can't be gamed by memorising a key:
pip install ambertrace-rlvr
python examples/run_alignment_matrix.py --model <your-model>Then open a PR with your outputs/row_full_<model>.json;
examples/export_matrix_results.py folds it into the dataset this Space reads.
Full recipe in the repo.