// Track

Tests Pass ≠ System Works

Operational validation as a distinct discipline

Test coverage measures code integrity. Operational validation measures whether the system produces its intended outcome. The gap between them is where the best-tested bot in the ecosystem goes 3 weeks without placing a trade. This track is the cultural correction.

3 lessons~21 min total

Lessons are shown in recommended order. Complete them in sequence for the best experience — or jump to any lesson.

Lesson 1·8 min read

The 340-Test Bot That Never Traded

The Hermes story as case study. 95% coverage, 5 docs, a runbook, a watchdog — and zero real trades for weeks. What test coverage measures, what it does not, and why this gap is the cultural failure that defines production ML.

Lesson 2·6 min read

Last Real Trade as a Metric

Bot liveness dashboards. 'Hours since last real trade' as a first-class metric, displayed alongside test pass rate. Separate axis from code quality — and the single number that would have caught Hermes before the ceiling analysis debate.

Lesson 3·7 min read

Operational Validation Checklist

What 'done' actually means. Running system produces intended outcome equals done. Merged PR equals not done. Tests passing equals not done. Deployed equals not done. The cultural correction that closes the loop Hermes exposed.