Upstream bug reproductions
bunsen works around some defects in burn and its ecosystem. This page
explains how each one is recorded, so that its workaround is removed when
the defect is fixed rather than left behind to become a bug of its own. The
reproductions live in
crates/dev/burn_bug_repro,
whose crate doc is the reference.
Two tests per defect
Each defect gets a module, and the module carries two kinds of test:
| Test | Asserts | Fails when | Runs by default |
|---|---|---|---|
| the reproduction | the correct semantics | the bug is present | no, it is #[ignore]d |
| the behaviour pin | the current, wrong, behaviour | the bug is fixed | yes |
The reproduction is what you would send upstream. It says what should happen, and it fails today, so it is ignored to keep the suite green.
The behaviour pin is the half that protects bunsen. It asserts that the defect is still there. When a dependency update fixes it, the pin fails, and its message says what to do next: which workaround to remove, and which reproduction to un-ignore. Without the pin, an upstream fix would arrive silently, and the workaround would go on running against data that no longer needs it.
A pin names its workaround
A pin’s failure message is written for whoever next updates the dependency: it names the workaround the defect justified and says what to do with it. Some workarounds turn into bugs once the defect is fixed. A repair that undoes damage, applied to data that is no longer damaged, does damage of its own, and its pin asks for the repair’s removal.
Others are harmless after a fix. unfold on the CubeCL backends truncates an
outer stride, and bunsen avoids it by trimming an input to the span its
windows cover. The trim changes nothing once the defect is gone, so the pin
in the unfold module asks for a re-check rather than a removal. Write the
message for the workaround you have.
Writing a reproduction
- Public API only. Rest the reproduction on burn’s public API, so that taking it upstream means swapping the test helpers, not rewriting the test. A test of bunsen’s own workaround can sit in the same module; it just doesn’t travel with the reproduction.
- Document the defect in the module. The existing modules each say what the expected semantics are, what goes wrong, why it is easy to miss, its scope, and how to recover from it.
- Mind the CPU fallback. A kernel defect that the CPU backend doesn’t share can’t be seen on a CPU-only run, where a pin would report a fix that hasn’t happened. Gate such a pin on the backend features. A defect in a reader or a store is backend-independent and needs no gate (Testing and backends).
- Run it with a backend, in release mode. The crate doc has the command.
Why a separate crate
A reproduction is a statement about somebody else’s code: it carries a fixture, it wants a real accelerator to say anything, and its whole purpose is to stop being true. None of that belongs on a published library’s public surface.