# Bisect_ppx feature requests Ordered by priority: # Feature requests - Coverage for diffs - https://github.com/aantron/bisect_ppx/issues/305 (Mehdi Bouaziz) - integration with merbocop (Seb Mondet) - Integration with gitlab+coveralls (not sure if this makes sense) - https://github.com/aantron/bisect_ppx/issues/326 - Coverage per component, per test framework - @arvidj It would be nice if for each visited line, we could know which tests visited it. Unclear how to do this is in a framework-agnostic manner: - For instance, in our pytests, we could set an environment variable for each new test run. bisect_ppx would read this environment variable and so know which test is visiting a line. - But how to deal with alcotests, where tests are launched "from within the same binary"? Can't communicate through environment variables here - One of the possible use cases for this information is to re-run tests selectively based on what code has been changed. E.g. a step towards "Marco's Testing Dream" - Finer-grain coverage - for nodes in loop (todo: define loops in OCaml), we may want to have executions with 0, 1, 2+ visits - complex path coverage, when there are several (non-nested) conditionals following each other, make sure all paths are covered - MC/DC? - CI integration - Parallelize CI coverage job - Running the full test suite generates a lot of coverage data (gigabytes!). Finally, all traces are merged when making the report. Sometimes we might have to send coverage data from one job to another (I think). To reduce the size of the data, could we add a tool for combining the data before moving it around? # Bugs - Spurious errors *** invalid file: '_coverage_output/819770417.coverage' error: "unexpected end of file while reading magic number" I think it happens when tests fail abruptly (Philippe Bidinger) - stack overflow error https://gitlab.com/nomadic-labs/tezos/-/jobs/1094493391 (Philippe Bidinger) - some package incompatibilities between bisect.2.6.0 and other packages used by tezos (Raphaël Proust) - Investigate - https://gitlab.com/gitlab-org/gitlab/-/issues/300712 - https://gitlab.com/gitlab-org/gitlab/-/issues/15765 - https://gitlab.com/gitlab-org/gitlab/-/issues/300718 - https://gitlab.com/gitlab-org/gitlab/-/issues/270856 - @philippeb: Ref https://tezos-dev.slack.com/archives/C85U11HGD/p1615652695064700 : > There are currently three issues with ppx_bisect: > > - spurious errors *** invalid file: '_coverage_output/819770417.coverage' error: "unexpected end of > file while reading magic number" I think it happens when tests fail abruptly > - stack overflow error https://gitlab.com/nomadic-labs/tezos/-/jobs/1094493391 > - some package incompatibilities between bisect.2.6.0 and other packages used by tezos