Python Coverage boolean compound condition line exit missing branch coverage

2024-09-28 (Saturday) | 400 words (~2 minutes reading)

With branch coverage enabled in Coverage.py or pytest-coverage, you might see it report missing coverage on a compound boolean condition, like this:

def foobar() -> bool:
  return (
    condition_1()
    and condition_2()
    and condition_3()
    and condition_4()
    and condition_5()
  )

That might lead to a report that the line where the boolean compound condition begins to the exit of the function are missing coverage. The report will look like this:

---------- coverage: platform linux, python 3.12.1-final-0 -------

Name                  Stmts   Miss Branch BrPart  Cover   Missing
------------------------------------------------------------------
src/foobar.py            39      0     16      1    98%   37->exit
------------------------------------------------------------------

TOTAL                   188      0     68      1    99%

The “line to exit” part is 37->exit in the Missing column on the right.

This can be surprising if you know you have tests covering that compound boolean evaluating to True and False. This issue is caused by the branch coverage analysis in Coverage.py being quite thorough. It requires that each possible step through the compound boolean is covered.

This isn’t as bad as it sounds if you thought that meant you need to write tests for a Cartesian product of all of the possible permutations of conditions in the compound. You only need to cover each boolean step sequentially. For example, with a boolean compound with five sub-conditions, you’d need tests covering those sub-conditions in the following sequences:

False                            (overall False)
True, False                      (overall False)
True, True, False                (overall False)
True, True, True, False          (overall False)
True, True, True, True, True     (overall True)

Note that only one of the combinations evaluates to True overall.

When writing tests for this kind of boolean compound, it’s common to only test it evaluating to overall True, and perhaps one of the possible False evaluations, which is what leads Coverage.py to report missing branch coverage. Unfortunately Coverage.py will not tell you which step of the compound boolean is missing coverage, so the final report of [line]->exit is a bit vague and difficult to decipher.

Making sure that tests cover all of the possible paths through the compound boolean will ensure that Coverage.py reports 100% test coverage for it.


Tech mentioned