I recommend running coverage on your tests.
Here’s a couple of reasons why, from the past couple of months.
Example one #
When writing tests, it’s common to copy and paste test functions, but sometimes you forget to rename the new one (see also: the Last Line Effect).
For example:
def test_get_install_to_run_with_platform(patched_installs):
i = installs.get_install_to_run("<none>", None, "1.0-32")
assert i["id"] == "PythonCore-1.0-32"
assert i["executable"].match("python.exe")
i = installs.get_install_to_run("<none>", None, "2.0-arm64")
assert i["id"] == "PythonCore-2.0-arm64"
assert i["executable"].match("python.exe")
def test_get_install_to_run_with_platform(patched_installs):
i = installs.get_install_to_run("<none>", None, "1.0-32", windowed=True)
assert i["id"] == "PythonCore-1.0-32"
assert i["executable"].match("pythonw.exe")
i = installs.get_install_to_run("<none>", None, "2.0-arm64", windowed=True)
assert i["id"] == "PythonCore-2.0-arm64"
assert i["executable"].match("pythonw.exe")
The tests pass, but the first one is never run because its name is redefined. This clearly shows up as a non-run test in the coverage report. In this case, we only need to rename one of them, and both are covered and pass.
But sometimes there’s a bug in the test which would cause it to fail, but we just don’t know because it’s not run.
Example two #
This is more subtle:
im = Image.new("RGB", (1, 1))
for colors in (("#f00",), ("#f00", "#0f0")):
append_images = (Image.new("RGB", (1, 1), color) for color in colors)
im_reloaded = roundtrip(im, save_all=True, append_images=append_images)
assert_image_equal(im, im_reloaded)
assert isinstance(im_reloaded, MpoImagePlugin.MpoImageFile)
assert im_reloaded.mpinfo is not None
assert im_reloaded.mpinfo[45056] == b"0100"
for im_expected in append_images:
im_reloaded.seek(im_reloaded.tell() + 1)
assert_image_similar(im_reloaded, im_expected, 1)
It’s not so obvious when looking at the code, but Codecov highlights a problem:
The append_images generator is being consumed inside roundtrip(), so we have nothing
to iterate over in the for loop – hence no coverage. The
fix is to
use a list instead of a generator.
Header photo: Misplaced manhole cover (CC BY-NC-SA 2.0 Hugo van Kemenade).