Introduction
Code coverage is an indicator, not an metric.
In this post, I will share an example for your entertainment.
The code
public class Animal
{
public bool CanTalk { get; init; }
public bool CanWalk { get; init; }
}
public static class AnimalValidator
{
public static bool IsHuman(Animal animal)
{
return animal.CanTalk && animal.CanWalk;
}
}
The good test
Let's test all the input combinations.
It gives us 100% code coverage:
public class AnimalValidatorTest
{
[Theory]
[InlineData(true, true)]
public void IsHuman_WhenAnimalCanTalkAndCanWalk_ReturnsTrue(bool canTalk, bool canWalk)
{
var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
var isHuman = AnimalValidator.IsHuman(animal);
Assert.True(isHuman);
}
[Theory]
[InlineData(false, false)]
[InlineData(false, true)]
[InlineData(true, false)]
public void IsHuman_WhenAnimalCannotTalkOrCannotWalk_ReturnsFalse(bool canTalk, bool canWalk)
{
var animal = new Animal { CanTalk = canTalk, CanWalk = canWalk };
var isHuman = AnimalValidator.IsHuman(animal);
Assert.False(isHuman);
}
}
The 100% coverage test
One day, someone removed two test cases because they thought that the tests were running for too long .
[Theory]
// [InlineData(false, false)]
[InlineData(false, true)]
// [InlineData(true, false)]
Interestingly, we still get 100% code coverage.
The explanation
Let's put the code into SharpLab.
This code:
return animal.CanTalk && animal.CanWalk;
will compile into:
if (animal.CanTalk)
{
return animal.CanWalk;
}
return false;
It means:
CanTalk = true contributes 50%.
CanTalk = false contributes 50%.
The value of CanWalk doesn't matter.
The brittleness of 100% coverage test
Another day, someone changed the order of boolean because they thought it would improve performance because majority of animals cannot walk.
public static bool IsHuman(Animal animal)
{
// return animal.CanTalk && animal.CanWalk;
return animal.CanWalk && animal.CanTalk;
}
Now, our test coverage drops from 100% to 50%, because both of our test cases are CanWalk = true.
Conclusion
C# code coverage is based on compiled code.
Writing good unit tests is more important than code coverage.
Hope you find it interesting.