Advanced C# Testing: Property-Based Testing and Mutation Testing
Testing feels like flossing your teeth—everyone knows they should do it, but many developers only focus on the basics. Unit testing is the bread and butter of most test suites, but if you stick only to traditional unit tests, you may miss edge cases or leave holes in your code coverage. That’s where advanced testing techniques like property-based testing and mutation testing come in.
In this post, we’ll go beyond the basics. We'll explore FsCheck for property-based testing and Stryker.NET for mutation testing, diving deep into how these tools can help you write robust tests and improve code quality.
Why Go Beyond Unit Testing?
Imagine you’re testing a function that calculates discounts based on a user’s membership level. A typical unit test might verify:
- Bronze members receive a 5% discount.
- Silver members receive a 10% discount.
- Gold members receive a 20% discount.
While these tests may pass, they only check a few predefined inputs. What happens when you throw unexpected inputs at the function? What if you introduce a bug in your code that the tests don’t catch?
Property-based testing and mutation testing address these blind spots by forcing your code to handle a wide range of scenarios and ensuring your tests can detect subtle bugs.
Property-Based Testing: FsCheck to the Rescue
What is Property-Based Testing?
Unlike traditional unit testing, where you define specific inputs and expected outputs, property-based testing focuses on general properties of your code that should always hold true. You define rules, and the testing framework automatically generates random inputs to test those rules.
Real-world analogy: Think of property-based testing as stress-testing your house during construction. Instead of checking if the door opens and closes, you simulate hurricanes, earthquakes, and floods to ensure your house remains standing.
Why Use FsCheck?
FsCheck is a powerful property-based testing library for .NET. It automatically generates thousands of test cases based on properties you define, helping you uncover edge cases you didn’t think of.
Getting Started with FsCheck
First, install FsCheck via NuGet:
dotnet add package FsCheck
Now, let’s test a simple function:
public static class DiscountCalculator
{
public static decimal CalculateDiscount(string membershipLevel, decimal purchaseAmount)
{
return membershipLevel switch
{
"Bronze" => purchaseAmount * 0.05m,
"Silver" => purchaseAmount * 0.10m,
"Gold" => purchaseAmount * 0.20m,
_ => 0
};
}
}
The function calculates discounts based on membership levels. Let’s write a property-based test for it.
Writing a Property-Based Test
Here’s how we can use FsCheck to ensure the discounts are always positive and proportional to the purchase amount:
using FsCheck;
public class DiscountTests
{
public static bool DiscountIsAlwaysPositive(string membershipLevel, decimal purchaseAmount)
{
decimal discount = DiscountCalculator.CalculateDiscount(membershipLevel, purchaseAmount);
return discount >= 0;
}
public static bool DiscountIsProportional(string membershipLevel, decimal purchaseAmount)
{
decimal discount = DiscountCalculator.CalculateDiscount(membershipLevel, purchaseAmount);
return discount <= purchaseAmount;
}
public static void RunTests()
{
Prop.ForAll<string, decimal>((membershipLevel, purchaseAmount) =>
DiscountIsAlwaysPositive(membershipLevel, Math.Abs(purchaseAmount))
).QuickCheck();
Prop.ForAll<string, decimal>((membershipLevel, purchaseAmount) =>
DiscountIsProportional(membershipLevel, Math.Abs(purchaseAmount))
).QuickCheck();
}
}
Breaking Down the Code
Prop.ForAll: Defines the property being tested. FsCheck generates random inputs for the function.
Math.Abs(purchaseAmount): Ensures purchase amounts are positive (you can customize generators for more control).
QuickCheck: Runs the test quickly with a default number of random cases.
Run the RunTests method and watch FsCheck automatically test your code with hundreds of possible inputs.
Mutation Testing: Be Smarter About Test Coverage
What is Mutation Testing?
Mutation testing is a way to measure the effectiveness of your test suite. It works by introducing small bugs (mutations) into your code and checking if your tests catch them. If a mutation goes undetected, it’s a sign your tests are missing something.
Real-world analogy: Think of mutation testing as hiring a "friendly hacker" to test your defenses. If they can break into your system, you know your security measures need improvement.
Why Use Stryker.NET?
Stryker.NET is the leading mutation testing tool for C#. It’s easy to integrate into your project and provides a clear report on how well your tests detect injected mutations.
Getting Started with Stryker.NET
First, install Stryker.NET globally:
dotnet tool install -g dotnet-stryker
Next, run Stryker in your test project directory:
dotnet stryker
Stryker.NET will mutate your code and execute your existing test suite. It then reports a mutation score, which indicates how many mutations your tests detected.
Example Mutation Testing Workflow
Let’s mutate our discount calculator. If we accidentally change purchaseAmount * 0.05m to purchaseAmount * 0.15m, will our tests catch it?
Here’s a simple test suite for the discount calculator:
using Xunit;
public class DiscountCalculatorTests
{
[Fact]
public void BronzeDiscount_ShouldBe5Percent()
{
decimal discount = DiscountCalculator.CalculateDiscount("Bronze", 100);
Assert.Equal(5m, discount);
}
[Fact]
public void SilverDiscount_ShouldBe10Percent()
{
decimal discount = DiscountCalculator.CalculateDiscount("Silver", 100);
Assert.Equal(10m, discount);
}
[Fact]
public void GoldDiscount_ShouldBe20Percent()
{
decimal discount = DiscountCalculator.CalculateDiscount("Gold", 100);
Assert.Equal(20m, discount);
}
}
Run Stryker.NET and check the mutation score. If mutations like changing 0.05m to 0.15m go uncaught, you may need to add more tests.
Common Pitfalls (And How to Avoid Them)
1. Over-Reliance on Property-Based Testing
Problem: Property-based testing can generate irrelevant inputs (e.g., negative purchase amounts for discounts).
Solution: Customize input generators with FsCheck’sArbfeature.
Arb.Register<PositiveDecimalGenerator>();
2. Ignoring Mutation Testing Reports
Problem: Developers often look at the mutation score but don’t analyze which mutations went undetected.
Solution: Carefully review the Stryker.NET report and add tests for undetected mutations.
3. Performance Overhead
Problem: Property-based testing and mutation testing can be slow for large projects.
Solution: Run these tests in CI pipelines or only for critical core logic.
Key Takeaways
Property-Based Testing: Use FsCheck to test general properties of your code, ensuring robustness across a wide range of inputs.
Mutation Testing: Use Stryker.NET to measure the effectiveness of your test suite and identify gaps in coverage.
Improved Code Quality: These techniques force you to think critically about edge cases and vulnerabilities in your code.
Next Steps
Ready to level up your testing game? Here’s what you can do next:
Install FsCheck and Stryker.NET: Start experimenting with these tools in your own projects.
Learn Custom Generators in FsCheck: Explore how to create tailored input generators for complex scenarios.
Integrate Mutation Testing in CI/CD: Automate mutation testing to ensure continuous improvement in test quality.
Testing doesn’t have to feel like a chore. With property-based testing and mutation testing, you can transform your code into a fortress—ready to handle any input or bug that comes your way.
So, what are you waiting for? Start testing smarter today! 🚀
SOCIAL SHARE CARD GENERATOR