Skip to content

Update mutation-testing.md #46064

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
50 changes: 50 additions & 0 deletions docs/core/testing/mutation-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,56 @@ Stryker supports several types of mutations:

For additional mutation types, see the [Stryker.NET: Mutations](https://stryker-mutator.io/docs/stryker-net/mutations) documentation.

## Interpret mutation testing results

After running Stryker.NET, you'll receive a report that categorizes mutants as **killed**, **survived**, or **timeout**. Here's how to interpret and act on these results:

- **Killed**: These are changes that your tests successfully caught. A high number of killed mutants indicates that your test suite effectively detects logic errors.

- **Survived**: These changes weren't caught by your tests. Review them to identify gaps in test coverage or assertions that are too weak. Focus on adding targeted unit tests that would fail if the mutant were real.

- **Timeout**: These mutations caused your code to hang or exceed the allowed time. This can happen with infinite loops or unoptimized paths. Investigate the code logic or increase the timeout threshold if needed.

> [!NOTE]
> Don't chase a 100% mutation score. Instead, focus on high risk or business critical areas where undetected bugs would be most costly.

## Adding mutation testing to your CI/CD workflow

You can seamlessly integrate mutation testing into your continuous integration and delivery workflows. For instance, Stryker.NET can be configured to run within your Azure Pipelines or GitHub Actions setup, allowing you to enforce quality thresholds as part of your automated testing process.
```
"thresholds": {
"high": 85,
"low": 65,
"break": 0
}
```
Comment on lines +129 to +135
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this here? There's no mention of config yet, it's just a code block with values that are not described anywhere. Let's remove this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove it, but we are talking about "quality thresholds". This is an example how a threshold looks like. It's your call if you want it gone or not.


## Customization

Besides setting thresholds for your pipeline, Stryker.NET offers the possibility of having different configurations for each of your project needs. You can do this customization of behavior using the _stryker-config.json_ file.
```json
{
"ignore-mutations": [
"ToString",
"Equals",
"GetHashCode"
],
"ignore-methods": [
"*Logs"
]
"mutate": [
"!**/Migrations/*",
"!**/*.Designer.cs"
]
}
```

- **ignore-mutations**: These are the methods or expressions that you want to ignore because they are noisy, or you consider not needed based on your application logic needs. They will show up in your reports as `Ignored`.
- **ignore-methods**: You can use this to skip entire methods based on their signatures. Also, those will appear in your reports as Ignored. In our example, we ignore all methods ending in "Logs".
- **mutate**: Without this option, Styker will try to mutate all the files in your project. With this, you can ignore files or entire folders. In the example above, we ignore everything inside a _Migrations_ folder and all _.Designer.cs_ files (which are usually autogenerated) .

For more information, see [Stryker: Configuration](https://stryker-mutator.io/docs/stryker-net/configuration/).

## Incremental improvement

If, after changing your code, the unit tests pass successfully, then they aren't sufficiently robust, and the mutant survived.
Expand Down