spryker / mutation-testing
Playground repository for Mutation Testing
Requires (Dev)
- infection/infection: ^0.26.16
- phpstan/phpstan: ^1.9
- phpunit/phpunit: ^9.5
- spryker/code-sniffer: ^0.17.14
This package is auto-updated.
Last update: 2024-11-05 08:49:58 UTC
README
This repository contains some examples and workshops that can be used to play around and understand Mutation-Testing. We are using the great Infection Framework for this which is automatically installed when you set up this project.
Setup
Install all dependencies
composer install
Run tests with PHPUnit
Tests without coverage report
vendor/bin/phpunit
With coverage report
XDEBUG_MODE=coverage vendor/bin/phpunit --coverage-html tests/output
By default, path coverage is disabled to enable it run
XDEBUG_MODE=coverage vendor/bin/phpunit --coverage-html tests/output --path-coverage
Run tests with Infection
For performance reasons you should always run infection with the --threads=max
option
vendor/bin/infection --threads=max
To see which Mutants were created you can run the command with the --show-mutations
(-s
) option.
vendor/bin/infection -s --threads=max
Introduction
Mutation testing is a methodology to calculate the effectiveness and efficiency of your test suite. Traditional test frameworks like PHPUnit can generate codecoverage reports which are mostly line based ones.
Coverage
We have three types of coverage
- Line coverage
- Branch coverage
- Path coverage
In PHPUnit you can enable the path coverage in the configuration or via CLI command option. The most efficient one is the path coverage the least one is line coverage. One executed line of code doesn't mean that all possible cases are covered with tests.
What is Mutation Testing
- Mutates the code under Test
- Gives you an understanding of the test quality
- Tests the effectiveness of your test suite
- Each possible change results in a Mutant
- Tests are executed against the Mutants
- Gives you a way more precise coverage (MSI)
Each line of your code under test will be analysed and possible permuations out of the code will be generated. There are many ways to manipulate the code. See the list below for some of them:
- Changing operators e.g. $a + $b vs. $a - $b
- Remove code lines
- Remove type casting
- Change integer values e.g. 0 to 1
- etc
There are many more checkout https://infection.github.io/guide/mutators.html to get an overview.
Why should you do Mutation Testing
- Higher confidence in the test quality
- Finds code changes that could break the code
- Shows you what is really covered
- Kind of code review
References
Mutation Testing https://en.wikipedia.org/wiki/Mutation_testing
Framework https://infection.github.io/
Talk from the Infection Framework owner PHPKonf 2021 - Maks Rafalko: Mutation Testing in PHP https://www.youtube.com/watch?v=aDdXTY372Vo
Code Coverage https://doug.codes/php-code-coverage