ergebnis / phpunit-slow-test-detector
Provides facilities for detecting slow tests in phpunit/phpunit.
Installs: 2 352 371
Dependents: 63
Suggesters: 0
Security: 0
Stars: 122
Watchers: 4
Forks: 7
Open Issues: 10
Requires
- php: ~7.0.0 || ~7.1.0 || ~7.2.0 || ~7.3.0 || ~7.4.0 || ~8.0.0 || ~8.1.0 || ~8.2.0 || ~8.3.0 || ~8.4.0
- phpunit/phpunit: ^6.5.0 || ^7.5.0 || ^8.5.19 || ^9.0.0 || ^10.0.0 || ^11.0.0
Requires (Dev)
- ergebnis/composer-normalize: ^2.44.0
- ergebnis/license: ^2.6.0
- ergebnis/php-cs-fixer-config: ^6.38.0
- fakerphp/faker: ~1.20.0
- phpstan/extension-installer: ^1.4.3
- phpstan/phpstan: ^1.12.11
- phpstan/phpstan-deprecation-rules: ^1.2.1
- phpstan/phpstan-phpunit: ^1.4.1
- phpstan/phpstan-strict-rules: ^1.6.1
- psr/container: ~1.0.0
- rector/rector: ^1.2.10
- 2.17.0
- dev-main / 2.16.x-dev
- 2.16.1
- 2.16.0
- 2.15.1
- 2.15.0
- 2.14.0
- 2.13.0
- 2.12.0
- 2.11.0
- 2.10.0
- 2.9.0
- 2.8.0
- 2.7.0
- 2.6.0
- 2.5.0
- 2.4.0
- 2.3.2
- 2.3.1
- 2.3.0
- 2.2.0
- 2.1.1
- 2.1.0
- 2.0.0
- 1.0.0
- dev-dependabot/composer/ergebnis/composer-normalize-2.45.0
- dev-dependabot/github_actions/actions/cache-4.2.0
- dev-dependabot/github_actions/codecov/codecov-action-5.1.1
- dev-fix/maximum-duration
- dev-feature/seconds
This package is auto-updated.
Last update: 2024-12-09 12:33:18 UTC
README
This project provides a composer
package and a Phar archive with an extension for detecting slow tests in phpunit/phpunit
.
The extension is compatible with the following versions of phpunit/phpunit
:
phpunit/phpunit:^6.5.0
phpunit/phpunit:^7.5.0
phpunit/phpunit:^8.5.19
phpunit/phpunit:^9.0.0
phpunit/phpunit:^10.0.0
phpunit/phpunit:^11.0.0
Installation
Installation with composer
Run
composer require --dev ergebnis/phpunit-slow-test-detector
to install ergebnis/phpunit-slow-test-detector
as a composer
package.
Installation as Phar
Download phpunit-slow-test-detector.phar
from the latest release.
Usage
Bootstrapping the extension
Before the extension can detect slow tests in phpunit/phpunit
, you need to bootstrap it. The bootstrapping mechanism depends on the version of phpunit/phpunit
you are using.
Bootstrapping the extension as a composer
package
To bootstrap the extension as a composer
package when using
phpunit/phpunit:^6.5.0
adjust your phpunit.xml
configuration file and configure the
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" > + <listeners> + <listener class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + </listeners> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
To bootstrap the extension as a composer
package when using
phpunit/phpunit:^7.5.0
phpunit/phpunit:^8.5.19
phpunit/phpunit:^9.0.0
adjust your phpunit.xml
configuration file and configure the
extensions
element onphpunit/phpunit:^7.5.0
extensions
element onphpunit/phpunit:^8.5.19
extensions
element onphpunit/phpunit:^9.0.0
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" > + <extensions> + <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + </extensions> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
To bootstrap the extension as a composer
package when using
phpunit/phpunit:^10.0.0
phpunit/phpunit:^11.0.0
adjust your phpunit.xml
configuration file and configure the
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" > + <extensions> + <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + </extensions> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
Bootstrapping the extension as a PHAR
To bootstrap the extension as a PHAR when using
phpunit/phpunit:^7.5.0
phpunit/phpunit:^8.5.19
phpunit/phpunit:^9.0.0
adjust your phpunit.xml
configuration file and configure the
extensionsDirectory
attribute and theextensions
element onphpunit/phpunit:^7.5.0
extensionsDirectory
attribute and theextensions
element onphpunit/phpunit:^8.5.19
extensionsDirectory
attribute and theextensions
element onphpunit/phpunit:^9.0.0
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" + extensionsDirectory="directory/where/you/saved/the/extension/phars" > + <extensions> + <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + </extensions> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
To bootstrap the extension as a PHAR when using
phpunit/phpunit:^10.0.0
phpunit/phpunit:^11.0.0
adjust your phpunit.xml
configuration file and configure the
extensionsDirectory
attribute and theextensions
element onphpunit/phpunit:^10.0.0
extensionsDirectory
attribute and theextensions
element onphpunit/phpunit:^11.0.0
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" + extensionsDirectory="directory/where/you/saved/the/extension/phars" > + <extensions> + <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + </extensions> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
Configuring the extension
You can configure the extension with the following options in your phpunit.xml
configuration file:
maximum-count
, anint
, the maximum count of slow test that should be reported, defaults to10
maximum-duration
, anint
, the maximum duration in milliseconds for a test before the extension considers it as a slow test, defaults to500
The configuration mechanism depends on the version of phpunit/phpunit
you are using.
Configuring the extension
To configure the extension when using
phpunit/phpunit:^6.5.0
adjust your phpunit.xml
configuration file and configure the
The following example configures the maximum count of slow tests to three, and the maximum duration for all tests to 250 milliseconds:
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" > <listeners> - <listener class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + <listener class="Ergebnis\PHPUnit\SlowTestDetector\Extension"> + <arguments> + <array> + <element key="maximum-count"> + <integer>3</integer> + </element> + <element key="maximum-duration"> + <integer>250</integer> + </element> + </array> + </arguments> + </listener> </listeners> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
To configure the extension when using
phpunit/phpunit:^7.5.0
phpunit/phpunit:^8.5.19
phpunit/phpunit:^9.0.0
adjust your phpunit.xml
configuration file and configure the
arguments
element onphpunit/phpunit:^7.5.0
arguments
element onphpunit/phpunit:^8.5.19
arguments
element onphpunit/phpunit:^9.0.0
The following example configures the maximum count of slow tests to three, and the maximum duration for all tests to 250 milliseconds:
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" > <extensions> - <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + <extension class="Ergebnis\PHPUnit\SlowTestDetector\Extension"> + <arguments> + <array> + <element key="maximum-count"> + <integer>3</integer> + </element> + <element key="maximum-duration"> + <integer>250</integer> + </element> + </array> + </arguments> + </extension> </extensions> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
To configure the extension when using
phpunit/phpunit:^10.0.0
phpunit/phpunit:^11.0.0
adjust your phpunit.xml
configuration file and configure one or more
The following example configures the maximum count of slow tests to three, and the maximum duration for all tests to 250 milliseconds:
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="vendor/phpunit/phpunit/phpunit.xsd" bootstrap="vendor/autoload.php" > <extensions> - <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"/> + <bootstrap class="Ergebnis\PHPUnit\SlowTestDetector\Extension"> + <parameter name="maximum-count" value="3"/> + <parameter name="maximum-duration" value="250"/> + </bootstrap> </extensions> <testsuites> <testsuite name="unit"> <directory>test/Unit/</directory> </testsuite> </testsuites> </phpunit>
Configuring the maximum duration per test case
You can configure the maximum duration for a single test case with
- an
Attribute\MaximumDuration
attribute when usingphpunit/phpunit:^10.0.0
phpunit/phpunit:^11.0.0
- a
@maximumDuration
annotation in the DocBlock when usingphpunit/phpunit:^6.5.0
phpunit/phpunit:^7.5.0
phpunit/phpunit:^8.5.19
phpunit/phpunit:^9.0.0
- a
@slowThreshold
annotation in the DocBlock when usingphpunit/phpunit:^6.5.0
phpunit/phpunit:^7.5.0
phpunit/phpunit:^8.5.19
phpunit/phpunit:^9.0.0
The following example configures the maximum durations for single test cases to 5.000 ms, 4.000 ms, and 3.000 ms:
<?php declare(strict_types=1); use PHPUnit\Framework; use Ergebnis\PHPUnit\SlowTestDetector; final class ExtraSlowTest extends Framework\TestCase { /** */ #[SlowTestDetector\Attribute\MaximumDuration(5000)] public function testExtraExtraSlow(): void { // ... } /** * @maximumDuration 4000 */ public function testAlsoQuiteSlow(): void { // ... } /** * @slowThreshold 3000 */ public function testQuiteSlow(): void { // ... } }
Note
Support for the @slowThreshold
annotation exists only to help you move from johnkary/phpunit-speedtrap
. It will be deprecated and removed in the near future.
Running tests
When you have bootstrapped the extension, you can run your tests as usually:
vendor/bin/phpunit
When the extension has detected slow tests, it will report them:
PHPUnit 10.0.0 by Sebastian Bergmann and contributors. Runtime: PHP 8.1.0 Configuration: test/EndToEnd/Default/phpunit.xml Random Seed: 1676103726 ............. 13 / 13 (100%) Detected 11 tests where the duration exceeded the maximum duration. 1. 1.604 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#9 2. 1.505 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#8 3. 1.403 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#7 4. 1.303 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#6 5. 1.205 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#5 6. 1.103 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#4 7. 1.005 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#3 8. 0.905 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#2 9. 0.805 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#1 10. 0.705 (0.500) Ergebnis\PHPUnit\SlowTestDetector\Test\EndToEnd\Default\SleeperTest::testSleeperSleepsLongerThanDefaultMaximumDurationWithDataProvider#0 There is 1 additional slow test that is not listed here. Time: 00:12.601, Memory: 8.00 MB OK (13 tests, 13 assertions)
Understanding measured test durations
When using
-
phpunit/phpunit:^6.5.0
-
phpunit/phpunit:^7.5.0
-
phpunit/phpunit:^8.5.19
-
phpunit/phpunit:^9.0.0
-
the extension uses the hooks event system of
phpunit/phpunit
.
The hooks event system supports eleven hook methods that phpunit/phpunit
invokes during the execution of tests.
When the extension uses the hooks event system, it uses the PHPUnit\Runner\AfterTestHook
, which receives the duration of invoking PHPUnit\Framework\TestCase::runBare()
and more.
When phpunit/phpunit
invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods before the first test method in the class:
When phpunit/phpunit
invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods before every test method in the class:
PHPUnit\Framework\TestCase::setUp()
and methods annotated with@before
PHPUnit\Framework\TestCase::assertPreConditions()
When phpunit/phpunit
invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods after every test method in the class:
PHPUnit\Framework\TestCase::assertPostConditions()
PHPUnit\Framework\TestCase::tearDown()
and methods annotated with@after
When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods after the last test method in the class:
Note
Because of this behavior, the measured test durations can and will vary depending on the order in which phpunit/phpunit
executes tests.
When using
phpunit/phpunit:^10.0.0
phpunit/phpunit:^11.0.0
the extension uses the new event system of phpunit/phpunit
.
The new event system supports a wide range of events that phpunit/phpunit
emits during the execution of tests.
When the extension uses the new event system, it uses and subscribes to the PHPUnit\Event\Test\PreparationStarted
and PHPUnit\Event\Test\Finished
events and measures the duration between the points in time when phpunit/phpunit
emits the former and the latter.
When phpunit/phpunit
invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods before the first test method in the class:
When phpunit/phpunit
invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods before every test method in the class:
PHPUnit\Framework\TestCase::setUp()
and methods annotated with@before
PHPUnit\Framework\TestCase::assertPreConditions()
When phpunit/phpunit
invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods after every test method in the class:
PHPUnit\Framework\TestCase::assertPostConditions()
PHPUnit\Framework\TestCase::tearDown()
and methods annotated with@after
When phpunit/phpunit invokes PHPUnit\Framework\TestCase::runBare()
, it will invoke the following methods after the last test method in the class:
Note
Because of this behavior, the measured test durations can and will vary depending on the order in which phpunit/phpunit
executes tests.
Changelog
The maintainers of this project record notable changes to this project in a changelog.
Contributing
The maintainers of this project suggest following the contribution guide.
Code of Conduct
The maintainers of this project ask contributors to follow the code of conduct.
General Support Policy
The maintainers of this project provide limited support.
You can support the maintenance of this project by sponsoring @localheinz or requesting an invoice for services related to this project.
PHP Version Support Policy
This project supports PHP versions with active and security support.
The maintainers of this project add support for a PHP version following its initial release and drop support for a PHP version when it has reached the end of security support.
Security Policy
This project has a security policy.
License
This project uses the MIT license.
Credits
This package is inspired by johnkary/phpunit-speedtrap
, originally licensed under MIT by John Kary.
Social
Follow @localheinz and @ergebnis on Twitter.