hammerstone / flaky
A Laravel package to elegantly handle flaky operations.
Requires
- php: ^8.0
- illuminate/cache: ^8.0|^9.0|^10
- illuminate/console: ^8.0|^9.0|^10
- illuminate/support: ^8.0|^9.0|^10
Requires (Dev)
- mockery/mockery: ^1.3.3
- orchestra/testbench: ^6.0|^7.0|^8.0
- phpunit/phpunit: >=8.5.23|^9
README
Flaky for Laravel is a package that helps you handle operations that may have intermittent failures due to unreliable third-parties.
This is very much in beta! Please open issues / PRs for any problems.
Installation
You can install the package via Composer
composer require hammerstone/flaky
Usage
Let's say you have a flaky piece of code that fails 20% of the time:
if (Lottery::odds(1 / 5)->choose()) { throw new Exception("Oops"); }
But you don't care if it fails, as long as it doesn't fail for more than an hour. Then you could wrap that code up in Flaky protections.
Flaky::make('my-flaky-code') ->allowFailuresForAnHour() ->run(function() { if (Lottery::odds(1 / 5)->choose()) { throw new Exception("Oops"); } })
Now, exceptions will be silenced unless the operation hasn't succeeded in an hour.
Each instance of flaky code requires a unique ID passed through to the make
method. This is how we keep track of
failures over time. You can make up whatever you want, it's just a cache key.
Flaky uses your default cache store. That may need to be configurable in the future.
Throwing Exceptions
You have several different ways to control when exceptions are thrown:
Time Based
If you want to throw an exception after a certain period of time, you have several methods available to you.
allowFailuresForAMinute()
allowFailuresForMinutes($minutes)
allowFailuresForAnHour()
allowFailuresForHours($hours)
allowFailuresForADay()
allowFailuresForDays($days)
allowFailuresFor($seconds = 0, $minutes = 0, $hours = 0, $days = 0)
If your callback throws an exception, Flaky will check to see if it's still within the grace period. If it is, the exception will be captured.
If your callback succeeds, the deadline will be reset.
Consecutive Failures
If you'd prefer to take a numeric approach instead of a time-based approach, you can use the allowConsecutiveFailures
method.
Flaky::make('my-flaky-code') // It can fail ten times in a row. ->allowConsecutiveFailures(10) ->run(function() { // })
Now your function can fail 10 times in a row without alerting you, but on the 11th failure the exception will be thrown. If the callback succeeds, the consecutive failure counter will be reset.
Total Failures
If you want to throw an exception after a total number of failures, regardless of successes, you can use
the allowTotalFailures
method.
Flaky::make('my-flaky-code') // It can fail ten times total. ->allowTotalFailures(10) ->run(function() { // })
On the 11th failure, the exception will be thrown. The counter will be reset only after the exception has been thrown, but not for successful invocations. You can think of this as "Throw every 11th exception, regardless of successes."
Combining
You can combine the three methods in any way you like.
Flaky::make('my-flaky-code') // Alert after an hour. ->allowFailuresForAnHour() // Alert after the third consecutive failure. ->allowConsecutiveFailures(3) // Alert after the tenth failure. ->allowTotalFailures(10) ->run(function() { // })
Reporting instead of throwing
By default, Flaky will actually throw
the exception if it occurs outside of the bounds you have define. You can choose
to report
that exception instead of throw it, using Laravel's report
method.
Flaky::make('my-flaky-code') ->allowFailuresForAnHour() // Don't throw, but use `report()` instead. ->reportFailures() ->run(function() { // })
This allows you to still get the alert, but carry on processing if you need to. (This is helpful for loops or long-running processes.)
Retrying
If you want to immediately retry a bit of flaky code, you can use the retry
method, which uses Laravel's retry
helper under the hood. Any failures that happen as a part of the retry process don't count toward the total or
consecutive failures. If your function is retried the maximum times and does not succeed, then that counts as one
failure.
Flaky::make('my-flaky-code') ->allowFailuresForAnHour() // Retry 3 times, with 500ms between. ->retry(3, 500) ->run(function() { // })
You can also choose to retry a single type of exception
Flaky::make('my-flaky-code') ->allowFailuresForAnHour() // Only retry TimeoutExceptions ->retry(3, 500, TimeoutException::class) ->run(function() { // })
Or multiple types of exceptions
Flaky::make('my-flaky-code') ->allowFailuresForAnHour() // Only retry TimeoutExceptions and FooBarExceptions ->retry(3, 500, [TimeoutException::class, FooBarException::class]) ->run(function() { // })
Or pass through your own method:
Flaky::make('my-flaky-code') ->allowFailuresForAnHour() // Pass through your own $when callback ->retry(3, 500, function($exception) { // }) ->run(function() { // })
Accessing the result
Flaky will return a Result
class for your use.
$result = Flaky::make('my-flaky-code') ->allowFailuresForAnHour() ->run(function() { return 1; }); $result->value; // 1 $result->failed; // false $result->succeeded; // true $result->exception; // null. Would be populated if an exception was thrown. $result->throw(); // Throws an exception if present. Is a noop if not.
Flaky Commands
If you have entire commands that are Flaky, you can use the FlakyCommand
class as a convenience.
class FlakyTestCommand extends Command { protected $signature = 'flaky {--arg=} {--flag}'; public function handle() { FlakyCommand::make($this) ->allowFailuresForAnHour() ->run([$this, 'process']); } public function process() { throw new Exception('oops'); } }
This command will now have Flaky protections, but only when invoked by the scheduler. If you run this command manually, Flaky is not engaged and you'll get all the exceptions as you would have otherwise.
Flaky Command IDs
The FlakyCommand::make($this)
call will set the unique Flaky ID for you, based on the command's signature. By default,
every invocation of an command is treated under the same key. If you want to vary that based on user input, you can use
the varyOnInput
method.
class FlakyTestCommand extends Command { protected $signature = 'flaky {--arg=} {--flag}'; public function handle() { FlakyCommand::make($this) ->allowFailuresForAnHour() // Consider the `arg` and `flag` input when creating the unique ID. ->varyOnInput() ->run([$this, 'process']); } }
If you want to vary on particular input instead of all the input, you can pass an array of keys. This is useful for when
each --arg
should have its own flaky protections, but varying the --flag
shouldn't create a unique set of
protections.
class FlakyTestCommand extends Command { protected $signature = 'flaky {--arg=} {--flag}'; public function handle() { FlakyCommand::make($this) ->allowFailuresForAnHour() // Consider only the `arg` input when creating the unique ID. ->varyOnInput(['arg']) ->run([$this, 'process']); } }