czim / laravel-processor
Pipelined processor framework for Laravel.
Installs: 4 140
Dependents: 1
Suggesters: 0
Security: 0
Stars: 2
Watchers: 3
Forks: 0
Open Issues: 0
Requires
- php: ^7.1|^8.0
- czim/laravel-dataobject: ^1.0|^2.0
- illuminate/database: ^5.0|^6.0|^7.0|^8.0
- illuminate/support: ^5.0|^6.0|^7.0|^8.0
Requires (Dev)
- czim/laravel-repository: *
- mockery/mockery: ^1.0
- orchestra/testbench: ^5.0|^6.0
- phpunit/phpunit: ^8.0|^9.0
README
Framework for building modular, pipelined data processors.
The idea behind this is to have a configurable, clean and testable setup for complex data processing. It carries a lot of overhead, of course, so this only makes sense for fairly demanding (background) processing.
Usage example: This was constructed to better handle extensive product and debtor datasheet imports for a particular project. The imported data is converted to a relational database structure spanning many tables. Using pipelined processing, this can be done in discrete, separately testable process step classes that each have their own responsibility.
Install
Via Composer
$ composer require czim/laravel-processor
Usage
Extend Czim\PipelineProcessor
(or Czim\AbstractProcessor
) and write implementations for the abstract methods.
Processing is done by calling the process()
method on your class.
The parameter for this method must be an implementation of Czim\DataObject\Contracts\DataObjectInterface
(see the czim\laravel-dataobject for more information).
$processor = new Your\Processor(); $data = new Your\DataObject($someData); $result = $processor->process($data); if ( ! $result->success) { ... }
The returned result is an instance of Czim\Processor\DataObjects\ProcessorResult
.
This is a DataObject with a boolean success
property, as well as warnings
and errors
MessageBags by default.
Pipeline Processor
A pipeline processor consists of a series of process steps, which are executed in sequence.
A process context is passed into the pipeline and from step to step. It contains the data to be processed, cache, settings and such and its contents may be modified to affect the way subsequent steps behave.
When exceptions are thrown, the pipeline ends and the remaining steps are not executed.
To use it, extend Czim\PipelineProcessor
and add the following to your class:
/** * @return array */ protected function processSteps() { // Set a series of process step classnames and return it // these steps must extend Czim\Processor\Steps\AbstractProcessStep // or otherwise implement Czim\Processor\Contracts\ProcessStepInterface return [ Your\ProcessSteps\ClassNameHere::class, Your\ProcessSteps\AnotherClassNameHere::class, ]; }
For more configuration options, see the PipelineProcessor source.
Process Steps
Process steps can extend Czim\Processor\Steps\AbstractProcessStep
and implement the process()
method:
protected function process() { // Define your custom processing here. // The data object can be accessed through $this->data // and the process context through $this->context }
Process Context
A ProcessContext is an instance that represents the context in which the pipeline steps take place.
It stores the data passed into the process()
method. It can also store settings and a cache.
A ContextRepositoryTrait
for your own extensions is also provided,
in case you want to store repositories with the czim\laravel-repository package in the context.
Database Transaction
By default, the (main) pipeline is executed in a database transaction; it is comitted on succesfully completing all the steps, and rolled back on any exception thrown.
To run the process without a database transaction, set the following property in your PipelineProcessor
extension:
protected $databaseTransaction = false;
Simple Processor
If a pipeline is overkill, you can also use a simpler approach.
Extend Czim\AbstractProcessor
and add the following to your class:
protected function doProcessing() { // Define your custom processing here. // The data object can be accessed through $this->data // The result data object that will be returned can // be modified through $this->result }
For more configuration options, see the AbstractProcessor source.
To Do
- Make App/Container injectable, remove dependency on laravel's app() function
Contributing
Please see CONTRIBUTING for details.
Credits
License
The MIT License (MIT). Please see License File for more information.