flow-php / flow
PHP ETL - Extract Transform Load - Data processing framework
Installs: 3 993
Dependents: 0
Suggesters: 0
Security: 0
Stars: 488
Watchers: 9
Forks: 28
Open Issues: 17
Requires
- php: ~8.1.0 || ~8.2.0 || ~8.3.0
- composer-runtime-api: ^2.1
- ext-bcmath: *
- ext-dom: *
- ext-hash: *
- ext-json: *
- ext-mbstring: *
- ext-xml: *
- ext-xmlreader: *
- ext-xmlwriter: *
- ext-zlib: *
- coduo/php-humanizer: ^5.0
- doctrine/dbal: ^3.6 || ^4.0
- elasticsearch/elasticsearch: ^7.6|^8.0
- google/apiclient: ^2.13
- halaxa/json-machine: ^1.0
- jawira/case-converter: ^3.4
- meilisearch/meilisearch-php: ^1.1
- monolog/monolog: ^2.0||^3.0
- packaged/thrift: ^0.15.0
- php-http/discovery: ^1.0
- psr/http-client: ^1.0
- psr/http-message: ^1.0 || ^2.0
- psr/log: ^2.0 || ^3.0
- psr/simple-cache: ^1.0 || ^2.0 || ^3.0
- symfony/console: ^6.3 || ^7.0
- symfony/http-foundation: ~5.4.0 || ~6.4.0 || ~7
- webmozart/glob: ^3.0 || ^4.0
Requires (Dev)
- aeon-php/calendar: ^1.0
- cocur/slugify: ^4.5
- fakerphp/faker: ^1.23
- fig/log-test: ^1.1
- nikic/php-parser: ^4.18
- nyholm/psr7: ^1.8
- php-http/curl-client: ^2.2
- php-http/mock-client: ^1.5
- ramsey/uuid: ^4.5
- symfony/cache: ^6.2 || ^7.0
- symfony/dotenv: ^6.2 || ^7.0
- symfony/finder: ^6.3 || ^7.0
- symfony/uid: ^6.3 || ^7.0
Replaces
- flow-php/array-dot: 0.10.0
- flow-php/azure-sdk: 0.10.0
- flow-php/doctrine-dbal-bulk: 0.10.0
- flow-php/doctrine-dbal-bulk-tools: 0.10.0
- flow-php/dremel: 0.10.0
- flow-php/etl: 0.10.0
- flow-php/etl-adapter-avro: 0.10.0
- flow-php/etl-adapter-chartjs: 0.10.0
- flow-php/etl-adapter-csv: 0.10.0
- flow-php/etl-adapter-dbal-tools: 0.10.0
- flow-php/etl-adapter-doctrine: 0.10.0
- flow-php/etl-adapter-elasticsearch: 0.10.0
- flow-php/etl-adapter-filesystem: 0.10.0
- flow-php/etl-adapter-google-sheet: 0.10.0
- flow-php/etl-adapter-http: 0.10.0
- flow-php/etl-adapter-json: 0.10.0
- flow-php/etl-adapter-logger: 0.10.0
- flow-php/etl-adapter-logger-tools: 0.10.0
- flow-php/etl-adapter-meilisearch: 0.10.0
- flow-php/etl-adapter-parquet: 0.10.0
- flow-php/etl-adapter-text: 0.10.0
- flow-php/etl-adapter-xml: 0.10.0
- flow-php/filesystem: 0.10.0
- flow-php/filesytem-azure-bridge: 0.10.0
- flow-php/monolog-http-bridge: 0.10.0
- flow-php/parquet: 0.10.0
- flow-php/parquet-viewer: 0.10.0
- flow-php/rdsl: 0.10.0
- flow-php/snappy: 0.10.0
- flow-php/symfony-http-foundation-bridge: 0.10.0
README
Flow is a PHP-based, strongly typed data processing framework with a low memory footprint.
Flow PHP
Flow is the most advanced PHP ETL (Extract, Transform, Load) framework.
<?php declare(strict_types=1); use function Flow\ETL\Adapter\Parquet\{from_parquet, to_parquet}; use function Flow\ETL\DSL\{data_frame, lit, ref, sum, to_output, overwrite}; use Flow\ETL\Filesystem\SaveMode; require __DIR__ . '/vendor/autoload.php'; data_frame() ->read(from_parquet(__DIR__ . '/orders_flow.parquet')) ->select('created_at', 'total_price', 'discount') ->withEntry('created_at', ref('created_at')->cast('date')->dateFormat('Y/m')) ->withEntry('revenue', ref('total_price')->minus(ref('discount'))) ->select('created_at', 'revenue') ->groupBy('created_at') ->aggregate(sum(ref('revenue'))) ->sortBy(ref('created_at')->desc()) ->withEntry('daily_revenue', ref('revenue_sum')->round(lit(2))->numberFormat(lit(2))) ->drop('revenue_sum') ->write(to_output(truncate: false)) ->withEntry('created_at', ref('created_at')->toDate('Y/m')) ->saveMode(overwrite()) ->write(to_parquet(__DIR__ . '/daily_revenue.parquet')) ->run();
$ php daily_revenue.php +------------+---------------+ | created_at | daily_revenue | +------------+---------------+ | 2023/10 | 206,669.74 | | 2023/09 | 227,647.47 | | 2023/08 | 237,027.31 | | 2023/07 | 240,111.05 | | 2023/06 | 225,536.35 | | 2023/05 | 234,624.74 | | 2023/04 | 231,472.05 | | 2023/03 | 231,697.36 | | 2023/02 | 211,048.97 | | 2023/01 | 225,539.81 | +------------+---------------+ 10 rows
Community Contributions
Flow PHP is not just a tool, but a growing community of developers passionate about data processing and PHP. We strongly believe in the power of collaboration and welcome contributions of all forms. Whether you're fixing a bug, proposing a new feature, or improving our documentation, your input is invaluable to the growth of Flow PHP.
How You Can Contribute
- Submitting Bug Reports and Feature Requests: Encounter an issue or have an idea for an enhancement? Submit an issue on our GitHub repository. Please provide a clear description and, if possible, steps to reproduce the bug or details of the feature request.
- Code Contributions: Interested in directly impacting the development of Flow PHP? Check out our issue tracker for areas where you can contribute. From simple fixes to substantial feature additions, every bit of help is appreciated.
- Improving Documentation: Good documentation is key to any project's success. If you notice gaps, inaccuracies, or areas that could use better explanations, we encourage you to submit updates.
- Community Support: Help out fellow users by answering questions on our community channels, Stack Overflow, or other forums where Flow PHP users gather.
- Spread the Word: Share your experiences using Flow PHP, write blog posts, tutorials, or speak at meetups and conferences. Let others know how Flow PHP has helped in your projects!
- Leave a GitHub Star: If you find Flow PHP useful, consider giving it a star on GitHub. Your star is a simple yet powerful way to show support and helps others discover our project.
Contribution Guidelines
To ensure a smooth collaboration process, we've put together guidelines for contributing. Please take a moment to read our Contribution Guidelines before starting your work. This will help you understand our process and make contributing a breeze.
Questions?
If you have any questions about contributing, please feel free to reach out to us. We're more than happy to provide guidance and support.
Join us in shaping the future of data processing in PHP — every contribution, big or small, makes a significant difference!
GitHub Stars
Sponsors
Flow PHP is sponsored by:
- Blackfire - the best PHP profiling and monitoring tool!