umpirsky / extraload
Powerful ETL library.
Requires
- php: >=5.6
- symfony/event-dispatcher: ^3.0
Requires (Dev)
- behat/behat: v3.1.0rc1
- behat/mink-selenium2-driver: ^1.3
- doctrine/dbal: ^2.5
- phpspec/phpspec: ^2.4
- phpunit/phpunit: ^5.1
- symfony/console: ^3.0
- symfony/property-access: ^3.0
Suggests
- doctrine/dbal: Allows loading data into database using DBAL loader
- symfony/console: Allows displaying tabular data in console
- symfony/property-access: Allows applying transformer on property by path
This package is not auto-updated.
Last update: 2024-10-26 19:33:10 UTC
README
symfony upgrade fixer • twig gettext extractor • wisdom • centipede • permissions handler • extraload • gravatar • locurro • country list • transliterator
Extraload
Powerful ETL library.
Examples
Dumping CSV data into the console
Input data is given in csv format:
"99921-58-10-7", "Divine Comedy", "Dante Alighieri"
"9971-5-0210-0", "A Tale of Two Cities", "Charles Dickens"
"960-425-059-0", "The Lord of the Rings", "J. R. R. Tolkien"
"80-902734-1-6", "And Then There Were None", "Agatha Christie"
With:
(new DefaultPipeline( new CsvExtractor( new \SplFileObject('books.csv') ), new NoopTransformer(), new ConsoleLoader( new Table(new ConsoleOutput()) ) ))->process();
It can be dumped as table to console:
+---------------+--------------------------+------------------+
| 99921-58-10-7 | Divine Comedy | Dante Alighieri |
| 9971-5-0210-0 | A Tale of Two Cities | Charles Dickens |
| 960-425-059-0 | The Lord of the Rings | J. R. R. Tolkien |
| 80-902734-1-6 | And Then There Were None | Agatha Christie |
+---------------+--------------------------+------------------+
In this example NoopTransformer
is used, but various transformations can be applied. Transformers can also be chained using TransformerChain
.
Dumping a Doctrine query into the console
First of all make sure to load the fixtures into a database -- this example works with MySQL:
mysql> source /home/standard/projects/Extraload/fixtures/mysql/books.sql
So the following code:
(new DefaultPipeline( new QueryExtractor($conn, 'SELECT * FROM books'), new NoopTransformer(), new ConsoleLoader( new Table($output = new ConsoleOutput()) ) ))->process();
Will dump these results to the console:
+---------------+--------------------------+------------------+
| 99921-58-10-7 | Divine Comedy | Dante Alighieri |
| 9781847493583 | La Vita Nuova | Dante Alighieri |
| 9971-5-0210-0 | A Tale of Two Cities | Charles Dickens |
| 960-425-059-0 | The Lord of the Rings | J. R. R. Tolkien |
| 80-902734-1-6 | And Then There Were None | Agatha Christie |
+---------------+--------------------------+------------------+
Dumping a Doctrine prepared query into the console
The following code:
// ... $sql = "SELECT * FROM books WHERE author = :author"; $values = [ [ 'parameter' => ':author', 'value' => 'Dante Alighieri', 'data_type' => PDO::PARAM_STR // optional ] ]; (new DefaultPipeline( new QueryExtractor($conn, $sql, $values), new NoopTransformer(), new ConsoleLoader( new Table($output = new ConsoleOutput()) ) ))->process();
Will dump these results to the console:
+---------------+---------------+-----------------+
| 99921-58-10-7 | Divine Comedy | Dante Alighieri |
| 9781847493583 | La Vita Nuova | Dante Alighieri |
+---------------+---------------+-----------------+
Dumping a Doctrine query into a table
The following code:
// ... (new DefaultPipeline( new QueryExtractor($conn, 'SELECT * FROM books'), new NoopTransformer(), new DbalLoader($conn, 'my_books') ))->process();
Will dump the results into the my_books
table:
mysql> select * from my_books;
+----------------+--------------------------+----------------------------+
| isbn | title | author |
+----------------+--------------------------+----------------------------+
| 9781503262140 | Faust | Johann Wolfgang von Goethe |
| 978-0156949606 | The Waves | Virgina Woolf |
| 99921-58-10-7 | Divine Comedy | Dante Alighieri |
| 9781847493583 | La Vita Nuova | Dante Alighieri |
| 9971-5-0210-0 | A Tale of Two Cities | Charles Dickens |
| 960-425-059-0 | The Lord of the Rings | J. R. R. Tolkien |
| 80-902734-1-6 | And Then There Were None | Agatha Christie |
+----------------+--------------------------+----------------------------+
7 rows in set (0.00 sec)
See more examples.