keboola / extractor-bundle
Keboola Extractor Development Bundle
Requires
- php: >=5.4.0
- guzzlehttp/retry-subscriber: ~0.1.2
- keboola/json-parser: ~1.0.8
- keboola/php-csvtable: ~0.1.3
- keboola/php-temp: ~0.1.0
- keboola/php-utils: ~0.1.8
- syrup/component-bundle: ~1.13.2
Requires (Dev)
- phpunit/phpunit: 3.7.*
- dev-master
- 2.0.x-dev
- 1.2.1
- 1.2.0
- 1.1.x-dev
- 1.1.7
- 1.1.6
- 1.1.5
- 1.1.4
- 1.1.3
- 1.1.2
- 1.1.1
- 1.1.0
- 1.0.7
- 1.0.6
- 1.0.5
- 1.0.4
- 1.0.3
- 1.0.2
- 1.0.1
- 1.0.0
- 1.0.0-beta13
- 1.0.0-beta12
- 1.0.0-beta11
- 1.0.0-beta10
- 1.0.0-beta9
- 1.0.0-beta8
- 1.0.0-beta7
- 1.0.0-beta6
- 1.0.0-beta5
- 1.0.0-beta4
- 1.0.0-beta3
- 1.0.0-beta2
- 1.0.0-beta1
- 1.0.0-beta
- 1.0.0-alpha1
- 1.0.0-alpha
- 0.11.4
- 0.11.3
- 0.11.2
- 0.11.1
- 0.11.0
- 0.10.3
- 0.10.2
- 0.10.1
- 0.10.0
- 0.9.13
- 0.9.12
- 0.9.11
- 0.9.10
- 0.9.9
- 0.9.8
- 0.9.7
- 0.9.6
- 0.9.5
- 0.9.4
- 0.9.3
- 0.9.2
- 0.9.1
- 0.9.0-alpha
- 0.8.12
- 0.8.11
- 0.8.10
- 0.8.9
- 0.8.8
- 0.8.7
- 0.8.6
- 0.8.5
- 0.8.4
- 0.8.3
- 0.8.2
- 0.8.1
- 0.8.0
- 0.7.6
- 0.7.5
- 0.7.4
- 0.7.3
- 0.7.2
- 0.7.1
- 0.7.0
- 0.6.1
- 0.6.0
- 0.5.2
- 0.5.1
- 0.5.0
- 0.4.5
- 0.4.4
- 0.4.3
- 0.4.2
- 0.4.1
- 0.4.0
- 0.3.0
- 0.2.3
- 0.2.2
- 0.2.1
- 0.2.0
- 0.1.8
- 0.1.7
- 0.1.6
- 0.1.5
- 0.1.4
- 0.1.3
- 0.1.2
- 0.1.1
- 0.1.0
- 0.0.3
- 0.0.2
- 0.0.1
This package is auto-updated.
Last update: 2019-02-20 18:17:14 UTC
README
This readme assumes knowledge of Syrup environment. Please refer to Syrup's documentation for basic informations!
0. Environment init
Connect to elasticsearch
Ensure you can connect to Elasticsearch and the databases specified in parameters_shared.yml!
Skip if yer developing on Devel!
KBC Devel:
ssh kbc-devel-02.keboola.com -L 9200:VPC-ELB-ElasticSearch-Syrup-Test-651343387.us-east-1.elb.amazonaws.com:9200 -L 3306:localhost:3306
1. Create the extractor
$ git clone git@github.com:keboola/extractor-generator.git
$ php extractor-generator/generate.php
Follow interactive interface
Remove generator
$ rm -rf extractor-generator
2. Prepare the Elasticsearch Index
php ./vendor/keboola/syrup/app/console syrup:create-index
3. Get to work!
- Edit YourAppExtractor.php and YourAppExtractorJob.php
Edit Resources/config/services.yml if you need to use parameters.yml values in the application:
Example:
services.yml:
ex_twitter.extractor: class: Keboola\TwitterExtractorBundle\TwitterExtractor arguments: ['%twitter%']
TwitterExtractor.php
/** @var array */ protected $apiKeys; public function __construct($twitter) { $this->apiKeys = $twitter; }
parameters.yml
twitter: api-key: WoWSuchApiKey16777216489 api-secret: OMGICantBelieveH0wS3cr3tIAmTh4t5cr42yTr00l0l0lOhai
I found it easier for development to print out messages to stdout when using the run-job command. That can be done by editing the monolog parameter in vendor/keboola/syrup/app/config/config_dev.yml
:
monolog:
handlers:
syslog:
type: stream
path: php://stdout
level: debug
bubble: false
For more information about Syrup bundles check out the documentation