keboola / data-loader
This package is abandoned and no longer maintained.
No replacement package was suggested.
Helper to load data in docker sandboxes
1.8.1
2020-09-25 07:35 UTC
Requires
- php: >=7.3
- ext-json: *
- keboola/datadir-tests: ^2.3
- keboola/input-mapping: ^9.4
- keboola/php-utils: ^2.3
- keboola/syrup-php-client: ^5.4
- monolog/monolog: ^1.22
Requires (Dev)
- jakub-onderka/php-parallel-lint: ^1.0
- keboola/coding-standard: >=7.0
- phpstan/phpstan-shim: ^0.11
- phpunit/phpunit: ^7.0
- dev-master
- 1.8.1
- 1.8.0
- 1.7.4
- 1.7.3
- 1.7.2
- 1.7.1
- 1.7.0
- 1.6.0
- 1.5.3
- 1.5.2
- 1.5.1
- 1.5.0
- 1.4.1
- 1.4.0
- 1.3.0
- 1.2.5
- 1.2.4
- 1.2.3
- 1.2.2
- 1.2.1
- 1.2.0
- 1.1.0
- 1.0.3
- 1.0.2
- 1.0.1
- 1.0.0
- 0.10.4
- 0.10.3
- 0.10.2
- 0.10.1
- 0.10.0
- 0.9.0
- 0.8.2
- 0.8.1
- 0.8.0
- 0.7.0
- 0.6.0
- 0.5.0
- 0.4.2
- 0.4.1
- 0.4.0
- 0.3.4
- 0.3.3
- 0.3.2
- 0.3.1
- 0.3.0
- 0.2.2
- 0.2.1
- 0.2.0
- 0.1.1
- 0.1.0
- 0.0.14
- 0.0.13
- 0.0.12
- 0.0.11
- 0.0.10
- 0.0.9
- 0.0.8
- 0.0.7
- 0.0.6
- 0.0.5
- 0.0.4
- 0.0.3
- 0.0.2
- 0.0.1
- dev-odin-branch-0-10-1
- dev-Branch_0.10.1
- dev-changedSince
- dev-changed-since
- dev-trans-config
- dev-storage-api-url
This package is auto-updated.
Last update: 2024-04-26 20:38:55 UTC
README
Application which loads data from Storage API and stores them in a data folder. It is used to load data to Docker Sandboxes. There are two modes of operation:
- A transformation sandbox is being created -
KBC_CONFIG_ID
and related variables need to be provided. - A plain sandbox is being created -
KBC_EXPORT_CONFIG
variable needs to be provided.
Usage
The following environment variables are used for configuration (see .env.template):
KBC_EXPORT_CONFIG
- Serialized JSON configuration of input mapping, see description.KBC_TOKEN
- Storage API token.KBC_DATADIR
- Optional target directory, defaults to/data/
KBC_RUNID
- Optional RunID, that appends to the logKBC_STORAGEAPI_URL
- Optional Storage API URL, if it's different fromhttps://connection.keboola.com
KBC_COMPONENT_ID
- optional Id of the transformation component (only for V2 transformations).KBC_CONFIG_ID
- Id of the transformation configuration.KBC_CONFIG_VERSION
- Version of theKBC_CONFIG_ID
transformation.KBC_ROW_ID
- optional Id of a row of theKBC_CONFIG_ID
transformation. (only for legacy transformations)KBC_VARIABLE_VALUES_ID
- Optional, id of the variable values (only for v2 transformations)KBC_VARIABLE_VALUES_DATA
- Optional, array of variable values data (only for v2 transforamtions)
Either KBC_EXPORT_CONFIG or the combination of KBC_CONFIG_ID
, KBC_CONFIG_VERSION
, KBC_ROW_ID
is required.
Run the loader with php src/run.php
.
Development
Init
git clone https://github.com/keboola/data-loader
cd data-loader
docker-compose build
docker-compose run --rm dev composer install
Tests
Create .env
file:
KBC_TEST_URL=https://connection.keboola.com/
KBC_TEST_TOKEN=
Run tests:
docker-compose run --rm dev composer ci
License
MIT licensed, see LICENSE file.