keboola/output-mapping

Shared component for processing SAPI output mapping and importing data to KBC

27.3.0 2025-10-06 11:15 UTC

Requires

• php: >=8.2
• ext-json: *
• keboola/input-mapping: *@dev
• keboola/key-generator: *@dev
• keboola/php-datatypes: ^8.0
• keboola/php-file-storage-utils: ^0.2
• keboola/sanitizer: ^0.1.1
• keboola/slicer: *@dev
• keboola/staging-provider: *@dev
• keboola/storage-api-client: ^18.0
• keboola/storage-api-php-client-branch-wrapper: ^6.0
• microsoft/azure-storage-blob: ^1.5
• psr/log: ^2.0|^3.0
• symfony/config: ^5.4|^6.0|^7.0
• symfony/finder: ^5.4|^6.0|^7.0
• symfony/serializer: ^5.4|^6.0|^7.0
• symfony/yaml: ^5.4|^6.0|^7.0

Requires (Dev)

• brianium/paratest: ^6.11
• keboola/coding-standard: >=14.0
• keboola/php-temp: ^2.0
• monolog/monolog: ^3.0
• phpstan/phpstan: ^1.9
• phpstan/phpstan-phpunit: ^1.3
• phpstan/phpstan-symfony: ^1.2
• phpunit/phpunit: ^9.5
• sempro/phpunit-pretty-print: ^1.4
• symfony/dotenv: ^5.4|^6.0|^7.0

MIT

• Keboola

README

Output mapping library for Keboola Runner and Workspaces. Processes component outputs and uploads them to Keboola Storage API.

Tables:

• Local staging: Uploads tables as gzipped CSV files to Storage API
  • Slicing: Large CSV files are automatically split into multiple compressed chunks for parallel upload (via external slicer binary)
    • Requires: output-mapping-slice feature flag, default CSV format (, delimiter, " enclosure), no custom columns mapping
    • Sliced files must have columns or schema specified in configuration
• Workspace staging (Snowflake/BigQuery): Loads tables directly from workspace database objects (no file upload, no slicing)

Files:

• Uploads files as-is to Storage API File Storage (works with all staging types)

Development

Prepare local environment

Create .env.local file from this .env template and fill the required environment variables:

cp .env .env.local

Prepare resources

You need to provide the following environment variables:

• STORAGE_API_URL - The Keboola Storage API URL (e.g., https://connection.keboola.com)
• STORAGE_API_TOKEN - A non-admin token with "Full Access" to Files, Components & Buckets and Trash
• STORAGE_API_TOKEN_MASTER - An admin user token from the same project (with role admin)

Optional (for BigQuery-specific tests only):

• BIGQUERY_STORAGE_API_URL - BigQuery Storage API URL (e.g., https://connection.keboola.com)
• BIGQUERY_STORAGE_API_TOKEN - A non-admin token with "Full Access" to Files, Components & Buckets and Trash from a project with BigQuery backend

Available composer commands

Development commands:

• composer phpcs - Check code style
• composer phpcbf - Automatically fix code style issues
• composer phpstan - Run static analysis

Testing commands:

• composer tests - Run tests with PHPUnit
• composer paratests - Run tests in parallel with Paratest

License

MIT licensed, see LICENSE file.