tobento/app-import-export

App import and export manager for bulk data processing with readers, writers, mappings, and job lifecycle hooks.

Maintainers

Package info

github.com/tobento-ch/app-import-export

Homepage

pkg:composer/tobento/app-import-export

Statistics

Installs: 0

Dependents: 0

Suggesters: 0

Stars: 0

Open Issues: 0

Security

Advisories: 0

Aikido package health analysis

2.0 2026-06-23 15:28 UTC

Requires

Requires (Dev)

MIT 947ebfa977e83a5bbbbcb743b6dc75bac7312ddc

packageimportappexporttobento

This package is auto-updated.

Last update: 2026-06-23 15:31:58 UTC

README

The app import and export provides the following features and uses the Read-Write Service for its readers, writers, and modifiers:

  • Import Export Feature - Provides a dedicated UI where all available readers and writers are listed, can be configured with options, and then executed.
  • Job Results Feature - Stores processed rows from import or export jobs. You can define via hooks which rows should be stored. Results can be edited inline and re-added, and they can also be exported using any registered writer.
  • Exported Files Feature - View and download all generated export files in one place.
  • CRUD Integration - Trigger import and export actions directly from CRUD index pages when enabled.

Zero-Config Bootstrapping

The Import & Export system works out-of-the-box with sensible defaults.

Simply register the Import Export Boot in your application, and the full import/export management UI becomes available immediately - no additional configuration required.

All built-in readers, writers and hooks are automatically discovered, listed in the UI, and ready to use. You can run imports, generate exports, inspect job results, and download generated files without writing any custom code.

You only need to customize configuration if you want to override defaults such as:

  • available readers or writers
  • storage locations for exported files
  • job result retention behavior
  • permissions and access control
  • custom reader or writer registries
  • CRUD Integration

Table of Contents

Getting Started

Add the latest version of the app import-export project running this command.

composer require tobento/app-import-export

Requirements

  • PHP 8.4 or greater

Highlights

  • runs fully in the background using queues, so imports and exports never block the request
  • processes data in time-budgeted chunks, allowing safe handling of large datasets
  • automatically resumes long-running jobs without losing progress
  • integrates naturally with existing CRUD controllers through action definitions
  • supports multiple output formats via service-read-write (CSV, JSON, PDF, repository, etc.)
  • easily extendable with custom readers, writers, filters, modifiers, and registry entries
  • keeps the system stateless by reconstructing readers and writers from registry identifiers
  • provides a clean UI for triggering imports/exports, reviewing job results, and downloading files
  • supports inline editing and reprocessing of import results through the Job Results Feature
  • allows triggering import/export actions directly from CRUD index pages when enabled

Documentation

App

Check out the App Skeleton if you are using the skeleton.

You may also check out the App to learn more about the app in general.

Import Export Boot

The import/export boot does the following:

use Tobento\App\AppFactory;
use Tobento\App\ImportExport\HooksInterface;
use Tobento\App\ImportExport\RegistriesInterface;
use Tobento\App\ImportExport\Queue;
use Tobento\App\ImportExport\Repo;

// Create the app
$app = new AppFactory()->createApp();

// Add directories:
$app->dirs()
    ->dir(realpath(__DIR__.'/../'), 'root')
    ->dir(realpath(__DIR__.'/../app/'), 'app')
    ->dir($app->dir('app').'config', 'config', group: 'config')
    ->dir($app->dir('root').'public', 'public')
    ->dir($app->dir('root').'vendor', 'vendor');

// Adding boots
$app->boot(\Tobento\App\ImportExport\Boot\ImportExport::class);
$app->booting();

// Implemented interfaces:
$hooks = $app->get(HooksInterface::class);
$registries = $app->get(RegistriesInterface::class);
$jobRepository = $app->get(Repo\JobRepositoryInterface::class);
$jobResultRepository = $app->get(Repo\JobResultRepositoryInterface::class);
$exportedFileRepository = $app->get(Repo\ExportedFileRepositoryInterface::class);
$queueHandler = $app->get(Queue\QueueHandlerInterface::class);

// Run the app
$app->run();

You may install the App Backend and boot the import export in the backend app.

Import Export Config

The configuration for the import-export is located in the app/config/import-export.php file at the default App Skeleton config location. Here you can configure features, registries, hooks and more.

Http Error Handler Boot

The Import & Export package provides an HTTP-level error handler that converts internal import/export exceptions into clean, user-friendly responses. This boot is optional but recommended when you want clean handling of reader, writer, and job-related errors during HTTP requests.

The example below shows how to register the error handler together with the main Import & Export boot:

use Tobento\App\AppFactory;
use Tobento\App\ImportExport\HooksInterface;
use Tobento\App\ImportExport\RegistriesInterface;
use Tobento\App\ImportExport\Queue;
use Tobento\App\ImportExport\Repo;

// Create the app
$app = new AppFactory()->createApp();

// Add directories:
$app->dirs()
    ->dir(realpath(__DIR__.'/../'), 'root')
    ->dir(realpath(__DIR__.'/../app/'), 'app')
    ->dir($app->dir('app').'config', 'config', group: 'config')
    ->dir($app->dir('root').'public', 'public')
    ->dir($app->dir('root').'vendor', 'vendor');

// Adding boots
$app->boot(\Tobento\App\ImportExport\Boot\HttpErrorHandler::class);
$app->boot(\Tobento\App\ImportExport\Boot\ImportExport::class);
$app->booting();

// Run the app
$app->run();

The HttpErrorHandler boot registers handlers for exceptions raised by the Import & Export module, such as invalid readers or writers, missing storages, or job configuration errors. It ensures these issues are returned as structured HTTP responses instead of raw exceptions, providing clearer feedback and consistent error output across the application.

Features

Import Export Feature

This feature provides an import/export page where users can create and run import/export operations using the configured registries and hooks.

Config

In the config file you can configure this feature:

'features' => [
    new Feature\ImportExport(
        // A menu name to show the link or null if none.
        menu: 'main',
        menuLabel: 'Import & Export Jobs',
        // A menu parent name (e.g. 'system') or null if none.
        menuParent: null,
        
        // you may disable the ACL while testing for instance,
        // otherwise only users with the right permissions can access the page.
        withAcl: false,
    ),
],

ACL Permissions

  • import-export User can access import/export
  • import-export.create User can create import/export
  • import-export.edit User can edit import/export
  • import-export.delete User can delete import/export
  • import-export.run User can run import/export

When using the App Backend, you can assign the permissions in the roles or users page.

Workflow

An import or export operation follows a simple workflow:

  1. Create a new job
    The user selects a reader and writer registry and configures their options using the generated CRUD fields.

  2. Configure mappings, modifiers and hooks
    Depending on the registries used, the user may define field mappings, enable modifiers that transform the data during processing, and attach hooks that react to events such as successful, skipped, or failed rows.

  3. Run the job
    The job is executed in the background. The reader loads the data, the writer processes it, and any registered hooks are triggered.

  4. Review results
    After completion, the job displays its status, duration, processed row counts, and any messages generated during processing.
    If a file-storage writer is used, the resulting file can be downloaded from the Exported Files Feature page.
    Hooks may also store individual rows (successful, skipped, or failed), which can then be viewed, exported, or edited in the Job Results Feature, depending on the selected hook options.

This workflow applies to both import and export operations, depending on the selected reader and writer registries.

All import/export jobs are executed through the queue handler configured in the app config file.

'interfaces' => [
    Queue\QueueHandlerInterface::class =>
    static function(QueueInterface $queue): Queue\QueueHandlerInterface {
        return new Queue\TimeBudgetQueueHandler(
            queue: $queue,
            
            // You may define a specific queue name. If null, the default queue is used.
            queueName: null,
            
            // Maximum processing time per execution (in seconds)
            timeBudget: 20,
        );
    },
],

To monitor queued jobs, view their status, or inspect failures and retries, you may also install the tobento-ch/app-job package, which provides a simple interface for observing queue activity.

Job Status Lifecycle

Import and export jobs move through a clear set of statuses that reflect their configuration state and processing progress. This ensures that jobs cannot be executed until they are fully configured, preventing issues such as incomplete mappings or invalid modifiers.

unconfigured
The job has been created but is not yet fully configured.
Mappings, modifiers, or hooks may still be missing or invalid.
Jobs in this state cannot be queued or executed.

ready
All required configuration is complete and valid.
Mappings, modifiers, and hooks are fully defined, and the job can safely be executed.
When the user presses Run, the job transitions from ready to queued.

queued
The job has been dispatched to the queue and is waiting to be processed by the queue worker.

processing
The queue worker is actively executing the job.
The reader loads data, the writer processes it, and all registered hooks are triggered.
The Job Lifecycle hook updates status, row counts, and timestamps during this phase.

completed
The job finished successfully.
All results, row counts, and messages are available for review.

failed
The job encountered an exception during processing.
Failure details and partial results (if any) can be reviewed by installing the tobento-ch/app-job package

Job Results Feature

This feature provides a job results page where users can review processed rows (successful, failed, skipped), edit values inline using the table editor, and re-add corrected rows for processing.
On the import/export page, users can define via hooks which rows should be stored. Stored rows can then be viewed, exported, edited, or downloaded within this feature.

Config

In the config file you can configure this feature:

'features' => [
    new Feature\JobResults(
        // A menu name to show the link or null if none.
        menu: 'main',
        menuLabel: 'Job Results',
        // A menu parent name (e.g. 'system') or null if none.
        menuParent: null,
        
        // you may disable the ACL while testing for instance,
        // otherwise only users with the right permissions can access the page.
        withAcl: false,
    ),
],

ACL Permissions

The Job Results feature uses the following permission:

import-export.results

This permission allows a user to:

  • access the Job Results overview page
  • view stored rows for completed jobs (successful, failed, skipped)
  • edit stored rows inline using the table editor
  • export or download stored rows
  • select rows for reprocessing
  • trigger the ReprocessBulkAction (one new job per original job)

If the permission is missing, the user cannot open the Job Results page or interact with stored rows.

Exported Files Feature

This feature provides a page where users can view and download all generated export files in one place.

Users can:

  • View a list of all exported files
  • Display individual files
  • Download individual files
  • Bulk delete selected or filtered files
  • Bulk download selected or filtered files as a ZIP archive

Config

In the config file you can configure this feature:

'features' => [
    new Feature\ExportedFiles(
        // A menu name to show the link or null if none.
        menu: 'main',
        menuLabel: 'Exported Files',
        // A menu parent name (e.g. 'system') or null if none.
        menuParent: null,
        
        // Automatically register the signed media features required
        // for displaying and downloading exported files.
        autoRegisterMediaFeatures: true,
        
        // The number of minutes signed URLs remain valid.
        // Used for both display and download links.
        signedUrlExpiresInMinutes: 5,
        
        // you may disable the ACL while testing for instance,
        // otherwise only users with the right permissions can access the page.
        withAcl: false,
    ),
],

ACL Permissions

  • import-export.exported-files User can access exported files

If withAcl is set to false, this permission is automatically granted.

Warning
Disabling ACL removes all access restrictions. Do not use this setting in production.

Signed Media Features

If autoRegisterMediaFeatures is enabled (default), the feature automatically registers:

both with: supportedStorages: ['uploads-private']

These are required to securely display and download exported files stored in the uploads-private storage.

If you prefer to manage these features manually, set:

autoRegisterMediaFeatures: false

Signed URL Expiration

You can control how long the signed display/download URLs remain valid:

signedUrlExpiresInMinutes: 5

This value is passed to the controller and used when generating signed URLs.

Imported Files Feature

The Imported Files feature offers a central place where users can view, download, and manage all files uploaded by your file-upload readers. It is particularly helpful for file-driven import workflows (e.g., CSV uploads), as it provides complete transparency over the stored import files and allows users to keep the import directory clean and organized.

Users can:

  • View a list of all imported files
  • Display individual files
  • Download individual files
  • Bulk delete selected or filtered files
  • Bulk download selected or filtered files as a ZIP archive

Config

In the config file you can configure this feature:

'features' => [
    new Feature\ImportedFiles(
        // A menu name to show the link or null if none.
        menu: 'main',
        menuLabel: 'Imported Files',
        // A menu parent name (e.g. 'system') or null if none.
        menuParent: null,
        
        // Automatically register the signed media features required
        // for displaying and downloading imported files.
        autoRegisterMediaFeatures: true,
        
        // The number of minutes signed URLs remain valid.
        // Used for both display and download links.
        signedUrlExpiresInMinutes: 5,
        
        // Disable ACL during development if needed.
        // When enabled, only users with the correct permissions
        // can access the Imported Files page.
        withAcl: false,
    ),
],

ACL Permissions

  • import-export.imported-files User can access imported files

If withAcl is set to false, this permission is automatically granted.

Warning
Disabling ACL removes all access restrictions. Do not use this setting in production.

Signed Media Features

If autoRegisterMediaFeatures is enabled (default), the feature automatically registers:

both with: supportedStorages: ['uploads-private']

These are required to securely display and download imported files stored in the uploads-private storage.

If you prefer to manage these features manually, set:

autoRegisterMediaFeatures: false

Signed URL Expiration

You can control how long the signed display/download URLs remain valid:

signedUrlExpiresInMinutes: 5

This value is passed to the controller and used when generating signed URLs.

Available Registries

Registries define the readers and writers available for import and export.
Each registry configures the CRUD fields used to edit its options, defines optional modifiers, and is responsible for creating the actual reader or writer instance used during processing.

Registries build on the concepts provided by the Read-Write Service, which supplies readers, writers, and modifiers.

Language configuration

Writers include built-in support for language handling. Each writer registry provides fields for selecting the language mode and the language to use during writing. By default, the available languages are resolved from the resources area when it exists, or otherwise from the default languages of the current application container, ensuring that writers respect the multilingual configuration of each environment. Registries may customize the available languages by overriding the resolveLanguages() method on a specific writer registry.

The following shows the default behavior used by all writer registries.

use Psr\Container\ContainerInterface;
use Tobento\App\ImportExport\Registry;
use Tobento\Service\Language\AreaLanguagesInterface;
use Tobento\Service\Language\LanguagesInterface;

class ProductRepositoryWriterRegistry extends Registry\RepositoryWriter
{
    protected function resolveLanguages(ContainerInterface $container): LanguagesInterface
    {
        // Default implementation:
        $areaLanguages = $container->get(AreaLanguagesInterface::class);
        
        if ($areaLanguages->has('resources')) {
            return $areaLanguages->get('resources');
        }
        
        return $container->get(LanguagesInterface::class);
    }
}

Customizing available languages

To provide a custom language set for a specific writer registry, override the resolveLanguages() method. The method must return a LanguagesInterface instance:

use Psr\Container\ContainerInterface;
use Tobento\App\ImportExport\Registry;
use Tobento\Service\Language\LanguageFactory;
use Tobento.Service\Language\Languages;
use Tobento\Service\Language\LanguagesInterface;

class ProductRepositoryWriterRegistry extends Registry\RepositoryWriter
{
    protected function resolveLanguages(ContainerInterface $container): LanguagesInterface
    {
        $factory = new LanguageFactory();

        return new Languages(
            $factory->createLanguage(locale: 'en', default: true),
            $factory->createLanguage(locale: 'de', fallback: 'en'),
            $factory->createLanguage(locale: 'de-CH', fallback: 'de'),
            $factory->createLanguage(locale: 'fr', fallback: 'en', active: false),
            $factory->createLanguage(locale: 'it', fallback: 'de', active: false),
        );
    }
}

For more detail see: App Languages

CRUD Controller Writer Registry

This registry provides a writer that uses any CRUD Controller as a write target for import-export jobs. It validates and writes data through the controller's repository using the CrudWriteRepository, ensuring that controller field rules and write logic are respected.

Config

In the config file you can configure this registry:

'registries' => [
    'products.controller.writer' => new Registry\CrudControllerWriter(
        // Set a display name:
        name: 'Products',
        
        // Set the CRUD controller:
        controller: ProductCrudController::class,
        
        // Define the fields that should be writable:
        withFields: ['sku', 'status', 'title', 'image', 'created_at'],
        
        // Fields that represent uploaded files and should be processed
        // by UploadedFileModifier before writing. See the modifier for details.
        withFileFields: ['image'],
    ),
]

Writer Behavior

The CRUD Controller Writer performs the following steps:

  • Resolves the configured CRUD controller from the container
  • Applies the ColumnMap defined in the job
  • Creates a CrudWriteRepository that wraps the controller's repository and applies controller field rules
  • Restricts writable fields to the fields defined in withFields
  • Applies dry-run mode by replacing the repository with a NullRepository (no write operations)
  • Creates a RepositoryWriter that writes rows using the controller's write logic
  • Uses the controller's entity ID name to determine update behavior (create vs. update)

This writer uses the following underlying components:

UI Options

The following option is available when configuring this writer in the UI:

Field Description
Dry run (no write operations) When enabled, the writer performs all processing steps but does not execute any create or update operations. Useful for testing and validating the import without modifying any data.

Customizing the CRUD Controller Writer

You can extend CrudControllerWriter to:

  • add custom UI fields
  • modify how the writer is created
  • add custom modifiers
  • change write behavior

Example: Adding a custom option, writer behavior, and modifiers

use Psr\Container\ContainerInterface;
use Tobento\App\Crud\Action\ActionInterface;
use Tobento\App\Crud\CrudWriteRepository;
use Tobento\App\Crud\Field;
use Tobento\App\Crud\Field\FieldInterface;
use Tobento\App\Crud\Field\FieldsInterface;
use Tobento\App\ImportExport\Registry\CrudControllerWriter;
use Tobento\App\ImportExport\Exception\WriterCreateException;
use Tobento\App\ImportExport\Modifier\DotNotationToArrayModifier;
use Tobento\App\ImportExport\Modifier\UploadedFileModifier;
use Tobento\App\ImportExport\Repo\JobEntityInterface;
use Tobento\Service\ReadWrite\Modifier;
use Tobento\Service\ReadWrite\ModifiersInterface;
use Tobento\Service\ReadWrite\RowInterface;
use Tobento\Service\ReadWrite\Writer;
use Tobento\Service\ReadWrite\WriterInterface;
use function Tobento\App\Translation\trans;

class CustomProductWriter extends CrudControllerWriter
{
    public function configureFields(ActionInterface $action): iterable|FieldsInterface
    {
        // Default fields (e.g. dry-run):
        yield from parent::configureFields($action);

        // Custom option:
        yield new Field\Radios(
            name: 'writer.skip_validation',
            label: trans('Skip validation'),
        )
            ->group(trans('Writer'))
            ->options(['0' => trans('No'), '1' => trans('Yes')])
            ->selected(value: '0', action: 'create|edit');
    }

    public function createWriter(ContainerInterface $container, JobEntityInterface $jobEntity): WriterInterface
    {
        $writer = parent::createWriter($container, $jobEntity);

        // Example: wrap or adjust writer based on custom option:
        if ($jobEntity->get('writer.skip_validation') === '1') {
            $controller = $container->get($this->controller);
            
            // Use the controller's raw repository without validation (custom behavior)
            return new Writer\RepositoryWriter(
                repository: $controller->repository(),
                idName: $controller->entityIdName(),
                columns: $this->withFields,
            );
        }

        return $writer;
    }

    public function createModifiers(ContainerInterface $container, JobEntityInterface $jobEntity): ModifiersInterface
    {
        // Start with the default modifiers:
        $modifiers = new Modifier\Modifiers(
            new Modifier\ColumnMap(map: $jobEntity->columnMap()),
            new DotNotationToArrayModifier(),
            new UploadedFileModifier($this->withFileFields(), $container),
        );

        // Add a custom modifier, e.g. trim title:
        // Note: Reader applies reader-specific modifiers.
        // ColumnMap and writer-specific modifiers are applied here.
        $modifiers->add(
            new Modifier\Format(
                field: 'title',
                formatter: function ($value, RowInterface $row): string {
                    return strtolower(trim((string)$value));
                }
            )
        );

        return $modifiers;
    }
}

Registering the custom writer

'registries' => [
    'products.custom.writer' => new CustomProductWriter(
        name: 'Products (Custom)',
        controller: ProductCrudController::class,
        withFields: ['sku', 'status', 'title', 'created_at'],
    ),
]

CSV File Storage Writer Registry

This registry provides a writer that generates CSV files using a FileStorage resource.
It defines the CRUD fields for configuring the output file (filename, delimiter, enclosure, escape, BOM), applies column mapping and language modifiers, and creates the CSV writer that stores the generated file in the configured storage and folder.

Config

In the config file you can configure this registry:

'registries' => [
    'csv.file.storage.writer' => new Registry\CsvFileStorageWriter(
        // Set a name:
        name: 'CSV File',

        // File storage where processed files are stored:
        storageName: 'uploads-private', // default

        // Folder inside the storage where files are placed:
        storageFolder: 'exports', // default
    ),
]

Writer Behavior

The CSV File Storage Writer performs the following steps:

  • Applies the ColumnMap defined in the job
  • Applies language modifiers (if supported and configured)
  • Applies any writer-specific modifiers
  • Creates a CSV resource writer (CsvResource) with the configured delimiter, enclosure, escape, and BOM settings
  • Stores the generated file in the configured storage and folder
  • Registers the exported file in the ExportedFileRepository

This writer uses two underlying components from the Read-Write service:

UI Options

The following options are available when configuring this writer in the UI:

Field Description
Filename Base filename (without extension). Must contain only letters, numbers, spaces, dots, underscores, and dashes.
Delimiter Character used to separate fields. Options: comma, semicolon, tab, pipe.
Enclosure Character used to wrap field values. Options: double quote, single quote.
Escape Character Character used to escape enclosure characters.
Write BOM Whether to write a UTF-8 BOM at the beginning of the file.
Language Fields Additional language‑related options provided by the system (e.g., locale selection), depending on your application setup.

CSV File Upload Reader Registry

This registry provides a reader that imports data from an uploaded CSV file.
It defines the CRUD fields for uploading and previewing the file, validates allowed extensions, and creates the CSV reader instance used during the processing workflow.

Config

In the config file you can configure this registry:

'registries' => [
    'csv.file.reader' => new Registry\CsvFileUploadReader(
        // Set a name:
        name: 'CSV File Upload',
        
        // Limit the maximum upload size (in KB) or null for no limit:
        maxFileSizeInKb: null, // default
        
        // File storage where uploaded files are stored:
        storageName: 'uploads-private', // default
        
        // Folder inside the storage where files are placed:
        storageFolder: 'imports', // default
    ),
]

If you change the file storage, you must also adjust the imported file repository so it points to the correct storage:

'interfaces' => [
    Repo\ImportedFileRepositoryInterface::class =>
    static function(StoragesInterface $storages): Repo\ImportedFileRepositoryInterface {
        return new Repo\ImportedFileRepository(
            storage: $storages->get('uploads-private'),
            rootFolder: 'imports',
        );
    },
]

Reader Behavior

The CSV File Upload Reader performs the following steps:

  • Validates the uploaded file using a CSV-specific validator
    (allowed extension, filename rules, max size, strict characters)
  • Stores the uploaded file in the configured storage and folder
  • Reads user-selected CSV settings (delimiter, enclosure, escape)
  • Auto-detects the delimiter when the user selects auto
  • Normalizes the delimiter to ensure it is a single valid character
  • Creates a CSV stream reader (CsvStream) using the uploaded file's stream
  • Provides row-by-row CSV parsing for the import-export pipeline

This reader uses the following underlying components:

UI Options

The following options are available when configuring this reader in the UI:

Field Description
File Source Uploads a CSV file. The field validates file extension (.csv), filename length, allowed characters, and maximum file size.
Delimiter (edit/update only) Character used to separate columns. Supports auto-detection, comma, semicolon, tab, pipe, or colon.
Enclosure (edit/update only) Character used to wrap values containing delimiters. Typically double-quote or single-quote.
Escape Character (edit/update only) Character used to escape enclosure characters inside values. Usually backslash or double-quote.

File Storage Reader Registry

This registry provides a reader that imports data from files stored in a configured FileStorage.
It lists available files in the storage (optionally including subfolders), filters them by allowed extensions, and creates the appropriate reader based on the selected file type (json, ndjson, or csv).

Config

In the config file you can configure this registry:

'registries' => [
    'storage.file.reader' => new Registry\FileStorageReader(
        // File storage where importable files are located:
        storageName: 'uploads-private',

        // Folder inside the storage (null = root):
        storageFolder: 'imports',

        // Whether to include subfolders when listing files:
        includeSubfolders: true, // default

        // Sorting: null, 'name', or 'lastModified':
        sortFilesBy: 'name', // default

        // Display name:
        name: 'Stored Files',

        // Allowed file extensions:
        allowedFileExtensions: ['json', 'ndjson', 'csv'], // default
    ),
]

If you change the file storage, you must also adjust the imported file repository so it points to the correct storage:

'interfaces' => [
    Repo\ImportedFileRepositoryInterface::class =>
    static function(StoragesInterface $storages): Repo\ImportedFileRepositoryInterface {
        return new Repo\ImportedFileRepository(
            storage: $storages->get('uploads-private'),
            rootFolder: 'imports',
        );
    },
]

Reader Behavior

The File Storage Reader performs the following steps:

  • Resolves the configured storage from the container
  • Lists files in the configured folder (optionally including subfolders)
  • Filters files by allowed extensions (json, ndjson, csv)
  • Sorts files by name or last modified timestamp (if configured)
  • Loads the selected file and ensures it exists and has a readable stream
  • Creates the appropriate reader based on the file extension:
    • JsonStream for .json
    • NdJsonStream for .ndjson
    • CsvStream for .csv
  • Provides row-by-row reading for the import-export pipeline
  • Applies no modifiers (this reader does not use ColumnMap or language modifiers)

This reader uses the following underlying components:

UI Options

The following option is available when configuring this reader in the UI:

Field Description
File Selects a file from the configured storage. Files are filtered by allowed extensions and optionally sorted by name or last modified timestamp.

HTML File Storage Writer Registry

This registry provides a writer that generates HTML files using a FileStorage resource.
It supports configurable templates, titles, descriptions, image rendering, and language-aware output.
The writer uses the HtmlResource writer from the Read-Write service and stores the generated HTML file in the configured storage and folder.

Config

In the config file you can configure this registry:

'registries' => [
    'html.file.storage.writer' => new Registry\HtmlFileStorageWriter(
        // Set a display name:
        name: 'HTML File',

        // Set the allowed file extensions:
        allowedFileExtensions: ['html'],

        // File storage where processed files are stored:
        storageName: 'uploads-private', // default

        // Folder inside the storage where files are placed:
        storageFolder: 'exports', // default
    ),
]

If you change the file storage, you must also adjust the exported file repository so it points to the correct storage:

'interfaces' => [
    Repo\ExportedFileRepositoryInterface::class =>
    static function(StoragesInterface $storages): Repo\ExportedFileRepositoryInterface {
        return new Repo\ExportedFileRepository(
            storage: $storages->get('uploads-private'),
            rootFolder: 'exports',
        );
    },
]

Note on Image Rendering

This writer supports embedding images in the generated output.
To ensure images render correctly, you must configure the
Image Html Modifier and enable the
Media FileDisplay Feature so that image files stored in a file storage
can be resolved into public URLs.

Without this configuration, images referenced in HTML or PDF exports may
fail to load or appear as broken links.

Custom Templates via views()

This writer allows you to define reader-specific HTML templates.
This is useful when different readers (CSV, JSON, API, etc.) should produce HTML output using different layouts.

You can configure custom templates using the views() method:

'registries' => [
    'html.file.storage.writer' => new Registry\HtmlFileStorageWriter(
        name: 'HTML File',
    )
        // Optionally define reader‑specific templates:
        ->views([
            'my.custom.reader' => 'custom/template',
        ]),
]

When generating the HTML file, the writer resolves the template in this order:

  1. User-selected template (from the UI)
  2. Reader-specific template defined via views()
  3. Default template: import-export/html/export-table

This allows global defaults, per-reader templates, and per-job overrides.

Writer Behavior

The HTML File Storage Writer performs the following steps:

  • Applies the ColumnMap defined in the job
  • Applies language modifiers (if supported and configured)
  • Applies writer-specific modifiers, including the optional ImageModifier
  • Determines the template:
    • user-selected template, or
    • reader-specific template (if configured), or
    • default table layout
  • Prepares template data (title, description, image settings, etc.)
  • Creates an HTML resource writer (HtmlResource) using the selected template
  • Ensures all values are converted to safe strings using the internal ensureString helper
  • Stores the generated HTML file in the configured storage and folder
  • Registers the exported file in the ExportedFileRepository

This writer uses the following underlying components:

UI Options

The following options are available when configuring this writer in the UI:

Field Description
Filename Base filename (without extension). Must contain only letters, numbers, spaces, dots, underscores, and dashes.
Template Selects the HTML layout. If none is selected, a reader‑specific template may be applied automatically.
Title Optional document title. Can be displayed in the template.
Show Title Whether the title should be rendered as a heading in the template.
Description Optional description text rendered in the template.
Render Images Enables image rendering inside the HTML output.
Max Image Width / Height Maximum dimensions for rendered images.
Language Fields Additional language-related options depending on your application setup.

JSON File Storage Writer Registry

This registry provides writers that store processed rows as JSON-based files, including standard JSON and NDJSON (newline-delimited).
It defines the CRUD fields for configuring the output file, validates allowed extensions, and creates the writer that saves the generated file to the configured storage and folder.

Config

In the config file you can configure this registry:

'registries' => [
    // JSON-only writer
    'json.file.storage.writer' => new Registry\JsonFileStorageWriter(
        // Set a name:
        name: 'JSON File',
        
        // Set the allowed file extensions:
        allowedFileExtensions: ['json'],
        
        // File storage where processed files are stored:
        storageName: 'uploads-private', // default
        
        // Folder inside the storage where files are placed:
        storageFolder: 'exports', // default
    ),
    
    // NDJSON-only writer
    'ndjson.file.storage.writer' => new Registry\JsonFileStorageWriter(
        // Set a name:
        name: 'NDJSON File',
        
        // Set the allowed file extensions:
        allowedFileExtensions: ['ndjson'],
        
        // File storage where processed files are stored:
        storageName: 'uploads-private', // default
        
        // Folder inside the storage where files are placed:
        storageFolder: 'exports', // default
    ),
    
    // Combined writer with selectable output format
    'json.nd.file.storage.writer' => new Registry\JsonFileStorageWriter(
        name: 'JSON or NDJSON File',
        allowedFileExtensions: ['json', 'ndjson'],
    ),    
]

If you change the file storage, you must also adjust the exported file repository so it points to the correct storage:

'interfaces' => [
    Repo\ExportedFileRepositoryInterface::class =>
    static function(StoragesInterface $storages): Repo\ExportedFileRepositoryInterface {
        return new Repo\ExportedFileRepository(
            storage: $storages->get('uploads-private'),
            rootFolder: 'exports',
        );
    },
]

Writer Behavior

The JSON File Storage Writer performs the following steps:

  • Applies the ColumnMap defined in the job
  • Applies language modifiers (if supported and configured)
  • Applies any writer‑specific modifiers
  • Creates a JSON or NDJSON resource writer (JsonResource or NdJsonResource)
  • Stores the generated file in the configured storage and folder
  • Registers the exported file in the ExportedFileRepository

This writer uses the following underlying components:

UI Options

The following options are available when configuring this writer in the UI:

Field Description
Filename Base filename (without extension) for the generated file. Must contain only letters, numbers, spaces, dots, underscores, and dashes.
Output Format Selects the file extension/format. Supported formats: json and ndjson. Only shown if multiple formats are available.
Language Fields Additional language-related options provided by the system (e.g., locale selection), depending on your application setup.

JSON File Upload Reader Registry

This registry provides a reader that loads data from an uploaded JSON-based file, including standard JSON and NDJSON (newline-delimited JSON).
It defines the CRUD fields for uploading and previewing the file, validates allowed extensions, and creates the reader used during the processing workflow.

Config

In the config file you can configure this registry:

'registries' => [
    // JSON-only upload reader
    'json.file.reader' => new Registry\JsonFileUploadReader(
        // Set a name:
        name: 'JSON File Upload',
        
        // Set the allowed file extensions:
        allowedFileExtensions: ['json'],
        
        // Limit the maximum upload size (in KB) or null for no limit:
        maxFileSizeInKb: null, // default
        
        // File storage where uploaded files are stored:
        storageName: 'uploads-private', // default
        
        // Folder inside the storage where files are placed:
        storageFolder: 'imports', // default
    ),
    
    // NDJSON-only upload reader
    'ndjson.file.reader' => new Registry\JsonFileUploadReader(
        // Set a name:
        name: 'NDJSON File Upload',
        
        // Set the allowed file extensions:
        allowedFileExtensions: ['ndjson'],
        
        // Limit the maximum upload size (in KB) or null for no limit:
        maxFileSizeInKb: null, // default
        
        // File storage where uploaded files are stored:
        storageName: 'uploads-private', // default
        
        // Folder inside the storage where files are placed:
        storageFolder: 'imports', // default
    ),
    
    // Reader that accepts both JSON and NDJSON uploads
    'json.nd.file.reader' => new Registry\JsonFileUploadReader(
        name: 'JSON or NDJSON File Upload',
        allowedFileExtensions: ['json', 'ndjson'],
    ),    
]

If you change the file storage, you must also adjust the imported file repository so it points to the correct storage:

'interfaces' => [
    Repo\ImportedFileRepositoryInterface::class =>
    static function(StoragesInterface $storages): Repo\ImportedFileRepositoryInterface {
        return new Repo\ImportedFileRepository(
            storage: $storages->get('uploads-private'),
            rootFolder: 'imports',
        );
    },
]

Reader Behavior

The JSON File Upload Reader performs the following steps:

  • Validates the uploaded file using the NDJSON validator
    (allowed extensions, filename rules, max size, strict characters)
  • Stores the uploaded file in the configured storage and folder
  • Loads the selected file and ensures it exists and has a readable stream
  • Creates the appropriate reader based on the file extension:
    • JsonStream for .json
    • NdJsonStream for .ndjson
  • Provides row-by-row reading for the import-export pipeline
  • Applies no modifiers (this reader does not use ColumnMap or language modifiers)

This reader uses the following underlying components:

UI Options

The following option is available when configuring this reader in the UI:

Field Description
File Source Uploads a JSON or NDJSON file. The field validates file extension (json, ndjson), filename length, allowed characters, and maximum file size.

PDF File Storage Writer Registry

This registry provides a writer that generates a PDF file from the processed data.
It defines the CRUD fields for configuring the output file and creates the writer that stores the generated PDF in the configured storage and folder.

Config

In the config file you can configure this registry:

'registries' => [
    'pdf.file.storage.writer' => new Registry\PdfFileStorageWriter(
        // Set a name:
        name: 'PDF File',
        
        // File storage where processed files are stored:
        storageName: 'uploads-private', // default
        
        // Folder inside the storage where files are placed:
        storageFolder: 'exports', // default
    ),
]

If you change the file storage, you must also adjust the exported file repository so it points to the correct storage:

'interfaces' => [
    Repo\ExportedFileRepositoryInterface::class =>
    static function(StoragesInterface $storages): Repo\ExportedFileRepositoryInterface {
        return new Repo\ExportedFileRepository(
            storage: $storages->get('uploads-private'),
            rootFolder: 'exports',
        );
    },
]

Note on Image Rendering

This writer supports embedding images in the generated output.
To ensure images render correctly, you must configure the
Image Html Modifier and enable the
Media FileDisplay Feature so that image files stored in a file storage
can be resolved into public URLs.

Without this configuration, images referenced in HTML or PDF exports may
fail to load or appear as broken links.

Custom Templates via views()

This writer allows you to define reader-specific PDF templates.
This is useful when different readers (CSV, JSON, API, etc.) should produce PDF output using different layouts.

You can configure custom templates using the views() method:

'registries' => [
    'pdf.file.storage.writer' => new Registry\PdfFileStorageWriter(
        name: 'PDF File',
    )
        // Optionally define reader‑specific templates:
        ->views([
            'my.custom.reader' => 'custom/pdf-template',
        ]),
]

When generating the PDF file, the writer resolves the template in this order:

  1. User-selected template (from the UI)
  2. Reader-specific template defined via views()
  3. Default template: import-export/pdf/export-table

This allows global defaults, per-reader templates, and per-job overrides.

Writer Behavior

The PDF File Storage Writer performs the following steps:

  • Applies the ColumnMap defined in the job
  • Applies language modifiers (if supported and configured)
  • Applies writer-specific modifiers, including the optional ImageModifier for rendering images
  • Collects PDF configuration options such as title, orientation, paper size, DPI, margins, pagination format, compression, and password
  • Prepares template data (title, description, image settings, etc.)
  • Creates a PDF resource writer (PdfResource) using the configured options
  • Renders the PDF document using the underlying PDF engine
  • Stores the generated PDF file in the configured storage and folder
  • Registers the exported file in the ExportedFileRepository

This writer uses the following underlying components:

UI Options

The following options are available when configuring this writer in the UI:

Field Description
Title Optional title displayed at the top of the PDF.
Show title in PDF Whether the title should appear in the generated PDF.
Description Optional description text rendered in the template.
Render images in PDF Enables or disables rendering of images from the data.
Max Image Width / Height Maximum dimensions (in px) for rendered images.
Paper Paper size (A4, A3, A5).
Orientation Portrait or Landscape.
Margin (mm) Page margin in millimeters.
DPI Rendering resolution (72, 150, 300, 600).
Pagination Format Page numbering style (e.g., {PAGE}, {PAGE}/{PAGES}, etc.).
Compression PDF compression level (0, 3, 6, 9).
Password Protection Optional password required to open the PDF.

Repository Reader Registry

This registry provides a reader that loads data directly from a repository implementing
ReadRepositoryInterface. It is useful for exporting data that already exists inside the application domain (e.g., products, users, orders) without requiring file uploads.

The registry resolves the repository from the container, applies optional default filtering and sorting rules, converts entities to arrays if needed, and creates a RepositoryReader instance used during the import‑export pipeline.

Config

In the config file you can configure this registry:

'registries' => [
    'products.repository.reader' => new Registry\RepositoryReader(
        // Repository class to read from:
        repository: ProductReadRepository::class,

        // Optional default filtering:
        defaultWhere: ['status' => 'active'],

        // Optional default sorting:
        defaultOrderBy: ['created_at' => 'DESC'],

        // Optional entity-to-array converter:
        objectToArray: fn(Product $p) => [
            'id' => $p->id(),
            'sku' => $p->sku(),
            'title' => $p->title(),
        ],

        // Optional preview row count:
        previewRows: 5,

        // Display name:
        name: 'Active Products',
    ),
]

Reader Behavior

The Repository Reader performs the following steps:

  • Resolves the configured repository from the container
  • Validates that the repository implements ReadRepositoryInterface
  • Applies default filtering conditions (defaultWhere) or dynamic conditions from baseWhere()
  • Applies default sorting rules (defaultOrderBy) or dynamic rules from baseOrderBy()
  • Applies an object-to-array converter if provided, or falls back to baseObjectToArray()
  • Uses a preview row limit (default: 3) for preview mode
  • Creates a RepositoryReader that streams rows from the repository
  • Provides row-by-row reading for the import-export pipeline
  • Applies no modifiers by default, but subclasses may override createModifiers() to add custom behavior

This reader uses the following underlying components:

UI Options

This registry does not define any UI fields.
Custom readers extending this class may add fields for filtering, sorting, or dynamic options.

Extensibility

Custom readers may extend this class to:

  • Add UI fields (filters, sorting, dynamic options)
  • Override baseWhere() to apply dynamic filtering
  • Override baseOrderBy() to apply dynamic sorting
  • Override baseObjectToArray() to transform entities
  • Override basePreviewRows() to change preview behavior
  • Add custom read modifiers via createModifiers()
  • Add fields via configureFields()

This makes RepositoryReader a flexible foundation for building domain‑specific readers.

Repository Writer Registry

This registry provides a writer that stores processed rows into a repository implementing
WriteRepositoryInterface. It is useful for import jobs that write directly into the application domain (e.g., creating or updating products, users, orders) without generating files.

The registry resolves the repository from the container, applies a ColumnMap modifier, supports dry-run mode, and creates a RepositoryWriter instance used during the import-export pipeline.

Config

In the config file you can configure this registry:

'registries' => [
    'products.repository.writer' => new Registry\RepositoryWriter(
        // Repository class to write to:
        repository: ProductWriteRepository::class,

        // Name of the identifier column:
        idName: 'id',

        // Columns that may be written:
        withFields: ['sku', 'title', 'price', 'status'],

        // Optional display name:
        name: 'Product Writer',
    ),
]

Writer Behavior

The Repository Writer performs the following steps:

  • Resolves the configured repository from the container
  • Validates that the repository implements WriteRepositoryInterface
  • Applies a ColumnMap modifier to map incoming columns to repository fields
  • Supports dry-run mode:
    • When enabled, replaces the repository with NullRepository
    • Allows testing imports without modifying data
  • Creates a RepositoryWriter with:
    • The resolved repository
    • The identifier column (idName)
    • The list of writable columns (withFields)
  • Writes rows into the repository during the import-export pipeline
  • Performs create/update operations depending on repository behavior
  • Does not apply additional modifiers unless subclasses override createModifiers()

This writer uses the following underlying components:

UI Options

The following option is available when configuring this writer in the UI:

Field Description
Dry run (no write operations) If enabled, the writer uses a NullRepository so no data is written. Useful for testing imports.

Extensibility

Custom writer registries may extend this class to:

  • Add UI fields (e.g., write options, flags, validation modes)
  • Override createModifiers() to add custom write modifiers
  • Override configureFields() to expose additional writer configuration
  • Override the constructor to provide dynamic writable fields
  • Implement domain-specific write logic by wrapping or extending the repository

This makes RepositoryWriter a flexible foundation for building domain-specific writers.

Storage Reader Registry

This registry provides a reader that loads data directly from a storage table using
StorageInterface. It is useful for reading structured data stored in a database‑like storage layer (e.g., SQL tables, JSON tables, flat storage tables) without requiring file uploads.

The registry resolves the storage service from the container, applies an optional query callable, supports preview mode, and creates a StorageReader instance used during the import‑export pipeline.

Config

In the config file you can configure this registry:

'registries' => [
    'products.storage.reader' => new Registry\StorageReader(
        // Storage service ID or class name:
        storage: 'storage.mysql',

        // Table to read from:
        table: 'products',

        // Optional query callable:
        query: function($query) {
            return $query->where('status', '=', 'active');
        },

        // Optional preview row count:
        previewRows: 5,

        // Optional display name:
        name: 'Active Products Table',
    ),
]

Reader Behavior

The Storage Reader performs the following steps:

  • Resolves the configured storage service from the container
  • Validates that the storage implements StorageInterface
  • Creates a StorageReader with:
    • The resolved storage
    • The configured table name
    • The provided query callable, or the default from baseQuery()
    • The preview row limit (previewRows or basePreviewRows())
  • Executes the query before reading rows
  • Provides row-by-row reading for the import-export pipeline
  • Applies no modifiers by default, but subclasses may override createModifiers() to add custom behavior

This reader uses the following underlying components:

UI Options

This registry does not define any UI fields.
Custom readers extending this class may add fields for filtering, sorting, or dynamic options.

Extensibility

Custom readers may extend this class to:

  • Add UI fields (filters, sorting, dynamic options)
  • Override baseQuery() to apply dynamic filtering
  • Override basePreviewRows() to change preview behavior
  • Add custom read modifiers via createModifiers()
  • Add fields via configureFields()
  • Implement domain-specific query logic using the storage query builder

This makes StorageReader a flexible foundation for building storage-driven readers.

Storage Writer Registry

This registry provides a writer that stores processed rows into a table of a StorageInterface implementation. It is useful for import jobs that write directly into storage tables (e.g., SQL tables, JSON tables, or other storage backends) without generating files.

The registry resolves the storage service from the container, applies a ColumnMap modifier, supports dry-run mode via an in-memory storage, and creates a StorageWriter instance used during the import-export pipeline.

Config

In the config file you can configure this registry:

'registries' => [
    'products.storage.writer' => new Registry\StorageWriter(
        // Storage service ID or class name:
        storage: 'storage.mysql',

        // Table to write to:
        table: 'products',

        // Name of the identifier column:
        idName: 'id',

        // Columns that may be written:
        columns: ['sku', 'title', 'price', 'status'],

        // Optional display name:
        name: 'Products Storage Writer',
    ),
]

Writer Behavior

The Storage Writer performs the following steps:

  • Resolves the configured storage service from the container
  • Validates that the storage implements StorageInterface
  • Supports dry-run mode:
    • When enabled, replaces the storage with an InMemoryStorage instance
    • Allows testing imports without modifying persistent data
  • Creates a StorageWriter with:
    • The resolved storage table (storage->table($table))
    • The identifier column (idName)
    • The list of writable columns (columns)
  • Writes rows into the storage table during the import-export pipeline
  • Performs create/update operations depending on storage behavior
  • Applies a ColumnMap modifier to map incoming columns to storage fields
  • Does not apply additional modifiers unless subclasses override createModifiers()

This writer uses the following underlying components:

UI Options

The following option is available when configuring this writer in the UI:

Field Description
Dry run (no write operations) If enabled, the writer uses an in-memory storage so no data is written. Useful for testing imports.

Extensibility

Custom writer registries may extend this class to:

  • Add UI fields (e.g., write options, flags, validation modes)
  • Override createModifiers() to add custom write modifiers
  • Override configureFields() to expose additional writer configuration
  • Override the constructor to provide dynamic writable columns
  • Implement domain-specific write logic by wrapping or extending the storage table

This makes StorageWriter a flexible foundation for building storage-driven writers.

XML File Storage Writer Registry

This registry provides a writer that generates XML files using a FileStorage resource. It produces XML output via XmlResource and supports filename configuration, XML structure (root/row elements, optional wrapper), XML version, encoding, optional root attributes (namespaces), language modifiers, and column mapping.

The generated XML file is stored in the configured file storage and folder.

Config

In the config file you can configure this registry:

'registries' => [
    'xml.file.storage.writer' => new Registry\XmlFileStorageWriter(
        // Set a name:
        name: 'XML File',

        // File storage where processed files are stored:
        storageName: 'uploads-private', // default in AbstractFileStorageWriter

        // Folder inside the storage where files are placed:
        storageFolder: 'exports', // default in AbstractFileStorageWriter

        // Optionally restrict allowed file extensions (defaults to ['xml']):
        allowedFileExtensions: ['xml'],
    ),
]

If you change the file storage, you must also adjust the exported file repository so it points to the correct storage:

'interfaces' => [
    Repo\ExportedFileRepositoryInterface::class =>
    static function(StoragesInterface $storages): Repo\ExportedFileRepositoryInterface {
        return new Repo\ExportedFileRepository(
            storage: $storages->get('uploads-private'),
            rootFolder: 'exports',
        );
    },
]

UI Options

The following options are available when configuring this writer in the UI:

Field Description
Filename Base filename (without extension) for the generated XML file. Default: items.
Root Element Name of the root XML element. Default: items.
Row Element Name of the element used for each row. Default: item.
Row Wrapper (optional) Optional wrapper element around each row element.
Root Attributes Optional preset for XML namespaces (Atom, Google, Sitemap).
XML Version XML version (1.0 or 1.1). Default: 1.0.
Encoding XML encoding (UTF‑8, ISO‑8859‑1, UTF‑16). Default: UTF‑8.
Language fields Language mode and language selection (from SupportsWriterLanguages trait).

All element fields are validated to ensure valid XML element names.

Writer Behavior

The XML File Storage Writer performs the following steps:

  • Resolves the configured file storage from StoragesInterface
  • Validates that the configured storage exists ($storages->has($storageName))
  • Builds the target filename using:
    • The configured storageFolder (if any)
    • The writer.filename value (default: items)
    • The .xml extension
  • Resolves root attributes from the selected preset (atom, google, sitemap, or none)
  • Creates an XmlResource writer with:
    • FileStorage resource (storage + filename)
    • rootElement, rowElement, optional rowWrapper
    • rootAttributes (namespaces)
    • xmlVersion and encoding
  • Applies a ColumnMap modifier based on the job’s columnMap()
  • Applies language modifiers (from SupportsWriterLanguages) if configured
  • Writes rows as XML into the configured file storage and folder

This writer uses the following underlying components:

Available Hooks

Hooks allow you to react to events that occur during the import/export process.
They do not modify or validate data themselves; instead, they respond to lifecycle events and row-level outcomes reported by the processor.

The available hooks are defined in the config file and can be selected when editing an import/export job.
Each hook listens to one or more of the following events:

  • the process starting
  • the process completing or partially completing
  • the process failing due to an exception
  • a row being processed successfully
  • a row being skipped
  • a row failing with an exception

Hooks are typically used to store processed rows, update job metadata, log progress, or trigger side-effects.

Job Lifecycle Hook

This hook updates the job record during processing.
It is always active and cannot be selected or configured in the import/export job editor.
The queue handler automatically registers it for every job.

This hook writes job status and row statistics to the job repository at key points in the lifecycle:

  • processStarted
    Sets the job status to processing, stores the start time, and records the total number of rows reported by the reader.

  • partialProcess
    Updates the processed, successful, failed, and skipped row counts after a partial execution (e.g., when the time budget is reached).

  • processCompleted
    Marks the job as completed, updates all row counters, and stores the total runtime in seconds.

  • processFailed
    Marks the job as failed and updates row counters based on the result at the time of failure.

The hook ensures that the job entity always reflects the current processing state and progress.
It also prevents editing import/export jobs while they are in the processing state, ensuring that running jobs cannot be modified until they have completed or failed.

Registration

The hook is registered internally by the queue job handler:

namespace Tobento\App\ImportExport\Queue;

use Tobento\App\ImportExport\Hook;
use Tobento\Service\Queue\JobHandlerInterface;

class TimeBudgetJobHandler implements JobHandlerInterface
{
    protected function mandatoryHooks(): array
    {
        return [
            new Hook\JobLifecycle(name: 'Job Lifecycle'),
        ];
    }
}

This means the hook is always included for every import/export job and does not need to be defined in the configuration.

Customization

If you need to customize how lifecycle events are handled, you may extend the queue handler and override the mandatoryHooks() method. For example, you can replace or extend the default lifecycle hook:

use Tobento\App\ImportExport\Hook;

class CustomizedTimeBudgetQueueHandler extends TimeBudgetJobHandler
{
    protected function mandatoryHooks(): array
    {
        return [
            new MyCustomJobLifecycleHook(name: 'Custom Job Lifecycle'),
        ];
    }
}

Next, register your customized queue handler in the config file:

'interfaces' => [
    Queue\QueueHandlerInterface::class =>
    static function(QueueInterface $queue): Queue\QueueHandlerInterface {
        return new CustomizedTimeBudgetQueueHandler(
            queue: $queue,
            queueName: null,
            timeBudget: 20, // in seconds
        );
    },
],

This gives you full control over how jobs are processed, re-queued, and how lifecycle events are handled.

Notify Hook

The Notify Hook allows you to send notifications to a specific, fixed recipient such as a developer, support team, or monitoring system.
It is ideal for external alerts (mail, SMS, etc.) that should always go to the same destination, regardless of which user triggered the job.

This hook supports:

  • Any notifier channel (mail, sms, browser, storage, ...)
  • Custom notification subjects
  • Queueing
  • Selecting which job lifecycle events should trigger a notification
  • UI integration via defaultSelected

Config

Define the hook in your config file:

use Tobento\Service\Notifier\Recipient;

'hooks' => [

    'notify.dev' => new Hook\Notify(
        // Set a name:
        name: 'Notify Developer via Mail and SMS',
        
        // Define the recipient:
        recipient: new Recipient(
            email: 'dev@example.com',
            phone: '15556666666',
            channels: ['mail', 'sms'],
        ),
        
        // Customize the notification subject (optional):
        notificationSubject: 'Job :name :event',
        
        // Events to notify on:
        notifyOn: ['started', 'partialProcess', 'completed', 'failed'],
        
        // Send the notification via queue (optional):
        queueName: 'file', // null by default
        
        // Should this hook be pre-selected in the UI?
        defaultSelected: false,
        
        // Optional grouping label used in the UI (default: 'Notify'):
        group: 'Notify',
    ),
]

Additional Resources

To learn more about notifications, channels, recipients, and queueing, see:

Notify Current User Hook

The Notify Current User Hook sends notifications to the user who triggered the job.
It is ideal for providing real-time feedback during long-running import or export operations.

This hook supports:

  • Browser notifications (recommended default)
  • Any notifier channel (mail, storage, sms, ...)
  • Custom notification subjects
  • Queueing
  • Selecting which job lifecycle events should trigger a notification
  • UI integration via defaultSelected

Config

Define the hook in your config file:

'hooks' => [

    'notify.current-user' => new Hook\NotifyCurrentUser(
        // Set a name:
        name: 'Keep me updated about this job',
        
        // Define the channels to notify the current user:
        channels: ['browser'],
        
        // Customize the notification subject (optional):
        notificationSubject: 'Job :name :event',
        
        // Events to notify on (optional):
        notifyOn: ['started', 'partialProcess', 'completed', 'failed'],
        
        // Send the notification via queue (optional):
        queueName: 'file', // null by default
        
        // Should this hook be pre-selected in the UI?
        defaultSelected: true,
        
        // Optional grouping label used in the UI (default: 'Notify'):
        group: 'Notify',
    ),
]

Additional Resources

To learn more about notifications, channels, recipients, and queueing, see:

Notify Users Hook

The Notify Users Hook allows you to send notifications to all users matching specific roles.
It is ideal for notifying administrators, managers, operators, or any internal user group that should be informed about job activity.

With the notify users hook you send notifications to users using the User Repository, which is used to look up users by their roles before sending notifications.

This hook supports:

  • Notifying multiple users at once (role-based)
  • Any notifier channel (storage, browser, mail, sms, ...)
  • Custom notification subjects
  • Queueing
  • Selecting which job lifecycle events should trigger a notification
  • UI integration via defaultSelected
  • Limiting the maximum number of notified users
  • Optional grouping label for UI organization

Config

Define the hook in your config file:

'hooks' => [

    'notify.administrators' => new Hook\NotifyUsers(
        // Set a name:
        name: 'Notify Administrators via Account and Browser',
        
        // Define which user roles should be notified:
        roles: ['administrator'],
        
        // Define the channels to notify:
        channels: ['storage', 'browser'],
        
        // Customize the notification subject (optional):
        notificationSubject: 'Job :name :event',
        
        // Maximum number of users to notify:
        limit: 100,
        
        // Send the notification via queue (optional):
        queueName: 'file', // null by default
        
        // Events to notify on:
        notifyOn: ['started', 'partialProcess', 'completed', 'failed'],
        
        // Should this hook be pre-selected in the UI?
        defaultSelected: false,
        
        // Optional grouping label used in the UI (default: 'Notify'):
        group: 'Notify',
    ),
]

Additional Resources

To learn more about notifications, channels, recipients, and queueing, see:

Save Job Result Hook

This hook stores processed rows so they can be reviewed later in the Job Results Feature.
Depending on the selected mode, the hook saves either successful, skipped, or failed rows.
This is useful for inspecting problematic data, exporting results, or correcting and re-processing individual rows.

Config

In the config file you can configure this hook:

'hooks' => [
    'save.result.sussessful' => new Hook\SaveJobResult(
        // Set a name:
        name: 'Save Successful Rows',
        
        // Store rows that were processed successfully:
        mode: 'successful',
        
        // Optional grouping label used in the UI (default: 'Row'):
        group: 'Row',
    ),
    
    'save.result.skipped' => new Hook\SaveJobResult(
        name: 'Save Skipped Rows',
        mode: 'skipped',
    ),
    
    'save.result.failed' => new Hook\SaveJobResult(
        name: 'Save Failed Rows',
        mode: 'failed',
    ),
],

Additional Modifiers

In addition to the read/write modifiers, the Import & Export system provides extra modifiers that enhance formatting, localization, and media handling during the import-export pipeline.

Modifiers allow readers and writers to transform data before it is written to the final output. They extend the behavior of the core Read-Write components and integrate seamlessly with supported writers.

Column Mode Modifier

The ColumnModeModifier transforms specific columns during export according to a configured output mode.
This is useful when exporting structured or array-based fields into formats that require flat or string-based representations (CSV, HTML tables, XML attributes, etc.).

You define the transformation mode per column:

new ColumnModeModifier([
    'items' => 'split-dot',
    'gallery' => 'comma-separated',
    'attributes' => 'json',
]);

Each mode controls how the column value is flattened or serialized.

Supported Modes

1. json

Encodes the value as a JSON string.

Input

['color' => 'red', 'size' => 'L']

Output

'{"color":"red","size":"L"}'

Useful for

  • exporting structured data into CSV
  • embedding objects in text-based formats
2. comma-separated

Converts an array into a comma-separated string.

Input

['red', 'green', 'blue']

Output

'red, green, blue'

Non-scalar values are JSON-encoded automatically.

Useful for

  • CSV exports
  • HTML table exports
  • simple list fields
3. split-dot

Flattens a nested array using dot notation and prefixes keys with the column name.

Input

[
    'src' => [
        'en' => 'image-en.jpg',
        'de' => 'image-de.jpg',
    ],
]

Output

[
    'image.src.en' => 'image-en.jpg',
    'image.src.de' => 'image-de.jpg',
]

Useful for

  • exporting nested structures into flat formats
  • preparing data for tools that expect dotted keys
4. split-underlined

Same as split-dot, but uses underscore notation and normalizes the column name.

Input

[
    'src' => [
        'en' => 'image-en.jpg',
        'de' => 'image-de.jpg',
    ],
]

Output

[
    'image_src_en' => 'image-en.jpg',
    'image_src_de' => 'image-de.jpg',
]

Useful for

  • XML exports
  • systems that do not support dots in column names

When to Use ColumnModeModifier

Use this modifier when exporting:

  • nested arrays that must be flattened
  • multilingual fields that need separate columns
  • gallery or list fields
  • structured attributes that must be serialized
  • any field that must be converted into a string for CSV/HTML/XML/PDF writers

It is commonly applied by:

These registries apply the modifier automatically based on their export format.

For custom export pipelines, you may add it manually.

Example Usage

use Tobento\App\ImportExport\Modifier\ColumnModeModifier;
use Tobento\Service\ReadWrite\Modifier;

$modifiers = new Modifier\Modifiers(
    new ColumnModeModifier([
        'gallery' => 'comma-separated',
        'attributes' => 'json',
        'image' => 'split-dot',
    ]),
);

This ensures each configured column is transformed into the appropriate export format.

Dot Notation To Array Modifier

The DotNotationToArrayModifier converts flat row attributes that use dot-notation (e.g. image.src.en) into nested arrays.
This is essential for import pipelines where CSV or flat data sources represent structured fields using dotted keys.

It transforms rows like:

[
    'title' => 'Product A',
    'image.src.en' => 'products/en/image.jpg',
    'image.src.de' => 'products/de/image.jpg',
    'attributes.color' => 'red',
]

into:

[
    'title' => 'Product A',
    'image' => [
        'src' => [
            'en' => 'products/en/image.jpg',
            'de' => 'products/de/image.jpg',
        ],
    ],
    'attributes' => [
        'color' => 'red',
    ],
]

This modifier is typically used during import before writing data into repositories or storage tables that expect nested structures.

What It Does

  • Converts dotted keys into nested arrays
  • Supports arbitrarily deep nesting
  • Works with multilingual fields
  • Works with CRUD file fields that use dot-notation (image.src, image.alt.en, etc.)
  • Ensures writers receive properly structured data

It uses Arr::unflat() internally to rebuild the nested structure.

When to Use DotNotationToArrayModifier

Use this modifier when importing data that contains:

  • multilingual fields represented as field.locale
  • nested CRUD fields (e.g. image.src, image.alt.en)
  • JSON-like structures flattened into CSV columns
  • any dotted key format that should become a nested array

It is applied automatically by:

It is not applied automatically by generic writers such as:

For those registries, add it manually if your domain requires nested data.

Example Usage

use Tobento\App\ImportExport\Modifier\DotNotationToArrayModifier;
use Tobento\Service\ReadWrite\Modifier;

$modifiers = new Modifier\Modifiers(
    new DotNotationToArrayModifier(),
);

This ensures that all dotted keys in the imported rows are expanded into nested arrays before being passed to the writer.

File Export Modifier

The File Export Modifier transforms file-related fields during export so that internal storage paths are converted into URLs, signed URLs, or base64-encoded data URIs. This ensures exported data contains usable file references instead of internal storage paths.

It supports:

  • CRUD file fields (FileSource, File, Files)
  • multilingual file fields
  • arrays of files
  • arrays of multilingual files
  • public and private storage
  • base64 embedding for offline exports

Register it in your export pipeline:

use Tobento\App\ImportExport\Modifier\FileExportModifier;

new FileExportModifier(
    outputMode: 'url', // 'raw', 'url', or 'base64'
    signedUrlsExpiresInMinutes: 60, // for private storage
    container: $container,
);

Output Modes

1. raw

Returns the original storage path unchanged:

'path/image.jpg'

Useful for internal processing or debugging.

2. url

Converts file paths into public URLs or signed URLs depending on storage visibility.

Public storage example:

'https://example.com/media/uploads-public/products/image.jpg'

Private storage example (signed URL):

'https://example.com/media/signed/...&expires=1712345678'

Signed URLs expire after the configured number of minutes.

3. base64

Embeds the file content directly in the export:

'data:image/png;base64,iVBORw0KGgoAAA...'

Useful for:

  • PDF exports
  • HTML exports
  • API responses
  • offline bundles

Supported Field Types

1. CRUD File Fields

[
    'src' => 'products/image.jpg',
    'storage' => 'uploads-public',
]

Converted into a URL, signed URL, or base64 string.

2. Multilingual File Fields

[
    'src' => [
        'en' => 'products/image-en.jpg',
        'de' => 'products/image-de.jpg',    
    ],
]

Each locale is resolved independently.

3. Arrays of Files

[
    ['src' => 'gallery/1.jpg', 'storage' => 'uploads-public'],
    ['src' => 'gallery/2.jpg', 'storage' => 'uploads-public'],
]

Each file is resolved individually.

4. Arrays of Multilingual Files

Fully supported:

[
    [
        'src' => [
            'en' => 'gallery/en/1.jpg',
            'de' => 'gallery/de/1.jpg',
        ],
        'storage' => 'uploads-public',
    ],
    [
        'src' => [
            'en' => 'gallery/en/2.jpg',
            'de' => 'gallery/de/2.jpg',
        ],
        'storage' => 'uploads-public',
    ],
]

Each locale is resolved independently, producing:

[
    [
        'src' => [
            'en' => 'https://example.com/.../gallery/en/1.jpg',
            'de' => 'https://example.com/.../gallery/de/1.jpg',
        ],
        'storage' => 'uploads-public',
    ],
    ...
]

When to Use FileExportModifier

Use this modifier when exporting:

  • product images
  • user avatars
  • document attachments
  • multilingual file fields
  • galleries or arrays of files
  • CRUD resources containing file fields

The following registries apply the FileExportModifier automatically

Image Html Modifier

Some writers (such as HTML and PDF) support embedding images into the generated output. The Image Modifier resolves file-storage paths into public URLs so that images can be rendered correctly in browsers or PDF engines.

The Image Html Modifier enables:

  • mapping storage paths to public URLs
  • selecting which file storage to use for image resolution
  • ensuring embedded images load correctly in HTML and PDF exports

Image resolution relies on the Media FileDisplay Feature, which exposes files from a storage as HTTP URLs.

Make sure the storage containing your images is registered in the media config file.

'features' => [
    new Feature\FileDisplay(
        // define the supported storages:
        supportedStorages: ['uploads-public'],
    ),
],

Writers that support image embedding reference this modifier automatically.

Supported Input Formats

The Image Modifier supports several image representations commonly produced by ImportExport readers and CRUD systems.

1. CRUD File Format

['src' => 'image.jpg', 'storage' => 'uploads-public']

Multilingual

['src' => ['en' => 'image.jpg'], 'storage' => 'uploads-public']

2. Array of CRUD Files

[
    ['src' => 'image1.jpg', 'storage' => 'uploads-public'],
    ['src' => 'image2.jpg', 'storage' => 'uploads-public'],
]

Multilingual

[
    ['src' => ['en' => 'image.jpg'], 'storage' => 'uploads-public'],
    ['src' => ['en' => 'image.jpg'], 'storage' => 'uploads-public'],
]

3. Plain Image Maps

'https://example.com/image.jpg'

Multilingual

[
    'en' => 'https://example.com/image-en.jpg',
    'de' => 'https://example.com/image-de.jpg',
]

Unsupported or Invalid Values

If the modifier cannot resolve the value (missing src, invalid URL, non-image),
the original value is returned unchanged so other modifiers can handle it.

Modes

  • raw - returns the URL for the selected language
  • single - returns only the first selected language
  • all-in-one - returns multiple <img> tags

Image Resizing

The modifier can automatically scale images to fit the configured maximum width and height:

$modifier = new ImageModifier(
    mode: 'raw',
    selectedLanguages: ['en'],
    languages: $languages,
    imgWidth: 200,
    imgHeight: 150,
    container: $container,
);

Resizing Rules

  • If the original image size is known (CRUD file):

    • scaled proportionally by width
    • then scaled proportionally by height
    • never upscaled
    • never distorted

  • If the original size is unknown (absolute URLs, multilingual maps):

    • only the width is applied
    • height is omitted

Error Handling

The modifier safely handles invalid or incomplete data:

  • missing src, returns original value
  • non-image file, returns original value
  • invalid URL, returns original value
  • missing storage file, returns original value

This ensures the modifier never breaks the export pipeline.

Language Modifier

The Language Modifier allows writers to output localized values based on the selected language. It is used by writers that support multi-language output (e.g., HTML, PDF, JSON, XML).

The Language Modifier enables:

  • selecting a language for export
  • applying language-specific transformations
  • resolving localized fields or translations

This modifier is provided by the SupportsWriterLanguages trait and is available in writers that include language configuration fields.

Uploaded File Modifier

The Uploaded File Modifier converts import values into UploadedFile instances so that CRUD write operations can treat imported files exactly like files uploaded through forms.

This modifier enables:

  • importing files from remote URLs
  • importing files from data URIs
  • importing files from raw base64 strings
  • seamless integration with CRUD repositories expecting UploadedFileInterface

It is automatically applied when you register it in your Import Bulk Action:

use Tobento\App\ImportExport\Modifier\UploadedFileModifier;

new UploadedFileModifier($this->withFileFields(), $container)

Only the fields explicitly listed in $withFileFields are processed.

Supported Input Formats

The modifier supports several common file representations used in import pipelines.

1. Remote URLs

'https://example.com/image.jpg'

The file is downloaded and wrapped in an UploadedFile instance.

2. Data URIs

'data:image/png;base64,iVBORw0KGgoAAA...'

The base64 payload is decoded and stored as an in‑memory uploaded file.

3. Raw Base64 Strings

'iVBORw0KGgoAAAANSUhEUgAA...'

If the string is valid base64, it is decoded and converted into an uploaded file.

Field Selection

Only fields explicitly defined in the Import Bulk Action are processed:

use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\ImportExport\Crud\ImportBulkAction;

protected function configureActions(): iterable|ActionsInterface
{
    // other actions ...
    
    yield new ImportBulkAction(
        // Unique identifier for the bulk action (must be unique per CRUD resource)
        name: 'import',

        // The label shown in the bulk-action dropdown
        title: 'Import',

        // Fields available for mapping and writing during the import
        withFields: ['id', 'sku', 'title'],
        
        withFileFields: ['image'], // processed by UploadedFileModifier
    );
}

This makes the modifier predictable and prevents accidental conversion of unrelated fields.

Supported CRUD File Field Structures

The UploadedFileModifier automatically detects the internal structure of all CRUD file field types and converts their file values into UploadedFileInterface instances. Only the field name needs to be listed (e.g. image, gallery, meta.image); the modifier resolves the correct internal paths automatically.

FileSource Field

new Field\FileSource('image')

Import structure:

'image' => 'https://example.com/image.jpg'

A plain string path or URL.
Converted directly into an UploadedFileInterface.

File Field

new Field\File('image')

Import structure:

'image' => [
    'src' => 'https://example.com/image.jpg'
]

The modifier converts only the src value.
Other keys (e.g. storage) are ignored during import.

Translatable File Field

new Field\File('image')->translatable()

Import structure:

'image' => [
    'src' => [
        'en' => 'https://example.com/en.jpg',
        'de' => 'https://example.com/de.jpg',
    ]
]

Each locale value is converted individually.

Files Field

new Field\Files('image')

Import structure:

'gallery' => [
    ['src' => 'https://example.com/1.jpg'],
    ['src' => 'https://example.com/2.jpg'],
]

Each entry is processed and its src value converted.

Translatable Files Field
new Field\Files(name: 'files')
    ->translatable()
    ->file(function(Field\File $file): void {
        $file->translatable();
    });

Import structure:

'gallery' => [
    [
        'src' => [
            'en' => 'https://example.com/en.jpg',
            'de' => 'https://example.com/de.jpg',
        ],
    ],
]

Each locale value is converted individually.

Multilingual Map (non-CRUD file field)

Useful for custom fields storing localized file paths.

Import structure:

'manual' => [
    'en' => 'https://example.com/manual-en.pdf',
    'de' => 'https://example.com/manual-de.pdf',
]

Mapping CRUD File Fields to withFileFields()

CRUD Field Type Example What to put in withFileFields()
FileSource new Field\FileSource('image') ['image']
File new Field\File('image') ['image']
Files new Field\Files('images') ['images']
Multilingual map custom ['manual']

Error Handling

The modifier is designed to fail gracefully:

  • any exception during file creation is caught
  • the original value is preserved
  • the import continues without interruption

This ensures robustness even when dealing with inconsistent or user-generated import data.

Example Usage in ImportBulkAction

To enable file importing, you must declare which fields should be treated as file fields using withFileFields(). These fields will later be processed by the UploadedFileModifier.

use Psr\Container\ContainerInterface;
use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\ImportExport\Crud\ImportBulkAction;

protected function configureActions(): iterable|ActionsInterface
{
    // other actions ...
    
    yield new ImportBulkAction(
        name: 'import',
        title: 'Import',

        // Fields available for mapping and writing during the import
        withFields: ['id', 'sku', 'title', 'image'],

        // Fields that should be processed by UploadedFileModifier
        withFileFields: ['image'],
    );
}

Declaring withFileFields: ['image'] does not apply the modifier by itself.
It only tells the system which fields should be treated as file fields.

The actual modifier is applied inside the ImportBulkAction's createModifiers() method:

public function createModifiers(ContainerInterface $container, JobEntityInterface $jobEntity): ModifiersInterface
{
    return new Modifier\Modifiers(
        // Maps incoming columns to the fields defined in withFields()
        new Modifier\ColumnMap(map: $jobEntity->columnMap()),

        // Converts dot-notation keys (e.g. "meta.image") into nested arrays
        new DotNotationToArrayModifier(),

        // Converts file values (URLs, base64, data URIs) into UploadedFile instances
        new UploadedFileModifier($this->withFileFields(), $container),
    );
}

The UploadedFileModifier runs before the writer persists the entity, ensuring that all file fields are already converted into UploadedFileInterface instances and can be handled by CRUD write operations exactly like normal uploaded files.

CRUD Integration

You can also trigger imports and exports directly from CRUD index pages when actions are enabled for a specific controller.

Export Bulk Action

The Export Bulk Action adds an export workflow to any CRUD resource. It allows users to export either the selected rows or all filtered rows directly from the bulk-action menu. When triggered, a modal opens where the user can configure the writer, mapping, and hooks used for the export job, keeping the index page clean and uncluttered.

After the configuration is saved, a new export job is created and listed in the Import/Export feature. The job is then pushed to the queue and processed in the background, ensuring that even large exports do not block the UI or impact performance.

Key capabilities

  • Export selected or filtered rows from any CRUD list.
  • Configure writer, mapping, and hooks through a modal.
  • Apply modifiers to transform fields before exporting.
  • Automatically create a queued export job that runs in the background.

Example

In your CRUD Controller add the ExportBulkAction in the configureActions method:

use Psr\Container\ContainerInterface;
use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\ImportExport\Crud\ExportBulkAction;
use Tobento\App\ImportExport\Repo\JobEntityInterface;
use Tobento\Service\ReadWrite\Modifier;
use Tobento\Service\ReadWrite\ModifiersInterface;
use Tobento\Service\ReadWrite\Reader\RepositoryReader;
use Tobento\Service\ReadWrite\RowInterface;

protected function configureActions(): iterable|ActionsInterface
{
    // other actions ...
    
    yield new ExportBulkAction(
        // Unique identifier for the bulk action (must be unique per CRUD resource)
        name: 'export',
        
        // The label shown in the bulk-action dropdown
        title: 'Export'
    )
        // Optional: apply modifiers to transform fields before export
        ->modifiers(static function (ContainerInterface $container, JobEntityInterface $jobEntity): ModifiersInterface {
            // You may conditionally adjust modifiers depending on the writer selected for the job
            // $writerId = $jobEntity->writerId(); // e.g. 'pdf.file.storage.writer'
            return new Modifier\Modifiers(
                // Example: transform the 'sku' field before writing
                new Modifier\Format(
                    field: 'sku',
                    formatter: function ($value, RowInterface $row) {
                        return strtoupper(trim((string)$value));
                    }
                ),
            );
        })
        
        // Optional: create custom reader
        ->reader(static function (ExportBulkAction $action, JobEntityInterface $jobEntity): RepositoryReader {
            return new RepositoryReader(
                repository: $action->controller()->repository(),
                where: ['type' => 'products'], // base filters
                objectToArray: static function (object $entity) use ($action): array {
                    return $action->controller()->createEntityFromObject($entity)->toArray();
                },
                previewRows: 3,
            );
        });
}

Note
The writer automatically applies the job's column mapping and any writer-specific modifiers (e.g., image rendering, language modifiers). The modifiers defined in the ExportBulkAction are merged with the writer's modifiers.

Check out the the Repository Writer for more details.

You may define multiple Export Bulk Actions for the same CRUD resource.
This is useful if you want different sets of modifiers or different predefined configurations.
Each action must have a unique name to avoid conflicts.

A list of available modifiers can be found at:
https://github.com/tobento-ch/service-read-write#modifiers

Example: Customizing Export Fields with modifyFields()

This example shows how to adjust the fields displayed in the export dialog. You can remove default fields that are not relevant for your export scenario and add your own fields to control export behavior.

use Psr\Container\ContainerInterface;
use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\Crud\Field;
use Tobento\App\Crud\Field\Fields;
use Tobento\App\Crud\Field\FieldsInterface;
use Tobento\App\ImportExport\Crud\ExportBulkAction;
use Tobento\App\ImportExport\Repo\JobEntityInterface;
use Tobento\Service\ReadWrite\Modifier;
use Tobento\Service\ReadWrite\ModifiersInterface;
use Tobento\Service\ReadWrite\RowInterface;

protected function configureActions(): iterable|ActionsInterface
{
    // other actions ...

    yield new ExportBulkAction(
        name: 'export',
        title: 'Export'
    )
        ->modifyFields(function (
            ActionInterface $action,
            FieldsInterface $fields,
            ExportBulkAction $export
        ): iterable|FieldsInterface {

            // Convert to array for modification
            $all = $fields->all();

            // Add a custom import mode selector
            $all[$export->fieldName('options.mode')] = new Field\Select(
                name: $export->fieldName('options.mode'),
                label: 'Export Mode',
            )
                ->options([
                    'basic' => 'Basic',
                    'advanced' => 'Advanced',
                    'custom' => 'Custom',
                ]);

            // Return new Fields instance
            return Fields::fromIterable($all);
        })

        // Apply modifiers depending on the mode:
        ->modifiers(static function (ContainerInterface $container, JobEntityInterface $jobEntity): ModifiersInterface {
            // Read the selected mode from job options
            $mode = $jobEntity->get('options.mode');
            
            // Apply conditional modifiers
            if ($mode === 'advanced') {
                return new Modifier\Modifiers(
                    new Modifier\Format(
                        field: 'sku',
                        formatter: function ($value, RowInterface $row) {
                            return strtoupper(trim((string)$value));
                        }
                    ),
                );
            }
            
            return new Modifier\Modifiers();
        })
}

Import Bulk Action

The Import Bulk Action adds a multi-step import workflow to any CRUD resource. It enables users to upload files such as CSV, JSON, or XML and import data directly into the system through a guided modal interface. The workflow is intentionally split into two steps to keep the index page clean while still allowing users to validate and configure the import before it is executed.

After the configuration is completed, a new import job is created and listed in the Import/Export feature. The job is then dispatched to the queue and processed in the background, ensuring that even large imports do not block the UI or impact performance.

Key capabilities

  • Import data from uploaded files (CSV, JSON, XML, ...).
  • Multi-step modal workflow:
    • Step 1: Choose the reader.
    • Step 2: Configure mapping, options, and hooks.
  • Supports a dry-run option that runs the full import pipeline - validation, mapping, and row-level result generation - without writing any data, making it easy to review which rows would succeed, skip, or fail before executing the real import.
  • Automatically create a queued import job that runs in the background.
  • Supports custom readers, mapping logic, and import options.
  • Works seamlessly with bulk selection or filtered lists.
  • Allows defining multiple Import Bulk Actions with different presets.

Example

In your CRUD Controller, add the ImportBulkAction in the configureActions method:

use Psr\Container\ContainerInterface;
use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\ImportExport\Crud\ImportBulkAction;
use Tobento\App\ImportExport\Modifier\DotNotationToArrayModifier;
use Tobento\App\ImportExport\Modifier\UploadedFileModifier;
use Tobento\App\ImportExport\Repo\JobEntityInterface;
use Tobento\Service\ReadWrite\Modifier;
use Tobento\Service\ReadWrite\ModifiersInterface;
use Tobento\Service\ReadWrite\RowInterface;

protected function configureActions(): iterable|ActionsInterface
{
    // other actions ...
    
    yield new ImportBulkAction(
        // Unique identifier for the bulk action (must be unique per CRUD resource)
        name: 'import',

        // The label shown in the bulk-action dropdown
        title: 'Import',

        // Fields available for mapping and writing during the import
        withFields: ['id', 'sku', 'title'],

        // Fields that should be processed by UploadedFileModifier
        withFileFields: ['image'],
    )
        // Optional: apply modifiers to transform fields before import
        ->modifiers(static function (ContainerInterface $container, JobEntityInterface $jobEntity): ModifiersInterface {
            // You may conditionally adjust modifiers depending on the reader selected for the job
            // $readerId = $jobEntity->readerId(); // e.g. 'csv.file.reader'
            return new Modifier\Modifiers(
                // Keep default modifiers
                new DotNotationToArrayModifier(),
                new Modifier\ColumnMap(map: $jobEntity->columnMap()),
                new UploadedFileModifier($jobEntity->withFileFields(), $container),
                
                // Example: transform the 'sku' field before writing
                new Modifier\Format(
                    field: 'sku',
                    formatter: function ($value, RowInterface $row) {
                        return strtoupper(trim((string)$value));
                    }
                ),
            );
        });
}

If the primary key field (for example id) is included in withFields and mapped during the import, the RepositoryWriter will attempt to update an existing entity.
If no entity with that ID exists, the update fails and the row is recorded as a failed import.
If the primary key is not mapped or the value is empty, a new entity is created instead.

This behavior is defined by the Repository Writer's logic:

  • If $attributes[$idName] exists updateById(...)
  • Otherwise create(...)

See: https://github.com/tobento-ch/service-read-write#repository-writer

A list of available modifiers can be found at:
https://github.com/tobento-ch/service-read-write#modifiers

Example: Custom Writer

You may optionally create a custom writer to control how imported rows are written into your repository.
Using CrudWriteRepository is recommended because it automatically applies the fields's validation rules, write-actions, and behaviors (such as create/update logic, and policies).
Alternatively, you may directly use your controller's repository.
This gives you maximum flexibility but also requires implementing any additional logic yourself, since validation, behaviors, and write-actions are not applied automatically.

Check out the Crud Write Repository
and the Repository Writer for more details.

use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\Crud\ActionProcessorInterface;
use Tobento\App\Crud\CrudWriteRepository;
use Tobento\App\ImportExport\Crud\ImportBulkAction;
use Tobento\App\ImportExport\Repo\JobEntityInterface;
use Tobento\Service\ReadWrite\Writer\RepositoryWriter;
use Tobento\Service\Repository\NullRepository;

protected function configureActions(): iterable|ActionsInterface
{
    // other actions ...
     
    yield new ImportBulkAction(
        name: 'import',
        title: 'Import',
        withFields: ['id', 'sku', 'title'],
    )
        // This example shows default implementation using CrudWriteRepository::class
        ->writer(static function (ImportBulkAction $action, JobEntityInterface $jobEntity): RepositoryWriter {
            $controller = $action->controller();
            $container = $action->container();
            $actionProcessor = $container->get(ActionProcessorInterface::class);
            
            // Dry run: replace the repository so no write operations are performed
            if ($jobEntity->get('options.dry_run') === '1') {
                $controller = $controller->withRepository(new NullRepository());
            }
        
            $repository = new CrudWriteRepository(
                controller: $controller,
                actionProcessor: $actionProcessor,
            )->onlyFields(...$action->withFields());

            return new RepositoryWriter(
                repository: $repository,
                idName: $controller->entityIdName(),
                columns: $action->withFields(),
            );
        })
        
        // Example without using CrudWriteRepository::class
        ->writer(static function (ImportBulkAction $action, JobEntityInterface $jobEntity): RepositoryWriter {
            $controller = $action->controller();

            return new RepositoryWriter(
                repository: $controller->repository(),
                idName: $controller->entityIdName(),
                columns: $action->withFields(),
            );
        });
}

Example: Customizing Import Fields with modifyFields()

This example shows how to adjust the fields displayed in the import dialog. You can remove default fields that are not relevant for your import scenario and add your own fields to control import behavior.

use Psr\Container\ContainerInterface;
use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\Crud\Field;
use Tobento\App\Crud\Field\Fields;
use Tobento\App\Crud\Field\FieldsInterface;
use Tobento\App\ImportExport\Crud\ImportBulkAction;
use Tobento\App\ImportExport\Repo\JobEntityInterface;
use Tobento\Service\ReadWrite\Modifier;
use Tobento\Service\ReadWrite\ModifiersInterface;
use Tobento\Service\ReadWrite\RowInterface;

protected function configureActions(): iterable|ActionsInterface
{
    // other actions ...

    yield new ImportBulkAction(
        name: 'import',
        title: 'Import',
        withFields: ['id', 'sku', 'title'],
    )
        ->modifyFields(function (
            ActionInterface $action,
            FieldsInterface $fields,
            ImportBulkAction $import,
            int $step,
            null|JobEntityInterface $jobEntity
        ): iterable|FieldsInterface {
            // Only modify fields on step 2 (mapping step)
            if ($step !== 2) {
                return $fields;
            }
            
            // Convert to array for modification
            $all = $fields->all();

            // Add a custom import mode selector
            $all[$import->fieldName('options.mode')] = new Field\Select(
                name: $import->fieldName('options.mode'),
                label: 'Import Mode',
            )
                ->options([
                    'basic' => 'Basic',
                    'advanced' => 'Advanced',
                    'custom' => 'Custom',
                ]);

            // Return new Fields instance
            return Fields::fromIterable($all);
        })

        // Apply modifiers depending on the mode:
        ->modifiers(static function (ContainerInterface $container, JobEntityInterface $jobEntity): ModifiersInterface {
            // Read the selected mode from job options
            $mode = $jobEntity->get('options.mode');
            
            // Apply conditional modifiers
            if ($mode === 'advanced') {
                return new Modifier\Modifiers(
                    new Modifier\Format(
                        field: 'sku',
                        formatter: function ($value, RowInterface $row) {
                            return strtoupper(trim((string)$value));
                        }
                    ),
                );            
            }
            
            return new Modifier\Modifiers();
        })
}

Multi-step modal workflow

The Import Bulk Action uses a structured two-step modal process designed for clarity and safety:

  1. Step 1:
    The user selects a reader from a dropdown (for example, CsvFileUploadReader).
    Depending on the selected reader, the form updates live - for file-based readers an upload field appears, and the user must upload a file before continuing to the next step.

  2. Step 2:
    The user configures mapping, options, and hooks.
    Any validation issues keep the modal open so the user can correct them.

  3. Step 2 success:
    When the import configuration is valid, the job is created and queued.

This design keeps the index page clean while still supporting flexible and complex import workflows.

Customization

Multiple Import Bulk Actions may be defined for the same CRUD resource.
This is useful when different readers, mapping presets, or import behaviors are required.
Each action must have a unique name to avoid conflicts.

ACL Permission for Bulk Actions

Bulk actions such as Import or Export can be protected using the ACL permissions provided by the Import/Export Feature. This ensures that only authorized users can trigger data-processing jobs from within a CRUD resource.

Available Permissions

The Import/Export Feature defines the following permissions:

  • import-export User can access import/export
  • import-export.create User can create import/export
  • import-export.edit User can edit import/export
  • import-export.delete User can delete import/export
  • import-export.run User can run import/export

For bulk actions, the most relevant permission is import-export.run, because bulk actions typically execute a job immediately.

Example: Protecting a Bulk Action

Inject the ACL service into your controller and conditionally register the bulk action only if the user has the required permission.

use Tobento\App\Crud\AbstractCrudController;
use Tobento\App\Crud\Action\ActionsInterface;
use Tobento\App\ImportExport\Crud\ImportBulkAction;
use Tobento\Service\Acl\AclInterface;

class ProductController extends AbstractCrudController
{
    public const RESOURCE_NAME = 'products';
    
    /**
     * Create a new ProductController.
     *
     * @param RepositoryInterface $repository
     */
    public function __construct(
        ProductRepository $repository,
        protected AclInterface $acl,
    ) {
        $this->repository = $repository;
    }

    protected function configureActions(): iterable|ActionsInterface
    {
        // Other actions...

        // Only register the import bulk action if the user is allowed to run imports
        if ($this->acl->can('import-export.run')) {
            yield new ImportBulkAction(
                name: 'import',
                title: 'Import',
                withFields: ['id', 'sku', 'title'],
            );
        }
    }
}

Why ACL for Bulk Actions?

Bulk actions often trigger powerful operations such as importing or exporting large amounts of data. These actions may create, update, or delete many records at once, or start long-running background jobs. Because of this, they should only be available to users who are explicitly allowed to perform such operations.

Using the ACL permissions from the Import/Export Feature ensures that:

  • only authorized users can start import or export jobs
  • sensitive data operations are not exposed to unintended roles
  • CRUD resources remain consistent with the global Import/Export permission model
  • permission checks behave the same whether the user runs a job from the Jobs page or from a CRUD bulk action

By applying the same permission rules across both areas, the system stays predictable, secure, and easy to reason about.

Learn More

Adding Registries Via App

In addition to add registries via config file, you can use the App on method to add registries only on demand:

use Tobento\App\Task\RegistriesInterface;
use Tobento\App\Task\Registry\CommandTask;

$app->on(
    RegistriesInterface::class,
    static function(RegistriesInterface $registries): void {
        $registries->add(id: 'prune.auth.tokens', registry: new CommandTask(
            name: 'Prune Auth Tokens',
            command: 'auth:purge-tokens',
        ));
    }
);

Adding Hooks Via App

In addition to add hooks via config file, you can use the App on method to add hooks only on demand:

use Tobento\App\Task\HooksInterface;
use Tobento\App\Task\Hook\Mail;

$app->on(
    HooksInterface::class,
    static function(HooksInterface $hooks): void {
        $hooks->add(id: 'mail.dev', registry: new Mail(
            name: 'Mail Developer',
            email: 'dev@example.com',
        ));
    }
);

Notifications for Background Jobs

Import/Export jobs and Reprocess jobs run asynchronously in the queue.
Notifications are handled through the configured notification hooks in the Import Export Config, which are available out of the box and can be selected by the user in the UI.

By default, the Import/Export package ships with several hooks, including:

Users can choose which hooks should run for each job directly in the UI.
Hooks marked with defaultSelected: true (such as notify.current-user) are pre-selected automatically.

To receive browser notifications, the user must have the notifications.browser ACL permission.
All other functionality is already set up and works out of the box.

You may set the permission manually (see ACL Service), or, if you are using the App Backend, you can assign this permission directly on the Roles or Users page.

Credits