insalyon/acme-plugin-etl

ACME Plugin to create ETL Job

v1.7.2-alpha 2025-05-22 06:02 UTC

Requires

• php: >=8.2
• insalyon/acme: ^v2.0.1-alpha

CC-BY-NC-4.0

• Charbel FOUREL
• Johan MAURICE
• Lucas DUPUY
• Aurelien Leygues

README

# ACME Plugin ETL

ETL Job is a synchronization processus between 2 different data sources.

With this plugin you can synchronized data in this architecture by running jobs :

flowchart TB
    subgraph ACME_ETL
        JOB_NAME_01([JOB_NAME_01])
        JOB_NAME_02([JOB_NAME_02])
    end
    subgraph DATA_SOURCE
        ORIGIN_LAYER[(ORIGIN_LAYER<p>postgres, oracle, mysql, api http, ldap</p>)]
        TARGET_LAYER[(TARGET_LAYER<p>postgres, oracle, mysql, api http, ldap</p>)]
    end
    ORIGIN_LAYER --> ACME_ETL --> TARGET_LAYER
    CLIENT --GET http://cname/api/v1/synch:job:name:01--> JOB_NAME_01
    CLIENT --GET http://cname/api/v1/synch:job:name:02--> JOB_NAME_02

See ACME Project

Configuration of plugin : ACMEPluginETLConfiguration.php

Implementation of plugin : ACMEPluginETL.php

Changelog versions

Prepare your development environement

Execute your development environement

Supervision and monitoring

When you enable ACME metrics layer in the cfg.ini of your connector, the plugin automatically push metrics in the configured metrics database.

You can use the [Grafana dashboard](./ACME CONNECTORS-1738771557548.json) definition to visualize the metrics (compatibility with plugin version >= v1.6.5-alpha).