keboola / job-queue-api-php-client
Client for Job queue API
Requires
- php: ^7.4|^8.0
- ext-json: *
- guzzlehttp/guzzle: ^6.3|^7.2
- psr/log: ^1.1
- symfony/config: ^4.4|^5.2
- symfony/validator: ^4.4|^5.2 !=4.4.33 !=4.4.34 !=5.3.10 !=5.3.11
Requires (Dev)
- infection/infection: ^0.18.2|^0.21.2
- keboola/coding-standard: ^13.0
- keboola/storage-api-client: ^12.0
- php-parallel-lint/php-parallel-lint: ^1.2
- phpstan/phpstan: ^1.4
- phpunit/phpunit: ^9.5
- dev-main
- 10.7.0
- 4.0.0
- 3.2.0
- 3.1.0
- 3.0.1
- 3.0.0
- 2.4.0
- 2.3.0
- 2.2.0
- 2.1.0
- 2.0.0
- 1.0.0
- 0.5.0
- 0.4.1
- 0.4.0
- 0.3.0
- 0.2.2
- 0.2.1
- 0.2.0
- 0.1.0
- 0.0.2
- 0.0.1
- dev-odin-KAB-318
- dev-jv-PST-1076-data-loader-tests
- dev-pepa_PST-1969_backports
- dev-pepa_PST-777_om_1
- dev-pepa_test
- dev-jv-fix-search-endpoints-naming
- dev-kacurez-gitops-lab-gar-push-ST-1901
- dev-miro-PST-1343-b
- dev-adamvyborny-PST-1179-internal-api-elastic-poc
- dev-dependabot/composer/apps/gelf-logger-server/aws/aws-sdk-php-3.304.4
- dev-dependabot/composer/apps/public-api/aws/aws-sdk-php-3.297.3
- dev-dependabot/composer/apps/public-api/symfony/security-http-6.4.4
- dev-dependabot/composer/apps/gelf-logger-server/guzzlehttp/psr7-1.9.1
- dev-dependabot/composer/apps/input-mapping/guzzlehttp/psr7-2.4.5
- dev-erik-GCP-233-phpcs-fixup
- dev-pepa_PST-955_indexHints
- dev-ujovlado-no-license
- dev-erik-PST-419-try2
- dev-pepa-hackathon
- dev-pepa_gha
- dev-internal-api-php-client-9.x
This package is auto-updated.
Last update: 2024-11-02 13:23:11 UTC
README
PHP client for the Job Queue API (API docs).
Usage
composer require keboola/job-queue-api-php-client
use Keboola\JobQueueClient\Client; use Keboola\JobQueueClient\JobData;use Psr\Log\NullLogger; $client = new Client( new NullLogger(), 'http://queue.conenection.keboola.com/', 'xxx-xxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx' ); $result = $client->createJob(new JobData( 'keboola.ex-db-snowflake', '123', [], 'run' )); var_dump($result['id']);
Development
-
Create a service principal to download the required images and login:
SERVICE_PRINCIPAL_NAME=[USERNAME]-job-queue-internal-api-pull ACR_REGISTRY_ID=$(az acr show --name keboolapes --query id --output tsv --subscription c5182964-8dca-42c8-a77a-fa2a3c6946ea) SP_PASSWORD=$(az ad sp create-for-rbac --name http://$SERVICE_PRINCIPAL_NAME --scopes $ACR_REGISTRY_ID --role acrpull --query password --output tsv) SP_APP_ID=$(az ad sp show --id http://$SERVICE_PRINCIPAL_NAME --query appId --output tsv)
-
Login and check that you can pull the image:
docker login keboolapes.azurecr.io --username $SP_APP_ID --password $SP_PASSWORD docker pull keboolapes.azurecr.io/job-queue-internal-api:latest docker pull keboolapes.azurecr.io/job-queue-api:latest
-
Add the credentials to the k8s cluster:
kubectl create secret docker-registry regcred --docker-server="https://keboolapes.azurecr.io" --docker-username="$SP_APP_ID" --docker-password="$SP_PASSWORD" --namespace dev-job-queue-api-php-client kubectl patch serviceaccount default -p "{\"imagePullSecrets\":[{\"name\":\"regcred\"}]}" --namespace dev-job-queue-api-php-client
-
Set the following environment variables in
set-env.sh
file (useset-env.template.sh
as sample):test_storage_api_url
- Keboola Connection URL.test_storage_api_token
- Token to a test project.
-
Set one of Azure or AWS resources (or both, but only one is needed).
AWS Setup
- Create a user (
JobQueueApiPhpClient
) for local development using theprovisioning/aws.json
CF template.- Create AWS key for the created user.
- Set the following environment variables in
set-env.sh
file (useset-env.template.sh
as sample):test_aws_access_key_id
- The created security credentials for theJobQueueApiPhpClient
user.test_aws_secret_access_key
- The created security credentials for theJobQueueApiPhpClient
user.test_kms_region
-Region
output of the above stack.test_kms_key_id
-KmsKey
output of the above stack.
Azure Setup
-
Create a resource group:
az account set --subscription "Keboola DEV PS Team CI" az group create --name testing-job-queue-api-php-client --location "East US"
-
Create a service principal:
az ad sp create-for-rbac --name testing-job-queue-api-php-client
-
Use the response to set values
test_azure_client_id
,test_azure_client_secret
andtest_azure_tenant_id
in theset-env.sh
file:{ "appId": "268a6f05-xxxxxxxxxxxxxxxxxxxxxxxxxxx", //-> test_azure_client_id "displayName": "testing-job-queue-internal-api-php-client", "name": "http://testing-job-queue-internal-api-php-client", "password": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", //-> test_azure_client_secret "tenant": "9b85ee6f-xxxxxxxxxxxxxxxxxxxxxxxxxxx" //-> test_azure_tenant_id }
-
Get ID of the service principal:
SERVICE_PRINCIPAL_ID=$(az ad sp list --display-name testing-job-queue-api-php-client --query "[0].objectId" --output tsv)
-
Deploy the key vault, provide tenant ID, service principal ID and group ID from the previous commands:
./set-env.sh # set $test_azure_tenant_id env az deployment group create --resource-group testing-job-queue-api-php-client --template-file provisioning/azure.json --parameters vault_name=test-job-queue-client tenant_id=$test_azure_tenant_id service_principal_object_id=$SERVICE_PRINCIPAL_ID
-
Show Key Vault URL
az keyvault show --name test-job-queue-client --query "properties.vaultUri" --output tsv
returns e.g. https://test-job-queue-client.vault.azure.net/
, use this to set values in set-env.sh
file:
- test_azure_key_vault_url
- https://test-job-queue-client.vault.azure.net/
Generate environment configuration
export db_password="cm9vdA==" #root export db_password_urlencoded="cm9vdA==" export test_azure_client_secret_base64=$(printf "%s" "$test_azure_client_secret"| base64 --wrap=0) export test_aws_secret_access_key_base64=$(printf "%s" "$test_aws_secret_access_key"| base64 --wrap=0) ./set-env.sh envsubst < provisioning/environments.yaml.template > provisioning/environments.yaml kubectl apply -f ./provisioning/environments.yaml kubectl apply -f ./provisioning/public-api.yaml queue_public_api_ip=`kubectl get svc --output jsonpath --template "{.items[?(@.metadata.name==\"dev-job-queue-api-service\")].status.loadBalancer.ingress[].ip}" --namespace=dev-job-queue-api-php-client` echo "queue_public_api_url: http://$(queue_public_api_ip):94"
Store the result queue_public_api_ip
in set-env.sh
.
Run tests
-
With the above setup, you can run tests:
docker compose build source ./set-env.sh && docker compose run tests
-
To run tests with local code use:
docker compose run tests-local composer install source ./set-env.sh && docker compose run tests-local
License
MIT licensed, see LICENSE file.