dpc-sdp/tide_logs

Log forwarder for SumoLogic OTel collector.

Maintainers

Details

github.com/dpc-sdp/tide_logs

Source

Installs: 20 881

Dependents: 1

Suggesters: 0

Security: 0

Stars: 0

Watchers: 9

Forks: 0

Type:drupal-module

2.0.1 2024-10-08 06:21 UTC

This package is auto-updated.

Last update: 2024-10-08 22:39:25 UTC


README

Provides a SumoLogic handler for Monolog.

Requirements

There is also a soft dependency on the following modules

Activation

  1. The module requires a working Sumo Logic OTEL Collector to which it can send logs.
    1. A helm chart has been created here for easy installation on Kubernetes clusters.
    2. To test locally, in a test project:
      1. Create a file at .docker/sumologic-otel/config.yaml with the following content:
        extensions:
          sumologic:
            # Get from https://service.au.sumologic.com/ui/#/security/installation-tokens
            install_token: <collector-install-token>
             # Highly recommend adding a unique suffix, otherwise you might run into conflicts with other collector instances.
            collector_name: SDP Syslog - OTEL - <random-suffix>
            clobber: true
        
        receivers:
          udplog:
            listen_address: "0.0.0.0:514"
            attributes:
              source_name: "foobar/otel/test"
            operators:
              - type: json_parser
              - type: metadata
                id: metadata/source_host
                if: '"source_host" in $$body'
                attributes:
                  source_host: 'EXPR($$body.source_host)'
              - type: metadata
                id: metadata/source_category
                if: '"source_category" in $$body'
                attributes:
                  source_category: 'EXPR($$body.source_category)'
        
        processors: {}
        
        exporters:
          logging:
            loglevel: debug
          sumologic:
            auth:
              authenticator: sumologic
            source_category: "%{source_category}"
            source_name: "%{source_name}"
            source_host: "%{source_host}"
            metadata_attributes:
              - source.*
        
        service:
          extensions: [sumologic]
          pipelines:
            logs:
              receivers: [udplog]
              exporters: [sumologic]
        
      2. Add a service to docker-compose.yml as follows:
        sumo-otel:
          image: "public.ecr.aws/sumologic/sumologic-otel-collector:${SUMO_OTEL_RELEASE_VERSION:-0.47.0-sumo-0}"
          volumes:
            - .docker/sumologic-otel:/etc/otel
          ports:
            - 514
          networks:
            - amazeeio-network
            - default
          labels:
            lagoon.type: none
      3. Running ahoy up will register the instance with Sumo Logic; it's now ready to start collecting logs. Run docker-compose logs -f sumo-otel to make sure there were no errors. The last line should be something like
        Everything is ready. Begin running and processing data.
        
  2. Enable the Tide Logs module. Go to /admin/config/development/tide_logs and ensure UDPlog host and UDPlog port correspond to the service's name and the port in the config.yaml respectively, if running with docker-compose. The module's default is udp://logger.default.svc:5514 (see the default config here).
  3. SUMOLOGIC_CATEGORY can also be set if a different category from the default (sdp/dev/tide) is required.
  4. The following search query can then be used in SumoLogic to view the logs:
    _collector="SDP Syslog - OTEL - <random-suffix>" and _sourceCategory="sdp/dev/tide"
    

Debug

Some very basic debug messages can be printed locally (or remotely if you have drush access) by setting the following config variable:

drush config:set tide_logs.settings debug 1