Collecting OTLP traces, Prometheus-style metrics and Docker logs (via Logspout) in a single Signoz setup #7831

AlexeyTozik · 2025-05-05T09:47:40Z

AlexeyTozik
May 5, 2025

Hi everyone! 👋

I’m trying to monitor a fleet of Dockerized services across multiple hosts with Signoz. My requirements:
Traces from all my applications via OTLP (gRPC+HTTP)
Metrics from both my apps (OTLP) and the host OS (hostmetrics + the collector’s own Prometheus endpoint)
Logs from only those containers whose name contains ncip, shipped via Logspout → tcplog/docker → OTLP

I’ve put together this compose + collector config on each host:

docker-compose.yaml

services:
  otel-collector:
    image: signoz/signoz-otel-collector:0.79.7
    container_name: signoz-otel-collector
    command:
      - "--config=/etc/otel-collector-config.yaml"
      - "--feature-gates=-pkg.translator.prometheus.NormalizeName"
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml:ro
    env_file:
      - ./.env
    environment:
      OTEL_RESOURCE_ATTRIBUTES: "host.name=signoz-host,os.type=linux"
      DOCKER_MULTI_NODE_CLUSTER: "false"
      LOW_CARDINAL_EXCEPTION_GROUPING: "false"
    ports:
      - "4317:4317"   # OTLP gRPC (traces)
      - "4318:4318"   # OTLP HTTP (traces)
      - "8888:8888"   # collector internal + hostmetrics
      - "2255:2255"   # logspout → collector (tcplog)
    restart: on-failure
    networks:
      signoz-net:
        aliases:
          - otel-collector

  logspout:
    image: gliderlabs/logspout:v3.2.14
    container_name: signoz-logspout
    volumes:
      - /etc/hostname:/etc/host_hostname:ro
      - /var/run/docker.sock:/var/run/docker.sock
    command: syslog+tcp://otel-collector:2255
    depends_on:
      - otel-collector
    restart: on-failure
    networks:
      - signoz-net

networks:
  signoz-net:
    driver: bridge

otel-collector-config.yaml

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "0.0.0.0:4317"
      http:
        endpoint: "0.0.0.0:4318"

  hostmetrics:
    collection_interval: 60s
    scrapers:
      cpu: {}
      memory: {}
      disk: {}
      filesystem: {}
      network: {}
      load: {}
      paging: {}
    root_path: "/"

  prometheus:
    config:
      global:
        scrape_interval: 60s
      scrape_configs:
        - job_name: otel-collector
          static_configs:
            - targets: ["localhost:8888"]

  tcplog/docker:
    listen_address: "0.0.0.0:2255"
    operators:
      - type: regex_parser
        regex: '^<([0-9]+)>[0-9]+ (?P<timestamp>[0-9T:\.Z\+\-]+) (?P<container_id>\S+) (?P<container_name>\S+) \d+ - -( (?P<body>.*))?$'
        timestamp:
          parse_from: attributes.timestamp
          layout: '%Y-%m-%dT%H:%M:%S.%LZ'
      - type: move
        from: attributes["body"]
        to: body
      - type: remove
        field: attributes.timestamp

processors:
  batch:
    send_batch_size: 1000
    send_batch_max_size: 1100
    timeout: 10s

  resourcedetection:
    detectors: [env, system]
    timeout: 2s
    system:
      hostname_sources: [os]

exporters:
  otlp:
    endpoint: ${env:OTEL_COLLECTOR_ENDPOINT}
    tls:
      insecure: false
    headers:
      signoz-ingestion-key: ${env:SIGNOZ_INGESTION_KEY}

  logging:
    loglevel: debug

extensions:
  health_check: {}
  zpages: {}

service:
  telemetry:
    metrics:
      address: "0.0.0.0:8888"

  extensions: [health_check, zpages]

  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]

    metrics:
      receivers: [otlp, prometheus, hostmetrics]
      processors: [batch, resourcedetection]
      exporters: [otlp]

    logs:
      receivers: [tcplog/docker]
      processors: [batch]
      exporters: [otlp]

Collector logs show it’s receiving nothing on tcplog/docker (even though Logspout is bound and sees containers).
In SigNoz UI I only see collector-internal metrics, not my apps’ host-metrics or Docker logs from ncip-* containers.
When I switch Logspout back to bridge mode, it only captures containers in that one bridge network (not useful when apps live in many networks).

Is this the “right” pattern to run Logspout in host mode and filter within the collector?
Should I instead be using the filelog receiver to tail /var/lib/docker/containers/.../*.log?
How can I debug whether tcplog/docker is actually getting any data at all?
Any best-practice examples for exactly this “multi-network, host-metrics + OTLP + Logspout” scenario?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Collecting OTLP traces, Prometheus-style metrics and Docker logs (via Logspout) in a single Signoz setup #7831

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Collecting OTLP traces, Prometheus-style metrics and Docker logs (via Logspout) in a single Signoz setup #7831

Uh oh!

AlexeyTozik May 5, 2025

Replies: 0 comments

AlexeyTozik
May 5, 2025