Observability · Platform & infrastructure

Metrics, logs and traces, wired to one cluster

Personal engineering work · public repository

Focus: Observability · Platform & infrastructure
Evidence: 3 documented proof points
Decision record: Firsthand trade-off included

The brief

What was built

Prometheus, Grafana and Alertmanager for metrics, an EFK pipeline for logs and Jaeger for traces on one EKS cluster — with two instrumented Go services routing OTLP through an OpenTelemetry Collector rather than exporting to each backend directly.

Evidence

Implemented evidence

One OTLP path per service instead of one exporter per backend: the Collector fans metrics out to Prometheus and traces to Jaeger, so swapping a backend is a Collector config change rather than a code change
Request counters, duration histograms and an active-request up/down counter per service, with trace context propagated across service-to-service HTTP calls
Jaeger backed by Elasticsearch so traces outlive the pod that produced them, and Fluent Bit shipping cluster logs into the same store

Decision record

The applications speak OTLP; the platform chooses the backend

I instrumented the Go services to emit metrics and traces through OTLP to the OpenTelemetry Collector instead of coupling application code directly to Prometheus or Jaeger. The Collector exposes metrics for Prometheus and forwards traces to Jaeger, while Fluent Bit keeps Kubernetes log collection on a separate path into Elasticsearch. Elasticsearch also persists Jaeger traces beyond the lifetime of the pods that produced them. That boundary makes an observability backend replaceable through platform configuration rather than application changes, but it makes the Collector and Elasticsearch shared dependencies that need high availability, capacity planning, retention controls and security before production.

Return to all work