Metrics & Observability

Weld gives you metric primitives and a renderer — not a magic endpoint. You declare the metrics you care about, update them in handlers, and expose them from a route you own, so you control the path, the listener, and the auth. Nothing proprietary appears on your public port.

metric orders_total: counter        # monotonic count
metric queue_depth:  gauge          # a value that goes up and down
metric pay_seconds:  histogram      # a distribution (latency, sizes, …)

route POST "/orders" (body o: Order) -> string {
    orders_total.inc()
    queue_depth.inc()
    respond "queued"
}

route POST "/pay" (body p: Payment) -> string {
    let t0 = now_ms()
    # … do the work …
    pay_seconds.observe(0.042)
    respond "paid"
}

listen admin { port 9090 }                 # keep metrics off the public port
route GET "/metrics" on admin -> string {  # YOUR endpoint — path, listener, auth are yours
    type "text/plain; version=0.0.4"
    respond metrics()                       # render everything as Prometheus text
}

Metric kinds

Kind	Methods	Use for
counter	.inc(), .inc(n)	things that only go up — requests, errors, bytes
gauge	.set(v), .inc([v]), .dec([v])	a current value — queue depth, connections, temperature
histogram	.observe(v)	a distribution — latency, payload size (fixed Prometheus buckets)

Counter increments are integers; gauge and histogram values accept ints or floats. Updates are lock-free atomics, safe to call from any handler (routes, gRPC, hubs, every timers).

metrics()

metrics() returns a string in Prometheus exposition format containing:

• Built-in request stats (always collected, a few atomics per request) — full RED per route, labelled by the route template so cardinality stays bounded: weld_http_requests_total{method="GET",route="/users/:id",status="2xx"}, weld_http_request_duration_seconds{method="GET",route="/users/:id"} (a histogram with _bucket/_sum/_count), and weld_http_requests_in_flight (a gauge of concurrent requests). Requests that match no route are counted under route="<unmatched>".
• Outbound dependency RED — the same rate/errors/duration for every upstream HTTP call, Postgres query, and outbound grpc call, so you can see which dependency is slow or failing: weld_upstream_requests_total{target="pay.charge",outcome="ok|error"} + weld_upstream_request_duration_seconds, weld_db_queries_total{query="db.findUser",…} + weld_db_query_duration_seconds, and weld_grpc_requests_total{target="pkg.Svc/Method",…} + weld_grpc_request_duration_seconds.
• Your declared metrics, each with its # TYPE line.
• Topic drop counters — weld_topic_dropped_total{topic="…"} for every broadcast topic, so a slow subscriber is visible.

Put it behind any route. Scrape it with Prometheus, curl, or anything that speaks the text format. Because it's a normal handler, you can gate it with a decorator (API key, IP allowlist) or bind it to a private listener as above.

Request logging

WELD_LOG controls per-request logging (separate from metrics):

• WELD_LOG=1 — a plain line per response: weld: GET /users 200 1.23ms.

• WELD_LOG=json — one structured JSON object per response, with a monotonic correlation id and the request's trace_id, for log pipelines:

  {"level":"info","msg":"request","id":42,"trace_id":"4bf92f3577b34da6a3ce929d0e0e4736","method":"GET","path":"/users","status":200,"duration_ms":1.234}

Trace context

Every request participates in a W3C trace: Weld reads an incoming traceparent header and continues its trace-id (minting a fresh span-id), or starts a new trace when there's none. Two builtins expose it (lowercase hex):

• trace_id() — the 32-char trace id (shared across the whole distributed request).
• span_id() — the 16-char span id for this service's work.

Use them to correlate your own logs, echo an id back to a caller, or stamp records:

route GET "/whoami" -> string {
    header "x-trace-id" trace_id()
    respond "handled by trace {trace_id()}"
}

Both HTTP routes and gRPC service methods join an incoming trace (or start one), and the trace-id already appears in every WELD_LOG=json line, so those logs line up with an upstream gateway's trace.

Trace propagation

Every outbound call carries the current trace forward — Weld injects a traceparent header (this service's span as the caller's parent) into upstream HTTP requests and grpc calls automatically. So a request flowing gateway → service A → service B → database shares one trace-id end to end, with no code changes. There's nothing to configure; a call made outside a request context (e.g. a background worker) simply omits the header.

OTLP span export

Set OTEL_EXPORTER_OTLP_ENDPOINT and Weld exports spans to any OTLP/HTTP collector (Tempo, Jaeger, Grafana Alloy, Honeycomb, …) — so traces are viewable, not just correlated in logs. Each request produces a server span, plus a nested client span for every outbound upstream HTTP call and Postgres query, giving a real latency waterfall (request → db.findUser, request → upstream.pay). The client span becomes the parent the downstream service sees, so the trace stays connected end to end.

OTEL_EXPORTER_OTLP_ENDPOINT=http://collector:4318 \
OTEL_SERVICE_NAME=checkout \
./checkout

Each span carries the trace/span/parent ids, start & end time, and http.request.method, url.path, http.response.status_code attributes (a 5xx marks the span's status ERROR); service.name comes from OTEL_SERVICE_NAME (default weld). Spans are buffered and flushed by a background thread as protobuf (application/x-protobuf) every ~2 s, so export never sits in the request path. OTEL_EXPORTER_OTLP_TRACES_ENDPOINT overrides the full traces URL; when neither variable is set, export is off and there's zero overhead. On an abrupt exit up to one flush interval of spans may be dropped.