Skip to content

Tail sampling with the Collector

Oriel ingests every span it receives; it has no in-ingest tail sampling. When you need to keep error and slow traces while dropping the bulk of healthy ones, run the OpenTelemetry Collector’s tail_sampling processor in front of Oriel and export the sampled output over OTLP.

Tail sampling buffers the spans of a trace until the trace is complete, then applies its policies to the whole trace. That requires all spans of a trace to reach the same Collector instance: run a single sampling Collector, or shard with a loadbalancing exporter keyed on trace ID ahead of a sampling tier.

  • An OpenTelemetry Collector that receives the full trace before export
  • An Oriel ingest token that permits traces
  • Enough Collector memory to hold traces during decision_wait
receivers:
otlp:
protocols:
grpc:
http:
processors:
tail_sampling:
decision_wait: 10s
policies:
[
{ name: errors, type: status_code, status_code: { status_codes: [ERROR] } },
{ name: slow, type: latency, latency: { threshold_ms: 500 } },
{ name: baseline, type: probabilistic, probabilistic: { sampling_percentage: 5 } },
]
exporters:
otlp/oriel:
endpoint: oriel.example.com:4317
headers:
Authorization: Bearer ${ORIEL_INGEST_TOKEN}
service:
pipelines:
traces:
receivers: [otlp]
processors: [tail_sampling]
exporters: [otlp/oriel]

The decision_wait is how long the Collector holds a trace’s spans before deciding; size it above your longest expected trace duration so late spans are not evaluated against an already-flushed decision. Only sampled traces are forwarded, so Oriel stores the kept subset.

Open Explore and query recent traces:

traces | sort start_time desc | limit 20

Then confirm that expected error traces still arrive:

traces | where error_count > 0 | limit 20