Tail sampling with the Collector
Oriel ingests every span it receives; it has no in-ingest tail sampling. When you
need to keep error and slow traces while dropping the bulk of healthy ones, run
the OpenTelemetry Collector’s tail_sampling processor in front of Oriel and
export the sampled output over OTLP.
Tail sampling buffers the spans of a trace until the trace is complete, then
applies its policies to the whole trace. That requires all spans of a trace to
reach the same Collector instance: run a single sampling Collector, or shard with
a loadbalancing exporter keyed on trace ID ahead of a sampling tier.
Requirements
Section titled “Requirements”- An OpenTelemetry Collector that receives the full trace before export
- An Oriel ingest token that permits traces
- Enough Collector memory to hold traces during
decision_wait
Collector configuration
Section titled “Collector configuration”receivers: otlp: protocols: grpc: http:
processors: tail_sampling: decision_wait: 10s policies: [ { name: errors, type: status_code, status_code: { status_codes: [ERROR] } }, { name: slow, type: latency, latency: { threshold_ms: 500 } }, { name: baseline, type: probabilistic, probabilistic: { sampling_percentage: 5 } }, ]
exporters: otlp/oriel: endpoint: oriel.example.com:4317 headers: Authorization: Bearer ${ORIEL_INGEST_TOKEN}
service: pipelines: traces: receivers: [otlp] processors: [tail_sampling] exporters: [otlp/oriel]The decision_wait is how long the Collector holds a trace’s spans before
deciding; size it above your longest expected trace duration so late spans are
not evaluated against an already-flushed decision. Only sampled traces are
forwarded, so Oriel stores the kept subset.
Verify in Oriel
Section titled “Verify in Oriel”Open Explore and query recent traces:
traces | sort start_time desc | limit 20Then confirm that expected error traces still arrive:
traces | where error_count > 0 | limit 20