Incomplete Trace Data
TOC
Problem Description
The trace query results exhibit the following issues of incomplete data:
- Recent queries (within the last 30 minutes) are missing some spans.
- Trace data older than 1 hour are experiencing disconnections.
Root Cause Analysis
1. Data Persistence Delay
The writing process in Elasticsearch requires a sequence of steps involving memory buffer → translog → segment files, which can result in visibility delays for the most recently written data.
2. Time Range Limitation
By default, when jaeger-query queries spans corresponding to traces, the time range extends one hour before and after the start time of the span.
For instance, if a span starts at 08:12:30 and ends at 08:12:32, the time range for querying that trace would be from 07:12:30 to 09:12:32.
Therefore, if the trace spans over 1 hour, querying through this span may not yield a complete trace.
Solution for Root Cause 1
Wait a moment and refresh the page to try the query again.
Solution for Root Cause 2
If the trace span in your environment is lengthy, you can adjust the query time range for a single trace using the --es.asm.span-trace-query-time-adjustment-hours startup parameter in jaeger-query.
The default value of this parameter is 1 hour, and you can increase this value as needed.