#Elasticsearch

#Introduction

Elasticsearch version is automatically retrieved from root/ping endpoint. Based on this version Jaeger uses compatible index mappings and Elasticsearch REST API. The version can be explicitly provided via version: config property.

Elasticsearch does not require initialization other than installing and running Elasticsearch. Once it is running, pass the correct configuration values to Jaeger.

#Configuration

A sample configuration for Jaeger with Elasticsearch backend is available in the Jaeger repository: config-elasticsearch.yaml. In the future the configuration documentation will be auto-generated from the schema. Meanwhile, please refer to config.go as the authoritative source.

#Shards and Replicas

Shards and replicas are some configuration values to take special attention to, because this is decided upon index creation. This article goes into more information about choosing how many shards should be chosen for optimization.

#Index Rollover

Elasticsearch rollover is an index management strategy that optimizes use of resources allocated to indices. For example, indices that do not contain any data still allocate shards, and conversely, a single index might contain significantly more data than the others. Rollover feature can be enabled by use_aliases: true config property.

Rollover lets you configure when to roll over to a new index based on one or more of the following criteria:

• max_age - the maximum age of the index. It uses time units: d, h, m.
• max_docs - the maximum documents in the index.
• max_primary_shard_size - the maximum size of a single primary shard. It uses byte size units tb, gb, mb.

To learn more about rollover index management in Jaeger refer to this article.

#Initialize

The following command prepares Elasticsearch for rollover deployment by creating index aliases, indices, and index templates:

nerdctl run -it --rm --net=host \
  build-harbor.alauda.cn/asm/jaeger-es-rollover:2.16.0-r1 \
  init http://localhost:9200

If you need to initialize archive storage, add -e ARCHIVE=true.

After the initialization Jaeger can be deployed with use_aliases: true.

#ILM support

Elasticsearch ILM automatically manages indices according to performance, resiliency, and retention requirements.

ILM support is an alternative to the manual rollover + lookback + index-cleaner workflow. When ILM is enabled, Elasticsearch manages rollover and retention automatically according to the configured policy.

For example:

• Rollover to a new index by size (bytes or number of documents) or age, archiving previous indices
• Delete stale indices to enforce data retention standards

To enable ILM support:

1. Create an ILM policy in Elasticsearch named jaeger-ilm-policy.

   For example, the following policy will rollover the "active" index when it is older than 1m and delete indices that are older than 2m.

   curl -X PUT \
     http://localhost:9200/_ilm/policy/jaeger-ilm-policy \
     -H 'Content-Type: application/json; charset=utf-8' \
     --data-binary @- << EOF
   {
     "policy": {
       "phases": {
         "hot": {
           "min_age": "0ms",
           "actions": {
             "rollover": {
               "max_age": "1m"
             },
             "set_priority": {
               "priority": 100
             }
           }
         },
         "delete": {
           "min_age": "2m",
           "actions": {
             "delete": {}
           }
         }
       }
     }
   }
   EOF

2. Run the Elasticsearch initializer with ES_USE_ILM=true:

   nerdctl run -it --rm --net=host \
     -e ES_USE_ILM=true \
     build-harbor.alauda.cn/asm/jaeger-es-rollover:2.16.0-r1 \
     init http://localhost:9200

   If you need to initialize archive storage, add -e ARCHIVE=true.

   WARNING

   While initializing with ILM support, make sure that an ILM policy named jaeger-ilm-policy is created in Elasticsearch beforehand (see the previous step), otherwise the following error message will be shown:

   "ILM policy jaeger-ilm-policy doesn't exist in Elasticsearch. Please create it and rerun init"

After the initialization, deploy Jaeger with use_ilm: true and use_aliases: true.