Elastic ILM Phases/Actions, Data Tiers, Rollover Aliases/Data Streams, and Explain/Troubleshooting

Learning outcomes

AtlasMart now wants the lifecycle intent from Lesson 1 implemented in Elasticsearch. The target is not merely “create an ILM policy”; it is to understand phase execution, data-tier placement, alias/data-stream rollover, cached phase definitions, explain output and safe error recovery.

1. ILM phase execution is a state machine with cached phase definitions

ILM uses five phase names: hot, warm, cold, frozen and delete. An index can progress only when the current phase actions complete and the next phase age requirement is met. If rollover occurs, later min_age calculations are relative to rollover time rather than the original creation time.

When an index enters a phase, Elasticsearch caches that phase definition in index metadata. A later policy edit increments policy version, but ILM only applies changes to the cached phase if they are safe. Otherwise the index finishes the phase using the cached definition and sees the newer policy when it enters a later phase. This is why “I edited the JSON” does not mean every managed index instantly follows the new behavior.

2. Build the disposable AtlasMart ILM policy

PUT _ilm/policy/atlasmart-life-es-v1
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": { "max_docs": 3 }
        }
      },
      "warm": {
        "min_age": "0ms",
        "actions": {
          "forcemerge": { "max_num_segments": 1 }
        }
      },
      "delete": {
        "min_age": "30d",
        "actions": { "delete": {} }
      }
    }
  }
}

PUT _index_template/atlasmart-life-es-template-v1
{
  "index_patterns": ["atlasmart-life-es-*"],
  "template": {
    "settings": {
      "index.number_of_shards": 1,
      "index.number_of_replicas": 0,
      "index.lifecycle.name": "atlasmart-life-es-v1",
      "index.lifecycle.rollover_alias": "atlasmart-life-es"
    },
    "mappings": {
      "properties": {
        "@timestamp": {"type": "date"},
        "service.name": {"type": "keyword"},
        "log.level": {"type": "keyword"},
        "message": {"type": "text"}
      }
    }
  }
}

PUT atlasmart-life-es-000001
{
  "aliases": {
    "atlasmart-life-es": {"is_write_index": true}
  }
}

3. Rollover alias versus data stream

An ILM rollover target may be a data stream or a compliant index alias. For an alias, the backing index name must have a numeric suffix such as -000001, the alias must point to it, and it must be the write index. A data stream carries its own backing-index/write-index model and is often the cleaner choice for append-heavy telemetry.

Do not copy alias rollover settings into a data stream blindly. Chapter 10 established data-stream semantics; this chapter keeps an alias deliberately so the same lifecycle intent can be compared with OpenSearch ISM.

POST atlasmart-life-es/_doc
{"@timestamp":"2026-09-11T12:00:00Z","service.name":"checkout","log.level":"INFO","message":"order accepted"}
POST atlasmart-life-es/_doc
{"@timestamp":"2026-09-11T12:00:01Z","service.name":"checkout","log.level":"WARN","message":"inventory retry"}
POST atlasmart-life-es/_doc
{"@timestamp":"2026-09-11T12:00:02Z","service.name":"checkout","log.level":"INFO","message":"order confirmed"}

GET atlasmart-life-es-000001/_ilm/explain?human
GET _cat/aliases/atlasmart-life-es?v
GET _cat/indices/atlasmart-life-es-*?v

ILM polls periodically. The expected invariant is eventual creation of a later generation and removal of write status from -000001 once rollover completes. Exact wall-clock timing is not guaranteed by the document ingest request.

4. Data tiers and migration

Elastic data tiers are node roles such as data_hot, data_warm, data_cold and data_frozen. ILM’s migrate action updates tier preference for warm/cold placement and is automatically injected in supported phases unless disabled. On a one-node lab there is no meaningful cost tier to demonstrate—the same machine owns all work—so the correct lesson is to inspect settings, not to invent a tier-performance improvement.

GET atlasmart-life-es-000001/_settings?filter_path=*.settings.index.routing.allocation.include._tier_preference
GET _cluster/allocation/explain
{
  "index": "atlasmart-life-es-000001",
  "shard": 0,
  "primary": true
}

In production, a tier migration is complete only if shard allocation actually moves to eligible capacity and query/recovery SLOs remain acceptable. A phase label alone does not prove placement.

5. Explain, error, retry, and dangerous manual movement

GET atlasmart-life-es-*/_ilm/explain?human
GET atlasmart-life-es-*/_ilm/explain?only_errors=true&human
GET _ilm/status

_ilm/explain identifies the policy, phase, action, step, timing and failure metadata. If an index enters ERROR, repair the root cause first, then POST /index/_ilm/retry. Do not treat retry as a root-cause fix.

6. Security and managed-service boundaries

ILM executes with the privileges associated with the user who last updated the policy, so lifecycle automation can fail after role/permission changes. Elastic Cloud Serverless does not expose ILM in the same cluster-management form; data stream lifecycle is the simpler serverless lifecycle mechanism. Do not present a self-managed ILM runbook as a Serverless runbook.

The next lesson implements the same retention intent with OpenSearch ISM and shows why the JSON and state semantics differ.

Summary

Elastic ILM combines phase/action execution with rollover and data-tier placement. Correct operation is demonstrated through explain, settings and allocation evidence—not policy presence alone. The next lesson maps the same business intent to OpenSearch ISM.

Authoritative references

• Elastic index lifecycle management — ILM scope, availability, and lifecycle concepts for indices and data streams.
• Elastic ILM phases and actions — Hot/warm/cold/frozen/delete phases, cached phase execution, and transition rules.
• Elastic Explain lifecycle API — Current phase/action/step, failures, and phase execution evidence.
• Elastic data tiers — Content/hot/warm/cold/frozen roles and lifecycle placement concepts.
• Elastic policy updates — Policy versions and cached phase-definition behavior.
• Elastic ILM troubleshooting — ERROR steps, retry behavior, and safe diagnosis.
• OpenSearch Index State Management — ISM model, job cadence, policy attachment, and managed-index workflow.
• OpenSearch ISM policies — States, actions, transitions, rollover, force merge, allocation, and templates.
• OpenSearch ISM API — Policy OCC, explain, retry, change-policy, and simulation APIs.
• OpenSearch ISM error prevention — Pre-action validation and explain diagnostics.
• OpenSearch artifacts by version — Current OpenSearch release baseline.
• Elasticsearch downloads — Current Elasticsearch release baseline.