Monitoring Elasticsearch
Elasticsearch is recognised by the port it listens on, and everything the agent needs comes
from the HTTP API the node already serves on 127.0.0.1:9200. No plugin, no exporter, no
credentials on a node that answers plain HTTP without authentication.
How Elasticsearch is recognised
The signed definition matches a host with a listener on 9200/tcp. Every minute the agent
opens a TCP connection to 127.0.0.1:9200 and makes three GET requests:
/_cluster/healthfor node counts, shard states, pending tasks, the active-shards percentage and the cluster status;/_stats/docs,storefor the document count and store size;/for the version.
The requests are plain HTTP with no credentials.
What is collected
Cluster health. Nodes and data nodes, active, unassigned and relocating shards, pending cluster tasks, the active-shards percentage, and the status as two flags: red and yellow. Red means a primary shard is missing; yellow means every primary is there but a replica is not.
Data. The document count and store size in bytes that _stats reports.
Facts shown on the host page: version, cluster name and the current cluster status.
| Signal | Metric |
|---|---|
| Nodes and data nodes | svc.elasticsearch.nodes, svc.elasticsearch.data_nodes |
| Shards active / unassigned / relocating | svc.elasticsearch.active_shards, svc.elasticsearch.unassigned_shards, svc.elasticsearch.relocating_shards |
| Active shards percentage | svc.elasticsearch.shard_health_pct |
| Cluster status flags | svc.elasticsearch.status_red, svc.elasticsearch.status_yellow |
| Pending cluster tasks | svc.elasticsearch.pending_tasks |
| Documents and store size | svc.elasticsearch.docs, svc.elasticsearch.store_bytes |
Names and units are in the metrics reference.
Alerts out of the box
| Rule | Fires when | Severity | Ships |
|---|---|---|---|
| Elasticsearch cluster is red | the status flag reads red for 5 minutes | warning | on |
| Elasticsearch shards unassigned | one or more shards are unassigned for 15 minutes | warning | off |
Red is the state where data is not fully readable, so it ships on; five minutes of it filters the moment a node restarts and its primaries come back. The unassigned-shards rule ships off because of the single-node case: a lone node hosting indices with replicas has nowhere to place those replicas, so it reports yellow with unassigned shards forever while serving every query correctly. Switch it on for a multi-node cluster, where an unassigned shard means a copy of your data is missing. Thresholds and hysteresis are in the rules catalog.
Limitations
- The stock definition speaks plain HTTP to
127.0.0.1:9200without credentials. A cluster with security enabled, or one serving HTTPS on that port, is not covered. - Only a node bound to the local interface on port 9200 is reached; the address is fixed in the definition.
- The document count and store size are the totals
_statsreports; there is no per-index breakdown, and JVM heap, garbage collection, thread pools and indexing latency are not collected. - No log collection ships for Elasticsearch; there is no logs definition for it.
- The yellow status flag is collected but has no built-in rule.
Related
- Monitoring MongoDB, Monitoring Redis
- Monitoring Kafka and Monitoring RabbitMQ for the rest of a data pipeline
- Features on the main site