Monitoring Kafka
Kafka does not speak a monitoring protocol of its own: its internals are behind JMX, which
the agent does not touch. The definition instead relies on kafka_exporter, the widely
used Prometheus exporter. When the exporter is listening, the agent scrapes it and checks
the broker port; when it is not, Kafka stays unmonitored and this page tells you why.
How Kafka is recognised
The signed Kafka definition matches a host with something listening on 9308/tcp, the
default port of kafka_exporter. The exporter is the evidence that a Kafka setup exists
and the source of every metric, so a broker without it does not match. Once matched, two
probes run every minute:
- a TCP connection to
127.0.0.1:9092, the broker's default listener, GET /metricson127.0.0.1:9308, parsed as a Prometheus exposition.
Five metric families of the exporter are mapped to Wakora names; the rest of the exposition is ignored.
What is collected
Cluster shape. The number of live brokers and the partition count of every topic.
Consumer health. Lag per consumer group and topic, the number that says how far behind a consumer is, plus the current offset of every topic partition.
Replication. Under-replicated partitions per topic: a partition whose follower fell out of the in-sync set, so a broker loss would cost data.
| Signal | Metric |
|---|---|
| Live brokers | svc.kafka.brokers |
| Partitions per topic | svc.kafka.partitions tagged by topic |
| Latest offset | svc.kafka.current_offset tagged by topic and partition |
| Consumer group lag | svc.kafka.consumer_lag tagged by consumergroup and topic |
| Under-replicated partitions | svc.kafka.under_replicated tagged by topic |
Names, tags and units are in the metrics reference.
There is no Kafka log definition; a broker or exporter running in Docker ships its output through the Docker log definition under the image name.
Alerts out of the box
| Rule | Fires when | Severity | Ships |
|---|---|---|---|
| Kafka partitions under-replicated | a topic has at least one under-replicated partition for 5 minutes | warning | on |
Under-replication is the one Kafka signal that means the same thing everywhere: the cluster
is one failure away from losing writes on that topic. The five-minute hold lets a rolling
broker restart pass, during which followers routinely leave and rejoin the in-sync set. The
rule opens one incident per topic, so a single sick topic does not blur into the rest. No
lag rule ships, because the lag that matters is different for a batch pipeline and a
payment stream; put a custom rule on svc.kafka.consumer_lag with the tag key
consumergroup in the console (/alerting/rules) once you know your baseline. Thresholds
and hysteresis are in the rules catalog.
Limitations
kafka_exporteris required, and it has to run on the monitored host. Nothing is read over JMX, so broker-internal numbers (request latency, garbage collection, log flush times, disk usage per log directory) are not collected.- The broker check dials
127.0.0.1:9092. A broker on another port or another host is not checked, even though the exporter metrics still flow; the check then reports the refused connection. - Each metric family carries at most 200 series per scrape. On a cluster with more topic and partition combinations than that, the surplus series are dropped; the per-partition offset metric is the one most likely to reach the cap.
- Lag and offsets are whatever the exporter computes; the agent adds no interpretation of its own.
- Matching is by the exporter's port, not by a Kafka process, so the exporter has to keep
its default port
9308.
Related
- Monitoring RabbitMQ and Monitoring NATS
- Monitoring Docker for a broker that runs as a container
- Features on the main site