Skip to main content

Monitoring NATS

NATS is recognised by its client port and read through the monitoring endpoint the server already exposes. Nothing to install, no credential to store: if the monitoring port is enabled, the numbers appear within a minute of the agent seeing the port.

How NATS is recognised​

The signed NATS definition matches a host with something listening on 4222/tcp, the NATS client port. A server from a package and a server in a container with the port published both qualify. Once matched, two probes run every minute:

  • a TCP connection to 127.0.0.1:4222, which has to be accepted,
  • GET /varz on the monitoring port, 127.0.0.1:8222, parsed for the server-wide counters.

The monitoring endpoint is unauthenticated by design in NATS, so no secret is involved.

What is collected​

Clients. Current connections and the cumulative count of connections since start, plus the number of active subscriptions. From the cumulative count the agent derives new connections per minute: on a healthy deployment clients connect once and stay, so that rate is the reconnect-storm signal.

Throughput. Messages received and messages delivered, both cumulative, and the per-second rates derived from them on the host.

Trouble signals. Slow consumers, the counter NATS increments when a client cannot keep up and the server drops it, as a total and per minute, and the server's memory footprint.

Facts shown on the host page: the NATS server version.

SignalMetric
Current and total connectionssvc.nats.connections, svc.nats.total_connections
New connections per minutesvc.nats.total_connections_per_min
Active subscriptionssvc.nats.subscriptions
Slow consumers, total and per minutesvc.nats.slow_consumers, svc.nats.slow_consumers_per_min
Messages in / out (cumulative)svc.nats.in_msgs, svc.nats.out_msgs
Messages in / out per secondsvc.nats.in_msgs_rate, svc.nats.out_msgs_rate
Server memorysvc.nats.mem_bytes

Names, tags and units are in the metrics reference.

There is no NATS log definition. A server running in Docker has its output shipped through the Docker log definition under the image name.

Alerts out of the box​

RuleFires whenSeverityShips
NATS has slow consumershalf a slow consumer per minute or more, sustained 5 minuteswarningauto
NATS clients keep reconnecting30 or more new connections a minute for 10 minuteswarningauto

Both rules watch rates the agent derives on the host, so they fire while the trouble lasts and close when it stops. A slow consumer every two minutes for five minutes is a client that cannot keep up, not a one-off hiccup. The reconnect rule has a floor to respect: the monitoring probe itself opens about one connection a minute, so the threshold sits well above that; thirty a minute is clients in a reconnect loop, which usually means a network path or an authentication problem rather than the server. Message rates are only meaningful against your own baseline and ship without a rule; svc.nats.in_msgs_rate dropping to zero during business hours is the usual custom rule. Thresholds and hysteresis are in the rules catalog.

Limitations​

  • The server has to expose its HTTP monitoring endpoint on 8222, which NATS does not do unless configured to. Without it the port check passes and the metrics probe fails, which the check reports.
  • A NATS in a container must publish both 4222 and 8222 for the host to see them.
  • JetStream is not read: streams, stream consumers and storage usage are not collected.
  • The numbers are server-wide. Per-account and per-connection detail, and the state of other cluster members, are not collected.
  • Message and connection counts are cumulative counters; read the charts for rates rather than the raw values.
  • Matching is by port. Anything else listening on 4222 matches the definition and fails the metrics probe.