Device Logs

Pings tell you that something broke. Only the device itself tells you why. A router that logs "ether3 link down" at 21:03, a radio that logs "DFS radar detected", a tower that logs a reboot, an admin login two minutes before everything went wrong - that is the evidence a diagnosis needs, and your equipment is already writing it.

Device logs let the probe listen for it. It accepts syslog from the devices you name, recognises what the messages mean, and keeps the raw stream on your own hardware. What reaches the panel is counts and categories, never a stream of text.

What you get

• A "Device log" block on a host page and a link page: at most five lines, in plain words. "ether3 link down, 6 times, last at 21:07".
• Better diagnoses. A fault the device itself reported is more certain than one only the pings saw, and a site that kept writing log lines all the way through an outage did not lose power, whatever the pings looked like.
• Diagnoses nothing else can make: a configuration change right before a fault, a device rebooting over and over, a DHCP pool running out, someone trying passwords on your router.
• Flapping the pings cannot see. A port that drops for twenty seconds is invisible to a five-minute measurement cycle. The device logs it anyway.

Switching it on

Two steps, both in Settings > Network monitoring > Device logs.

First, turn the feature on for the workspace in the card at the top of the tab. That card also holds the two switches worth thinking about: whether log events may raise alerts on their own, and whether identifiers (MAC addresses, PPPoE usernames, customer IP addresses) are hashed before they ever leave your site.

Then turn on the collector for a probe with the switch on its card, and tell it who is allowed to talk to it under Edit (Accept from, one network per line). Nothing else is accepted: besides the networks you list, the probe accepts only the subnets its discovery sweeps and the devices on its part of your map. A device outside all of that is counted and shown as an unknown sender, never stored.

If you do not yet know which devices will send, press Learning mode for an hour. For that hour the probe also accepts private addresses outside the list, so you can point a few devices at it and see who arrives. It expires on its own; there is no way to leave it on.

Publishing the port

The container listens on port 1514, not the standard 514. Ports below 1024 need a privilege the probe image deliberately does not carry, so the standard port is something the host publishes. Add this to the probe's docker run line:

-p 514:1514/udp

Add -p 514:1514/tcp as well if devices send over TCP. Devices then send to the usual port and nothing on them needs to know a container exists.

Docker cannot add a port to a running container: remove it and run it again with the extra lines. The ispbox-probe volume keeps the local archive. A docker run line the panel shows after device logs are on (a new key for the probe, or a new probe) already carries both lines.

If the probe runs as a container on a MikroTik, there is no host to publish the port. The router itself sends straight to the container's address on port 1514 (172.20.0.2 with the install script's default subnet). Every other device sends to the router on port 514, and a dst-nat rule on the router passes it on:

/ip/firewall/nat/add chain=dstnat protocol=udp dst-port=514 in-interface-list=LAN \
    action=dst-nat to-addresses=172.20.0.2 to-ports=1514 comment="ispbox-probe device logs"

Pointing devices at it

The Turn logging on at the device card at the bottom of the tab generates a ready paste for each kind of equipment. Enter the address the device sends to and the port (514 unless you published something else), pick the tab for the equipment, press Copy and paste it into the device.

MikroTik RouterOS

One logging action and a list of topics. Two things in that paste are deliberate and worth keeping:

• The account topic. It is what makes "somebody changed the configuration two minutes before this broke" possible, and it is the single most useful line in the list.
• No debug. One router on debug can use up the probe's whole rate limit by itself, and there is nothing in the catalogue that a debug line is the only source of.

The paste is written for current RouterOS 7, where the logging action uses remote-log-format=syslog add-topics-string=yes. Older firmware spells it differently, and the topics only reach the probe with the right spelling. Without them the MikroTik rules recognise nothing:

• Current RouterOS 7: remote-log-format=syslog add-topics-string=yes, as in the paste.
• RouterOS 7 releases without add-topics-string (7.19, for example): remote-log-format=default instead.
• RouterOS 6: bsd-syslog=no instead. With bsd-syslog=yes RouterOS 6 leaves the topics out.

On a MikroTik the panel already manages over the API you do not need to work this out. Open the router in Settings > Routers and press Configure logging on this router. The dialog shows the exact commands before anything is sent, and Run these commands tries the three spellings in that order until the router accepts one. It touches nothing else on the router, pins src-address when the router has more than one address, and running it again updates the existing ispbox action instead of adding a second one. The button appears once device logs are on for the workspace and at least one probe is listening, and it needs the settings permission.

The paste also asks for ISO timestamps. That is not cosmetic: the old BSD syslog format carries neither a year nor a timezone, so a device sending it cannot be checked against your clock at all, and around new year its timestamps can land twelve months out. The two older spellings above send no timestamp of their own, so a router on them gets no clock figure.

Ubiquiti airOS and airMAX

Either the web interface under Services > System Log, or the commands in the paste over SSH. If you use SSH, do not skip the cfgmtd line: without it the setting disappears the moment the radio restarts.

Cambium

ePMP, cnPilot and PMP450 each have a syslog server field in their own configuration page; the paste names the exact path for all three. On ePMP the GPS sync and DFS messages are the ones worth having, because they explain a whole sector going quiet at once.

Anything else

Point it at the probe and it will work. Both common syslog formats are understood, and so are devices that follow neither properly.

What the probe does with a message

Every line is matched against rule sets for MikroTik, Ubiquiti, Cambium and a general-purpose set, and turned into one of about forty categories: a link going down, a reboot, a radar hit, a pool running out, a configuration change, a failed login. Repeats are counted rather than repeated: a port flapping 300 times in a minute is one entry that says 300.

A message no rule recognises is not thrown away. It is stripped down to its shape - numbers, addresses and names replaced with placeholders - counted, and listed under Unrecognised messages in the same tab. That list is useful twice: you can see that a device has started saying something new, and each entry has a Suggest a rule action where you pick the event those messages mean, and that suggestion goes into the next rule set (or Ignore, to take it off the list). Rules are delivered from the panel, so a rule written today reaches every probe on its next check-in, without anybody rebuilding a container.

Where the raw lines live

On your probe, not in the panel. The full text of every message stays in the probe's own storage, with its own retention (30 days by default) and its own slice of the disk (a quarter by default). Logs are always the first thing removed when space runs short, so a device someone left on debug can cost you log history but never a measurement.

When you need the actual text, open a host page and press Open the raw window. The panel asks the probe for that one window, shows it, and keeps nothing. Reading it requires the settings permission and is recorded in the audit log, because those lines can contain customer usernames and addresses.

Alerts

Most log events do their work quietly, by making a diagnosis more or less certain. Only four situations raise an alert on their own, and only on infrastructure devices, never on a customer's router:

• an infrastructure device rebooting or losing power,
• a configuration change on one,
• a DHCP pool running out,
• a burst of failed logins.

The reason for that restraint is worth knowing: ordinary syslog is not authenticated, so anyone on your network could in principle send a message claiming anything. Treating what a device says as evidence rather than as proof is the honest way to use it. If that matters to you, the probe also accepts syslog over TLS, and the tab shows you the fingerprint to pin on the device.

Things that surprise people

"It says the event list was cut short." The probe aggregates at most 2,000 different events per minute. Past that, the line counts stay right but the list for that minute is incomplete - and the card says so rather than showing a short list silently. The raw minute is still on the probe; open the raw window to read it.

"A device is muted." A device that floods the probe for three minutes is muted for ten and reported. That is deliberate: the alternative is a full disk. Turn its logging down, usually by removing a debug topic.

"A sender appeared in Discovery." Something is sending syslog from an address that is not on your map. That is a device you probably want on the map, which is why it is proposed there.

"It says nothing about a device's clock." Only timestamps that carry a timezone can be compared with anything. A device sending the old BSD format gets no clock warning at all, rather than a wrong one.

This is not a log search. There is no query language and no months of text kept in the panel. Device logs exist to explain what the monitoring already noticed, and to catch the handful of things pings cannot see.