Argus:
the fleet that tends itself.

I started my career in operational excellence, the discipline that runs the hyperscale cloud. Define the service level before the incident. Measure what the customer feels, not only what the machine reports. Write the runbook, then automate the runbook, until a person is needed only for the part that is genuinely new.

Argus takes that discipline somewhere new: to devices on shelves and walls rather than servers in racks, and to a new instrument. A frontier model can read a body of evidence, say what it means and propose a remedy. It can also be wrong, expensive and eager to please. So the craft is no longer to put a model in front of a dashboard. It is to decide, precisely, where a model may reason, where arithmetic must decide, and where a person keeps the last word.

Argus is that decision, written as code and enforced by it. It looks after the Hubs and Halos that run Helios, in a single home or across a portfolio of sites. For a household, that means a Hub that is tended without sharing a single frame of video. For an operator, it means a fleet that states its own condition truthfully, and says so when it does not know. It holds itself to a short set of principles. Here is each one, as the code enforces it.

Argus ingests numbers: CPU pressure, memory, disk fill, SoC temperature, container state, tunnel health, the battery and heap of a Halo, the state of every patch. It has no ingest path for video, images or audio. A camera’s picture is summarised on the Hub into brightness, sharpness, noise and clipping, and only those figures travel. Camera passwords and account passwords are never telemetry. Hubs do not even ship their logs.

The agent that does this work is small enough to forget: about 60 MB of memory and under one percent of a CPU, a metrics collector and a sender that only ever speaks outward, through the Hub’s own tunnel. The next Halo firmware signs every telemetry packet per device with HMAC-SHA256, so a stranger who guesses a device name can neither read its command queue nor rewrite its history. And when a person genuinely needs a terminal on a Hub, the owner opens the session. The session key is stored only as KMS ciphertext, the operator steps up with a passkey, the launch token lives for sixty seconds and works once, and every step lands in an append-only audit log.

We measured our own fleet first. A Hub was emitting around 3,300 series every thirty seconds, much of it noise: log lines recast as metrics, tunnel decisions labelled by path and address, eight CPU modes across eight cores. The tiered model, which arrives with the redesign, drops that noise at the source. The Hub holds a single relabel file and swaps it whenever Argus reassigns the tier; it keeps no state of its own, so one patch serves the whole fleet. By that arithmetic, a Hub at the default tier sends about a thirteenth of the samples it did: roughly 500 series a minute against 3,300 every thirty seconds. Our design target is ten thousand Hubs at that tier on a single VictoriaMetrics node. That is a sizing calculation, not yet a measurement.

The point is not thrift for its own sake. A lean pipeline is one you can afford to leave on, everywhere, indefinitely, and the detail arrives exactly when the engine decides it needs it. A Hub that exceeds the budget for its tier raises a finding of its own.

Health is scored per signal against bands from the SRE canon. Below its warning threshold a signal scores 100; between warning and critical it falls to 50; beyond critical, to zero. Argus then takes the minimum across subsystems, because a weighted average lets a dying disk hide behind a healthy CPU. A signal that has gone quiet is shown as unknown and left out of the roll-up, so a silent Hub can never score a perfect 100. Thresholds carry hysteresis, so a value sitting on a boundary does not flap. And where our hardware differs from the textbook, as with 8 GB of memory and no swap, the thresholds follow the hardware.

Above that sits the anticipatory layer, the wisdom engine, which arrives with the redesigned Argus. It compares CPU, memory and camera bitrate against each Hub’s own seven-day baseline using a z-score, so a Hub is judged against itself and not against an average of strangers. It forecasts the day a disk will fill. Each rule is one fleet-wide query that runs every minute, so a fleet of two Hubs and a fleet of two thousand cost the same number of round trips to the database.

The order matters: rules first, anomaly detection second, the model last. More than thirty rules cover the Hub, its tunnel, CPU, memory, thermals, disk, network, the controller services, cameras, Halos, patches and the fleet’s own backups. Symptoms fold under their cause. When a Hub goes offline, its tunnel, cameras, Halos and patch reports are the same story told five times, so Argus tells it once, and each corroborating symptom raises the confidence of the cause.

Only then does a language model get a turn, and a narrow one: a root cause at high severity, or one that explains at least two other findings, once per distinct body of evidence, cached for six hours, capped at twenty calls an hour. Its prompt carries the rule’s facts and nothing else. It never sets severity and it never executes anything. If the key is missing or the budget is spent, the panel shows the rule’s own plain sentence instead.

Every finding carries evidence you can open as a query, and a prepared fix labelled by how much it could hurt: safe for read-only or idempotent actions, guarded for a restart or a clean-up that can be reversed, and destructive for a reboot, a deletion or a firmware flash, which always waits for the owner. Argus prepares the command. A person, or a policy a person set, runs it.

Seven playbooks cover the faults that recur: memory and disk pressure, a hot CPU, a container in a restart loop, a stopped service, network errors and runaway Docker logs. Each carries a risk level. Low-risk repairs may run on their own; medium and high stop and ask. A device is limited to three repairs an hour with a fifteen-minute cool-down, so a fix that does not take cannot become a loop. And self-healing is off until an owner switches it on, per device or for the fleet.

HolmesGPT, the open-source AI investigator, works alongside it. An operator asks a question in plain language and it reasons over live evidence; its runbooks for stuck video processing, memory pressure, a full disk, a stopped service and an offline Hub each name the remedy and whether that remedy needs approval.

A good council does not act on enthusiasm. Each member arrives with evidence, every course is laid on the table, the dangerous ones are argued down, and only then does anyone volunteer to carry the plan. Every week, the Argus council sits on that pattern. Its brief is the fleet’s global trends and its evolving issues. Its members are agents, and the stakes decide when a person joins.

Evidence first. Before any model reads a word, a deterministic pass reduces each group of Hubs to a digest of at most 15 KB, keeping the week’s trends, drift, change points and anomalies and discarding more than 99.5 percent of the raw rows. A fast model triages the digests. Regional synthesisers then read what survives, in batches, and each arrives at the table with a regional case: a firmware build that runs warm, a container that restarts after a particular update, a region where tunnels drop in the same hour.

Every course argued, the unsafe ones refused. A final author reasons across every region and drafts the playbook. Debate between the agents is capped at two rounds, so deliberation cannot become a loop. A separate guardrail model, reserved for what is novel or wide in its blast radius, argues against the plan. Then an entity validator does what no persuasive paragraph can: it rejects any playbook that names a Hub, container or metric that is not in the digest. A cost ceiling aborts the run, and a kill switch, a single file on disk, halts the council before its next model call or database write.

Qualified before it moves. Every run can be a dry run: every phase exercised, every outward effect withheld, no pull request and no email, and that is how the council ships until it is deliberately armed. Each playbook states its blast radius, and the change-control tier counts the Hubs it touches; more than five is Tier 3 without discussion. Confidence below 90 percent moves a plan one tier closer to a person, and a pattern the fleet has not seen before is forced to the top tier, however harmless it looks.

Autonomy runs as far as the risk allows. An approved Tier 1 or Tier 2 change is opened as a pull request into the patch repository every Hub already trusts; the publisher has no path to merge its own work. Tier 3 never becomes a pull request at all: it is queued for a person to author and to approve. What follows the merge is a staged release over the air, which the next chapter describes. Reversible, single-Hub changes travel with the least ceremony. Anything fleet-wide, irreversible or novel waits for a person.

Beside the council works the Fellowship, the same family of agents pointed at problems no threshold would catch because they have no fixed shape. Camera brands and models behave differently, and differently again with each firmware revision. Hardware efficiency drifts against the firmware running on the active Halos. Older cameras lose faces and plates at dusk. Each is a moving target, and each is fed by evidence the wisdom engine already gathers: clarity, bitrate and stream stability per camera, and the firmware version labelled on every Hub and Halo. The Fellowship reasons over that evidence, looks for the change that would make each perform better than it did last month, and drafts it as the next patch for the council to qualify.

OTA is how the fleet is repaired before anyone notices, and how it improves while its owners sleep. Proactive patches come from the Fellowship and the council, reasoned from fleet-wide evidence: a fix for a trend before it becomes an incident, a better setting for a camera model, a tuned firmware for the Halos. Reactive patches answer a finding that has already fired. Both run overnight in local time, so a home or a site is not asked to notice them.

The mechanism is deliberately plain. Each Hub checks the manifest, fetches the patch, applies it, and reports back what it applied or what failed, with the version, checksum, exit code and log. Every patch ships with its own rollback script and, where it matters, a health check. A patch can be aimed at a subset of Hubs. The redesigned Updates page shows every Hub and Halo against its target in one of five states: failed, rolling, pending, current or offline, with one action per row. A Halo firmware rollout is staged: our own lab first, then a small share of the fleet, then everyone.

The result is a fleet that stays healthy without asking for attention, and a Helios whose models grow more accurate over time, because the same channel that carries a fix carries a better model and a better setting for the camera in front of it.

Older cameras struggle in the same few ways: low resolution, poor light, and narrow-aperture lenses that starve the sensor at dusk. Argus scores every camera for clarity and says why in plain words: too dark, glare, soft focus, too wide for a face to be usable, a frame rate that has dropped, a vehicle camera with no plate check on record. Each reason comes from numbers the Hub measures against that camera’s own history, so a camera is judged against what it can do, not against a newer one.

That knowledge changes what we can do for it. It tells us which cameras, and which conditions, Helios finds hardest, and that is where we point the work on face and licence-plate recognition: low resolution, low light, the soft edges of an older lens. Every gain returns to the fleet over the same OTA channel and pushes the sunset date of those cameras later. Each sighting carries richer metadata, and the decision built on it, whether to sound the siren or to tell a person, lands with more precision and deters more.

Argus takes its design language from Helios, and the redesign borrows it outright: Inter on white, generous space, the same device images, and one colour that means one thing. A blue dot is healthy, amber needs a look, red is an error, white is offline or unknown, and yellow is a rollout in progress. Each dot pulses once and settles. The same dot sits on the Hub, camera and Halo images from the Helios app, on map markers and in lists.

Everything opens in a drawer from the right, so you never lose your place in the fleet; the back button closes it, and the link to a drawer opens the same drawer for a colleague. On our own fleet today, the whole view loads from fifteen cached queries in about half a second. And search is a sentence: hubs in Canada with high sustained CPU. A deterministic parser handles places, statuses, thresholds and time windows first. A language model is consulted only for the words it did not recognise, with a five-second limit, and each interpretation appears as a chip you can remove.

Autonomy comes first because a machine is consistent and awake at three in the morning. A person comes next because a person is accountable, and sees context that no telemetry carries. In Argus the hand-over is explicit: the engine finds, explains and prepares; a low-risk, reversible repair runs by itself once it has been enabled; anything fleet-wide, novel or irreversible is routed to someone with the authority to say yes.

We hold Argus itself to the same standard. Its server is defined in code and backed up hourly, and a pipeline can rebuild it from the latest backup into an isolated standby with no inbound access, as a drill. That pipeline is built and planned; its first drill is pending. The Environments page in the redesign tells the operator, without ceremony, where each environment stands: which side serves, whether builds match, how far replication lags, when the last failover drill ran and what it measured, and which CI runs are running or failing across the organisation. An unknown value is shown as unknown, never as healthy.