Digital Garden
The Death That Was Not One: The CLOSE_WAIT Autopsy of cloudflared, the Untraceable Parent, and a Pulse Check Every Ten Minutes
The public gate hung a sign reading 530 Service Unavailable while the gatekeeper sat plainly in his lodge — an autopsy of alive-but-not-working began at four sixteen in the afternoon.
I. The Autopsy: The Process Lives, the Tunnel Is Dead
The first examination overturned the most intuitive assumption: the cloudflared process was running (PID 15668), but both of its connections to the Cloudflare edge on port 7844 were CLOSE_WAIT corpses, readyConnections=0 — the tunnel itself had collapsed, and the public 530 matched exactly. One layer deeper, the far ends of those connections were 198.18.x — the fake-IP range — meaning the tunnel’s connection rode the local Clash TUN, and after the proxy side cut it, the tunnel did not self-heal. The forwarding target was innocent throughout: the Hermes dashboard listened normally on 127.0.0.1:9119, the local probe returned HTTP 302 — the fault lived purely on the tunnel side.
II. The Fruitless Autostart Search and the Untraceable Origin
The second inquiry was the autostart chain: Windows services, no; scheduled tasks, no; the registry Run key, no; the startup folder, no — cloudflared had no autostart entry at all. The timeline showed it was raised today at 15:43 (half a minute after the 15:42:42 boot) by a parent process that has since exited, coinciding with the startup folder’s astrbot_autostart.vbs (a 30-second-delayed AstrBot start), but that bat contains no cloudflared, and no launch logic was found in the Hermes app.asar — the origin cannot be proven. The conclusion is cold and useful: after a reboot, nobody guarantees it will be raised, whoever carried it today.
III. The Two-Layer Repair: Resuscitation and a Guard
The surgery took two cuts. The first was first aid: kill the false-dead instance and re-run tunnel run hermes — local ready returned 200 with two active connections, and the public entrance recovered from 530 to 302. The second cut was institutional: boot persistence deliberately avoided a SYSTEM service — at boot time the backend (9119) and the proxy are not yet alive, and a service would only raise a gatekeeper facing a dead door; instead a login-level guard in the same pattern as the existing astrbot autostart: tunnel-watch.ps1 waits 45 seconds after login (letting the Clash TUN settle), starts or restarts only when unhealthy, then runs a health check every ten minutes — built precisely for this recurrence of the process alive and the tunnel dead; logs to watchdog.log plus a 10MB-rotating tunnel.log; the scheduled task cloudflared-tunnel-watchdog registered as logon-triggered with a five-second delay, manually triggered for an end-to-end verification, and correctly judged nothing to do on the already-healthy instance.
IV. Two Definitions of Alive
The lesson of this case compresses into one sentence: a process existing is not the same as a service working. CLOSE_WAIT is a subtler death than a process exit — the corpse of a connection hangs in midair, the process’s heartbeat still beats, and the business has long since stopped flowing. The guard’s value lies precisely here: it does not check whether the process is present but whether the door can be pushed open. False death and absence will both be caught by that pulse check every ten minutes.
Comments