hyperstack: watch only polls VMs whose API port is currently reachable

Add watch_config_loaders that filters status_config_loaders results with
a TCP probe on each VM's WireGuard inference port. VMs with stale state
files (deleted from the console without `hyperstack.rb delete`) are
excluded from the watch loop. Falls back to all tracked loaders when
none are reachable so the watcher can still render error output when
WireGuard is down.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

1 files changed, 25 insertions, 4 deletions

diff --git a/lib/hyperstack/cli.rb b/lib/hyperstack/cli.rb
index 8568474..f4d1cef 100644
--- a/lib/hyperstack/cli.rb
+++ b/lib/hyperstack/cli.rb
@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require 'optparse'
+require 'socket'
 
 module HyperstackVM
   class CLI
@@ -181,11 +182,11 @@ module HyperstackVM
       )
     end
 
-    # Starts the VllmWatcher dashboard for all active VMs.
-    # Reuses status_config_loaders so it auto-discovers the same set of VMs
-    # that `status` would show (honours --config if given explicitly).
+    # Starts the VllmWatcher dashboard restricted to VMs that are currently reachable.
+    # Uses watch_config_loaders instead of status_config_loaders so VMs whose state
+    # files are stale (e.g. deleted from the console without `delete`) are excluded.
     def run_watch
-      loaders = status_config_loaders
+      loaders = watch_config_loaders
       raise Error, 'No active VMs found. Run `create` or `create-both` first.' if loaders.empty?
 
       VllmWatcher.new(config_loaders: loaders).run
@@ -211,6 +212,26 @@ module HyperstackVM
       build_manager(loaders.first.config).show_local_wireguard(expected_ips)
     end
 
+    # Returns only the loaders for VMs whose inference API port is currently reachable.
+    # Falls back to all state-tracked loaders when none are reachable (e.g. WireGuard down),
+    # so the watcher can still render meaningful error output instead of raising.
+    def watch_config_loaders
+      loaders   = status_config_loaders
+      reachable = loaders.select { |l| vm_api_reachable?(l.config) }
+      reachable.empty? ? loaders : reachable
+    end
+
+    # Quick TCP probe on the VM's inference port via WireGuard.
+    # A successful connect (immediately closed) means the API is up; any network
+    # error means the VM is down or unreachable — exclude it from the watch loop.
+    def vm_api_reachable?(config)
+      TCPSocket.new(config.wireguard_gateway_hostname, config.ollama_port).close
+      true
+    rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH, Errno::ETIMEDOUT,
+           Errno::ENETUNREACH, SocketError
+      false
+    end
+
     def status_config_loaders
       return [ConfigLoader.load(@config_path)] if @config_explicit