Compare commits

...

9 Commits

Author SHA1 Message Date
teknium1
bde66ef37a docs(egress): align user + dev docs with iron-proxy v0.39 actual behavior
The previous docs round (906b1da57) described the integration the way
we wanted it to work — `http_listens` plural with a docker bridge
bind, dedicated `audit.log` for per-request JSON records.  Live
testing against the real v0.39.0 binary in [905ce58a1] surfaced that
neither field exists in v0.39's config schema, and the docs were
making promises the daemon couldn't keep.

This commit walks every claim in the docs back to what the binary
actually does today, while keeping the upgrade path explicit so the
docs stay coherent when the pinned `_IRON_PROXY_VERSION` bumps:

website/docs/user-guide/egress/iron-proxy.md
- Bind policy section: rewritten.  Was "loopback + docker bridge IP
  on Linux"; now "loopback only" with an explicit explanation that
  v0.39 only supports one bind per daemon and that
  host.docker.internal -> host-gateway mapping is what sandboxes
  use to reach the loopback bind.
- Bind policy section adds a note on the metrics-port pin that the
  previous round of docs didn't even mention.
- State directory layout table: `audit.log` description rewritten
  to acknowledge it's a pre-created sentinel for future binary
  versions, NOT something the v0.39 daemon writes to.
- New section "Logging on iron-proxy v0.39" replaces the old
  "Audit log vs daemon log" section.  Explicitly tells operators
  the daemon log is the single source of truth for both audiences
  on v0.39, with the upgrade path called out.
- Data-flow diagram step 7: rewritten to send per-request records
  to `iron-proxy.log` on v0.39 with cross-link to the new logging
  section.
- Diagram caption updated.
- Security-model "allowlisted-host exfiltration" line: "audit log
  captures" -> "daemon log captures".
- Security-model "LAN peer leak" line: removed the docker-bridge
  claim.
- Troubleshooting section's per-request-inspection recipes:
  rewritten to use `iron-proxy.log` and explain when the split
  stream will land.
- Limitations list gets a new bullet calling out the
  single-bind + combined-log v0.39 constraints + the auto-upgrade
  posture.

website/docs/developer-guide/egress-internals.md
- Bind policy invariant: documents the singular `http_listen` v0.39
  schema constraint + dead-code-until-upgrade status of the
  bridge-bind path.
- New "Metrics port collision" invariant documenting why
  `metrics.listen: 127.0.0.1:0` is non-negotiable.
- Audit log fail-loud invariant adds the v0.39 schema constraint
  note + the new
  `test_audit_log_kwarg_does_not_inject_audit_path_v039` regression
  test.
- "Subscribing to per-request audit events" section updated to
  send watchers at `iron-proxy.log` for v0.39 with the upgrade
  pivot called out.

website/docs/reference/cli-commands.md
- Diagnostic shortcut for tailing the audit log: `tail audit.log |
  jq` -> `tail iron-proxy.log | jq` with the v0.39 note inline.

Build verification:
- `npx docusaurus build` succeeds across all three locales
  (en + zh-Hans + ko).
- New `#logging-on-iron-proxy-v039` anchor lands in the rendered
  HTML and the in-page cross-references resolve.
- No new broken anchors introduced (pre-existing warnings on
  unrelated zh-Hans pages are unchanged).
- No leftover stale `#audit-log-vs-daemon-log` or `#http_listens`
  references anywhere on the egress pages.
2026-05-29 00:34:36 -07:00
teknium1
905ce58a12 fix(egress): align proxy.yaml with iron-proxy v0.39 actual schema +
propagate handler exit codes

Live testing the full wizard against the real v0.39.0 binary
(downloaded + extracted via our own install_iron_proxy()) surfaced
three real bugs that the unit tests couldn't catch:

1. `proxy.http_listens` (plural) — NOT a field in v0.39's config struct.
   Our code emitted both `http_listen` (string) and `http_listens`
   (list) believing v0.39 accepts both forms.  The binary actually
   rejects with "field http_listens not found in type config.Proxy"
   at YAML unmarshal time, so the daemon fails to start.  Confirmed
   via strings(1) audit of the v0.39 binary — only `http_listen` is
   tagged.

2. `log.audit_path` — NOT a field in v0.39's config.Log struct.  Same
   class of error: "field audit_path not found in type config.Log".
   Per-request audit-log records are not separable from server-level
   logs at this binary version.

3. `metrics.listen` defaults to ":9090" — which is the SAME port as
   our default `tunnel_port: 9090`.  Result: every operator who runs
   `hermes egress setup` followed by `hermes egress start` gets
   "bind: address already in use" because the proxy listener and the
   metrics listener fight for port 9090.  We now explicitly pin
   `metrics.listen: 127.0.0.1:0` to give it an ephemeral loopback
   port that can never collide with tunnel_port regardless of what
   operator sets.

Plus a fourth bug — pre-existing but surfaced by the egress live
test — that affects every Hermes subcommand:

4. `hermes_cli/main.py` calls `args.func(args)` at the bottom of
   main() but discards the return value.  Every subcommand handler
   that returns a non-zero exit code (cmd_start refusing because
   `fail_on_uncovered_providers=true`, cmd_setup refusing because
   --from-bitwarden but BWS unreachable, etc.) was silently exiting 0.
   Fix: capture the handler's return value and `sys.exit(rc)` when
   it's a non-zero int.  Other subcommands' contracts unchanged
   because they either return 0/None or don't return at all.

Validation:
- 188/188 in test_iron_proxy.py + test_iron_proxy_cli.py +
  test_config.py pass post-fix.
- 5333/5337 in tests/hermes_cli/ pass; the 4 unrelated failures
  (test_managed_installs.py + test_update_hangup_protection.py)
  are pre-existing on main, not touched by this PR.
- Manual wizard run end-to-end with the v0.39.0 binary in an
  isolated HERMES_HOME:
    * `egress install` — downloads + SHA-256 verifies + extracts
    * `egress setup` — generates CA, mints tokens, writes
      proxy.yaml that the binary now accepts (no http_listens,
      no audit_path, metrics pinned to 127.0.0.1:0)
    * `egress start` — daemon binds 127.0.0.1:9090, listens=yes
    * `egress status` — shows pid + listening + mappings
    * `egress stop` — clean shutdown, pidfile + nonce removed
    * Idempotent re-start returns the running pid without spawning
    * curl through the proxy with the openrouter token gets
      forwarded; an attacker host gets HTTP 403 (allowlist works);
      169.254.169.254 gets HTTP 403 (deny CIDR works)
    * Refuse-start paths exit 1 with actionable messages:
      - `fail_on_uncovered_providers=true` + ANTHROPIC_API_KEY set
      - `credential_source=bitwarden` + BWS_ACCESS_TOKEN unset
    * `--rotate-tokens` confirmation gate fires via pty:
      typing 'cancel' aborts; typing 'rotate' proceeds and
      creates a mappings.json.rotated-<timestamp> backup

Test updates:
- `test_default_bind_is_loopback_not_zero_zero` — asserts the
  singular `http_listen` is loopback AND asserts `http_listens`
  (plural) is NOT in the rendered yaml.
- `test_default_bind_uses_loopback_on_linux` — replaces
  `test_default_bind_includes_docker_bridge_on_linux`.  v0.39
  only supports one bind per daemon process, so the docker bridge
  augmentation is dropped from the rendered config; sandboxes
  reach the daemon via host.docker.internal -> host-gateway
  mapping, so loopback-only is functional.
- `test_metrics_listener_pinned_to_loopback_ephemeral` — new
  regression test asserting `metrics.listen == "127.0.0.1:0"`.
- `test_audit_log_kwarg_does_not_inject_audit_path_v039` —
  replaces `test_audit_log_path_lands_in_yaml`.  audit_log kwarg
  is still accepted for forward compatibility but does NOT emit
  log.audit_path until upstream supports it.
2026-05-28 23:45:44 -07:00
teknium1
ec108c625e Merge origin/main into feat/iron-proxy
Single content conflict in hermes_cli/config.py — kept BOTH the
paste_collapse_threshold knobs from main and the proxy section from
this branch (they're independent additions to DEFAULT_CONFIG).

All 187 tests in test_iron_proxy.py + test_iron_proxy_cli.py +
test_config.py pass post-merge.
2026-05-25 18:37:06 -07:00
teknium1
906b1da57f docs(egress): comprehensive expansion — setup, config, troubleshooting,
internals reference

Pre-v3 the egress docs were 175 lines covering the basics: quick start,
slash commands, security model, failure modes.  After three rounds of
PR review we added a half-dozen new config knobs, two new flags, a
strict/warn tier split for uncovered providers, persisted-nonce
cross-process defense, audit-log + log-file separation, NODE_OPTIONS
append-merge, docker_env collision detection, etc. — none of which
the user-facing doc reflected.

This commit closes that gap end-to-end:

website/docs/user-guide/egress/iron-proxy.md (175 → 567 lines)
- Configuration section expanded with every new knob:
  fail_on_uncovered_providers, allow_env_fallback, upstream_deny_cidrs.
- Tables for default allowed hosts + default deny CIDRs.
- Bind policy section (loopback + docker bridge, NOT 0.0.0.0) with the
  operator-facing "why can't I hit the proxy from my LAN" answer.
- Uncovered providers section with the strict tier (Anthropic / Azure
  / Gemini — block when fail_on_uncovered_providers=true) vs warn tier
  (AWS, GCP appdefault — present on every dev laptop, never block).
- Bitwarden integration expanded: rotation semantics, fail-loud at
  start, the allow_env_fallback escape hatch, --no-bitwarden flag, the
  preserve-existing-source rule on plain re-setup.
- Slash commands section with --no-bitwarden, --rotate-tokens, and the
  token-rotation operator playbook (confirmation gate, backup file
  naming, restart-required caveat).
- State directory layout table covering all 9 files we create + their
  modes.
- Audit log vs daemon log distinction (the arshkumarsingh #2 fix that
  motivated the corrected diagram).
- CA distribution into the sandbox: full table of injected env vars,
  the Python/curl REPLACE vs Node ADD asymmetry caveat with the
  NODE_OPTIONS=--use-openssl-ca mitigation.
- docker_env collision detection: what gets blocked, what gets warned,
  the migration escape hatch.
- PID + nonce defense section explaining how iron-proxy.nonce works
  cross-CLI and the SIGKILL-suppress-on-recycle path.
- Security model expanded with the new defenses
  (IPv4-mapped-v6 IMDS bypass closure, env-var leakage prevention,
  LAN-peer-with-token-leak coverage).
- Failure modes extended for every new refuse-start path.
- Troubleshooting section (180 new lines) with grep-friendly error
  matchers for each common failure: BWS token missing, uncovered
  provider refused, port collision, slow bind, 403 from proxy, SSL
  verification errors inside the sandbox, 401 from upstreams, address-
  in-use orphan recovery, per-request audit log inspection.

website/docs/getting-started/quickstart.md
- One-paragraph mention of the egress proxy under "Sandboxed terminal"
  so operators discover the feature when they enable Docker isolation.

website/docs/reference/cli-commands.md
- Top-level command table now lists `hermes egress` alongside `hermes
  proxy` (different purpose, different direction — call it out).
- New `## hermes egress` section with full subcommand syntax, common
  flows (first-time setup, switching credential source, rotating
  tokens, adding upstream), and diagnostic shortcuts.

website/docs/reference/environment-variables.md
- New "Egress proxy (sandbox-injected)" section documenting every env
  var the Docker backend injects: HERMES_EGRESS_PROXY,
  HERMES_PROXY_TOKEN_<NAME>, HTTPS_PROXY/HTTP_PROXY/NO_PROXY,
  REQUESTS_CA_BUNDLE/SSL_CERT_FILE/CURL_CA_BUNDLE/NODE_EXTRA_CA_CERTS,
  NODE_OPTIONS append-merge, HERMES_IRON_PROXY_NONCE.
- Also fixes a stale layout issue with the Persistent Shell table that
  had two trailing rows getting orphaned in the v3 commit.

website/docs/developer-guide/egress-internals.md (NEW, 363 lines)
- Module layout map (which file owns what).
- Full lifecycle walkthrough for install / setup / start / stop with
  the actual function calls in order.
- "Security invariants" section enumerating every load-bearing property
  with the regression test name that guards it.  These are the rules
  contributors must preserve when touching the module:
  - filesystem perms (0o700 dir, 0o600 secrets, O_NOFOLLOW everywhere)
  - subprocess env minimisation (no os.environ.copy)
  - bind policy (loopback + docker bridge, never 0.0.0.0)
  - default deny CIDR coverage
  - audit log fail-loud
  - bitwarden fail-loud
  - docker_env collision detection
  - PID recycling defense
  - token preservation on re-setup
  - credential_source preservation
- Extension points: adding a bearer-token provider, adding a
  non-bearer provider, wiring iron-proxy into a non-Docker backend,
  subscribing to per-request audit events.
- Testing recipe (hermetic + E2E + CLI smoke).

website/sidebars.ts
- New `developer-guide/egress-internals` entry under Developer Guide
  → Internals (alongside acp-internals, cron-internals,
  trajectory-format).

Build verification
- `cd website && npm install && npx docusaurus build` succeeds locally.
- All three new pages render to static HTML in all three locales
  (en + zh-Hans + ko).
- No new broken links or broken anchors introduced (pre-existing
  warnings on translation stubs are unrelated).
2026-05-25 15:05:16 -07:00
teknium1
fa4e87b253 fix(egress): v3 round — GodsBoy/stephenschoettler/arshkumarsingh findings
GodsBoy 2nd-round P1 (all 4 addressed):
- _detect_docker_bridge_ip: replace `ip.count('.') == 3` heuristic with
  ipaddress.IPv4Address validation + reject unspecified/loopback/multicast/
  reserved/link-local/global addresses.  Hostile `ip` shim on PATH used to
  be able to inject 0.0.0.0 here and re-open INADDR_ANY binding.
- cmd_setup credential_source preservation: re-running `hermes egress
  setup` without --from-bitwarden no longer silently downgrades a previous
  bitwarden config back to env.  Require --no-bitwarden to switch
  explicitly; otherwise preserve the existing mode and surface the
  decision.
- fail_on_uncovered_providers docstring/default mismatch: docstring used
  to claim default=True; behavior was default=False.  Resolved by
  truth-in-advertising — docstring now correctly states default=False —
  AND splitting providers into a strict LLM-specific tier
  (_LLM_SPECIFIC_NON_BEARER_PROVIDERS, used by start blocking) and a
  generic uncovered tier (used by wizard warnings).  Generic cloud creds
  (AWS_*, GOOGLE_APPLICATION_CREDENTIALS) no longer trip refuse-start
  for operators using terraform/gcloud alongside Hermes.  New
  discover_blocked_providers() returns the strict subset.
- start_proxy poll-loop must verify listening before pidfile:
  previously fell through deadline-expired as success and wrote a
  pidfile for a non-listening daemon.  Refactored into a do-while
  shape, require `listening=True` for success, kill the child + unlink
  the pidfile on failure paths.

GodsBoy 2nd-round P2 (the worth-keeping subset):
- O_NOFOLLOW + 0o600 + st_uid check on iron-proxy.log open (symmetric
  with the pidfile and audit-log paths the same PR hardens).
- pidfile O_EXCL: refactored pidfile-write into _write_pidfile_safely
  which uses O_EXCL to detect concurrent starts.  EEXIST with a live
  pid means "another start in progress" — refuse with actionable
  message; EEXIST with a dead pid means "stale crash" — unlink and
  retry once.  Discriminates rather than racing.
- _VERSION_CACHE: invalidate on install_iron_proxy success;
  don't cache empty stdout (would poison `hermes egress status` for
  the lifetime of the process if first probe hit a corrupt binary).
- ensure_audit_log now RAISES on OSError instead of swallowing it as
  a warning.  Previous behavior let the daemon create the file under
  the default umask, exactly the world-readable scenario the helper
  was built to prevent.  cmd_setup catches the new RuntimeError and
  surfaces "✗" with the actionable message.
- SIGINT/SIGTERM handler scoped around the start_proxy poll loop:
  Ctrl-C while waiting for `hermes egress start` no longer leaks an
  orphan daemon with the port bound.  Handler kills the child +
  unlinks the pidfile before re-raising.
- pidfile written IMMEDIATELY after Popen, BEFORE the listening
  verification.  Parent dying during the poll loop now leaves a
  pidfile pointing at the orphan so the next `hermes egress stop` can
  clean up.  Failure paths in the poll loop explicitly unlink.
- _DEFAULT_UPSTREAM_DENY_CIDRS: add ::ffff:0:0/96 (IPv4-mapped IPv6 —
  closes the v6-resolved IMDS bypass), 100.64.0.0/10 (CGNAT / cloud
  overlays / K8s pod networks), 198.18.0.0/15 (RFC2544 benchmark).
- _NON_BEARER_PROVIDERS split into LLM-specific (Anthropic / Azure /
  Gemini — block when strict) vs generic-cloud (AWS_*, GCP appdefault
  — warn-only).
- docker.py except narrowing: load_config can raise yaml.YAMLError on
  a malformed config.yaml, not just ImportError.  Two callsites
  (collision check + precedence resolution) now catch yaml.YAMLError
  via a sentinel `import yaml` and fail-safe to enforced mode.

GodsBoy 2nd-round P3:
- _reset_for_tests: was a no-op claiming symmetry with bitwarden;
  now actually clears _VERSION_CACHE and _proxy_nonce so in-process
  callers (notebooks, pytest -p no:xdist) don't see state leakage.
- tests/test_iron_proxy_cli.py: replaced hardcoded Path("/tmp/...")
  with hermes_home/-derived fixtures.  Matches the same cleanup we
  did for test_iron_proxy.py in the previous round.
- --rotate-tokens confirmation gate: when there are existing tokens,
  prompt for "rotate" confirmation (skipped when stdin isn't a tty
  so CI/scripted use still works) AND back up the mappings to a
  timestamped sibling before overwriting.  Surface a no-op note when
  rotate is requested with no existing tokens.

stephenschoettler (runtime-boundary review):
- #1 BWS silent degrade at proxy start: when credential_source=bitwarden
  but the BWS access token or project_id is missing OR the fetch
  returns no values for mapped providers, raise instead of silently
  falling back to host env.  cmd_start also pre-checks at the wizard
  layer for actionable error messages.  Opt-in escape hatch via new
  `proxy.allow_env_fallback: true` config for migration scenarios.
- #2 docker_env collision detection extended: `docker_env:
  {OPENROUTER_API_KEY: sk-real}` in config.yaml with enforce_on_docker:
  true now raises just like an HTTPS_PROXY collision would.  The
  collision check pulls mapped provider names from load_mappings() at
  call time.
- #3 PID nonce persisted to disk: cross-CLI-invocation stale-pidfile
  defense now works.  start_proxy writes the nonce next to the pidfile
  (sibling 0o600), stop_proxy reads it back via _read_persisted_nonce()
  and uses it as a _pid_alive signal in the new process.  Falls back
  to argv0 basename matching when the file is missing (legacy install).

arshkumarsingh:
- #1 NODE_OPTIONS append-merge: egress dict no longer sets NODE_OPTIONS
  directly (would clobber the operator's --max-old-space-size etc.).
  Carry the egress flag in a sentinel key
  _HERMES_EGRESS_NODE_OPTIONS_APPEND; DockerEnvironment merges into the
  existing NODE_OPTIONS in env_args computation with de-duplication.
- #2 docs: structured per-request audit log is at audit.log, not
  iron-proxy.log (the latter is daemon stdout/stderr).  Diagram and
  step-7 text corrected; both file roles are now documented separately.

Tests
- Added 12 new tests in test_iron_proxy.py covering bridge-IP rejection
  (parametrized over 8 dangerous inputs), default deny-list adjacency
  (IPv4-mapped-v6 + CGNAT), blocked-providers strict-subset property,
  _pid_proc_starttime parser with paren-containing comm,
  stop_proxy SIGKILL suppression on starttime drift, _reset_for_tests
  clear behavior, iron_proxy_version don't-cache-empty, NODE_OPTIONS
  sentinel verification, ensure_audit_log raise-on-OSError, and
  persisted-nonce roundtrip.
- Added 1 new test in test_iron_proxy_cli.py covering cmd_start
  BWS-token-missing fail-loud.
- All 100 tests in test_iron_proxy + test_iron_proxy_cli pass; all 78
  tests in test_docker_environment + test_config still pass.

Acknowledged but not addressed:
- GodsBoy P3 dead-code `extra_env` kwarg: kept (removing is a breaking
  change for any out-of-tree caller; the kwarg is documented and works).
- Residual risks GodsBoy called out: iron-proxy in-memory secret
  zeroisation (Go-binary territory, out of scope); _PROXY_SUBPROCESS_ENV
  _ALLOWLIST cosmetic gaps (RUST_LOG, GOMAXPROCS); follow-up.
2026-05-24 04:22:53 -07:00
teknium1
4833acf046 fix(egress): silence CodeQL clear-text-logging on bws warning strings
The bws helper's warnings list contains non-secret status messages
('rate limited', 'project not found', etc.), but CodeQL's taint
analyzer can't distinguish those from the secrets dict returned by
the same call.  Log the count instead of the strings — the warnings
are still observable via 'hermes secrets bitwarden status'.
2026-05-23 23:13:03 -07:00
teknium1
128a6837b7 fix(egress): address PR review findings — P0/P1/P2/P3 + CI greens
P0 — must-fix
- iron_proxy: emit default upstream_deny_cidrs (loopback, IMDS
  169.254.0.0/16, RFC1918) when caller passes None.  Honours the docs
  promise that cloud-metadata IPs are refused regardless of allowlist.
- iron_proxy: bind 127.0.0.1 (+ docker0 bridge IP on Linux) instead of
  INADDR_ANY (':9090').  LAN peers with a leaked sandbox token could
  otherwise spend the operator's API quota against any allowlisted
  upstream.
- ensure_ca_cert: write the CA private key via os.open(..., 0o600)
  instead of shutil.copy2+os.chmod — closes the TOCTOU window where
  the key existed under the default umask.
- discover_uncovered_providers + proxy.fail_on_uncovered_providers
  config: refuse to start (when strict) if env vars for non-bearer
  providers (Anthropic native x-api-key, AWS SigV4, Azure OpenAI,
  etc.) are present.  Surfaces a wizard warning in non-strict mode.

P1 — should-fix
- start_proxy: build a minimal subprocess env (PATH/HOME/locale +
  only the env names referenced by mappings) instead of os.environ
  .copy().  Strips proxy-recursion vars (HTTPS_PROXY etc.).  Stops
  the proxy's /proc/<pid>/environ from leaking every host secret
  to same-uid local processes.
- start_proxy: optional Bitwarden refresh path
  (refresh_secrets_from_bitwarden=True, bitwarden_config=...).
  When credential_source=bitwarden, cmd_start wires it in — that's
  what delivers the rotation guarantee the docs make.
- build_proxy_config: wire audit_log into the rendered yaml
  (log.audit_path).  Parameter was accepted but never used.
- ensure_audit_log: pre-create the audit log with 0o600 perms so
  iron-proxy inherits tight permissions instead of relying on umask.
- Rename 'hermes proxy ...' → 'hermes egress ...' in user-facing
  strings (docstring, RuntimeError messages, post-setup banner).
- start_proxy: open log file with 0o600 perms and close the parent
  fd immediately after Popen — fixes the per-restart fd leak.
- DockerEnvironment: detect collisions between docker_env and the
  egress-controlling env vars (HTTPS_PROXY, SSL_CERT_FILE, etc.).
  When enforce_on_docker=true, fail loud rather than silently
  inverting the isolation; when false, warn and let docker_env win.
- proxy_cli: merge_mappings preserves existing tokens on re-setup;
  --rotate-tokens flag re-mints all of them.  Stops re-running
  `hermes egress setup` from invalidating tokens baked into
  already-running sandboxes.
- proxy_cli: --from-bitwarden fail-loud on disabled BW config,
  missing access token, or empty vault.  Previously fell through to
  the env path while still writing credential_source: bitwarden.
- docker.py: narrow `except Exception` → `except ImportError`;
  iron_proxy._read_tunnel_port_from_config: same.  Bare excepts
  were masking real config-load bugs.
- start_proxy: write pidfile via os.open with O_NOFOLLOW + 0o600
  + st_uid check.  Refuses to follow a pre-existing symlink at the
  pidfile path.
- mint_proxy_token docstring: document the 128-bit suffix entropy
  explicitly (sha256 truncated to 32 hex chars).

P2 — follow-up
- start_proxy: poll-with-timeout (100ms cadence on _port_listening)
  instead of an unconditional 5s sleep.  Saves several seconds per
  Docker container create when enforce_on_docker=true.
- docker.py: apply enforce_on_docker semantics when CA file vanishes
  between status.configured check and CA mount.  Previously returned
  empty args silently.
- docker.py: refuse to mount when mappings.json is empty/corrupt
  (was indistinguishable from upstream outage from inside the
  sandbox).
- install_iron_proxy: tarfile.extract(..., filter='data') to silence
  the PEP 706 deprecation and opt into the 3.14+ default.
- _proxy_state_dir: chmod 0o700 unconditionally; add
  _proxy_state_dir_ro() so read-only callers don't create the dir.
- stop_proxy: re-verify pid before SIGKILL via /proc/<pid>/stat
  starttime AND _pid_alive.  Prevents SIGKILL'ing a recycled pid.
- _pid_alive: tightened cmdline check — basename match on argv[0]
  plus an in-process nonce env var ('iron-proxy' in cmdline matched
  'tail iron-proxy.log' and editors with the log open).
- docker.py: NODE_OPTIONS=--use-openssl-ca so Node.js routes through
  the OpenSSL CA store SSL_CERT_FILE controls, narrowing the
  Python/curl-replace vs Node-add asymmetry waefrebeorn flagged.

P3 — polish
- proxy_cli: dest='egress_command' (was 'proxy_command' which
  collided lexically with the inbound OAuth subparser).
- iron_proxy_version: cache by binary path — get_status is called
  per Docker container create, version is constant per binary.
- Drop unused `import sys` from iron_proxy.
- proxy_cli: `is not None` check on --tunnel-port (was treating 0
  as falsy and silently substituting the default).
- proxy_cli cmd_disable: use get_status().pid instead of reaching
  into ip._read_pid() (stale pidfile from a crashed run would have
  fired a spurious "still running" warning).
- Tests: replace hardcoded /tmp/ca.* paths with tmp_path-derived
  fixtures so tests are hermetic across hosts.

CI
- Windows footguns scanner: os.kill(pid, 0) is now gated behind
  platform.system() != 'Windows' with a windows-footgun: ok marker;
  signal.SIGKILL falls back to SIGTERM on Windows via
  getattr(signal, 'SIGKILL', signal.SIGTERM).
- docs MDX compilation: replace bare `<https://…>` URLs with
  `[text](url)` syntax (MDX-jsx parser rejects the angle-bracket
  form).

Tests
- 32 new tests covering default deny CIDRs, bind policy, audit log
  wiring, subprocess env minimization, CA TOCTOU 0o600, state dir
  0o700, empty-mappings refusal, CA-vanished refusal, docker_env
  collision detection, token preservation/rotate, uncovered provider
  detection, and the proxy_cli command handlers + argparse wiring.
- All 156 tests in test_iron_proxy + test_iron_proxy_cli +
  test_docker_environment + test_config pass locally.

Acknowledged but not addressed in this revision
- E2E test for HTTPS CONNECT + TLS-MITM path: existing E2E exercises
  plain HTTP; full MITM coverage needs separate CI infra (real iron-
  proxy binary + curl with custom CA).  Tracked as follow-up.
- Cosign-style supply-chain verification for the binary checksum:
  upstream iron-proxy doesn't sign releases yet.  Accepted pattern
  (same as Bitwarden integration); tracked as follow-up.
- CA rotation CLI (`hermes egress rotate-ca`): scope-cut to a
  follow-up.

Reviewers: @annguyenNous @waefrebeorn @GodsBoy @erhnysr
2026-05-23 20:38:27 -07:00
Teknium
7a74492134 chore(infographic): add iron-proxy-egress bento-grid bold-graphic 2026-05-23 20:38:27 -07:00
Teknium
69ffb9cfd4 feat(egress): iron-proxy credential-injection firewall for sandboxes
Adds a TLS-intercepting egress proxy for remote terminal sandboxes (Docker
v1; Modal/SSH to follow).  When enabled, the sandbox holds opaque proxy
tokens; iron-proxy swaps them for real provider API keys at the egress
boundary.  Compromising the sandbox leaks tokens that only work from behind
the proxy.

Wraps ironsh/iron-proxy (Apache-2.0, Go binary).  Same lazy-install pattern
as the recently merged Bitwarden Secrets Manager integration — pinned
version, SHA-256 verified download into ~/.hermes/bin/iron-proxy, no apt
or sudo required.

Disabled by default.  Run `hermes egress setup` to mint tokens and
`hermes egress start` to launch.  The Docker backend then automatically
mounts the CA, sets HTTPS_PROXY + CA-bundle env vars, and adds the
host-gateway hostmap.

New surfaces:
  hermes egress install   — download the pinned iron-proxy binary
  hermes egress setup     — interactive wizard (supports --from-bitwarden)
  hermes egress start     — spawn the managed proxy daemon
  hermes egress stop      — SIGTERM (+SIGKILL after 5s grace)
  hermes egress status    — binary + config + pid + listening + mappings
  hermes egress disable   — flip proxy.enabled = false
  hermes egress config    — print the path to the generated proxy.yaml

Optional Bitwarden integration: `--from-bitwarden` sources the real
upstream credentials from a BSM project at proxy startup, so rotating a
key in the Bitwarden web app propagates to sandboxes on the next proxy
start without touching .env.

Hermes-side scope (v1):
  agent/proxy_sources/iron_proxy.py   — install + CA + config + lifecycle
  hermes_cli/proxy_cli.py             — `hermes egress` subcommand tree
  hermes_cli/config.py                — "proxy:" section in DEFAULT_CONFIG
  hermes_cli/main.py                  — argparse wiring (uses 'egress'
                                         because 'proxy' is the existing
                                         inbound OAuth reverse proxy)
  tools/environments/docker.py        — CA mount, HTTPS_PROXY, CA-bundle
                                         env vars, --add-host wiring

Hermetic tests cover the full lifecycle: token mint, mapping discovery,
config + mappings I/O, install pipeline (HTTP + tar + checksum all mocked),
subprocess lifecycle (Popen mocked), Docker backend arg builder.

A live E2E test (gated on HERMES_RUN_E2E=1) downloads the real iron-proxy
binary, spawns it, routes a curl request through it against a local fake
upstream, and verifies the Authorization header was swapped from the proxy
token to the real secret value (and the proxy token did NOT leak through
to upstream).

Failures (binary missing, port collision, bad token) never block agent
startup — they emit a warning and continue.  The Docker backend refuses to
start a sandbox when proxy.enabled=true but the daemon is dead, unless
proxy.enforce_on_docker is explicitly set to false.

Docs: website/docs/user-guide/egress/{index,iron-proxy}.md
Tests: tests/test_iron_proxy.py (35), tests/test_iron_proxy_e2e.py (1)
2026-05-23 20:38:27 -07:00
17 changed files with 6107 additions and 5 deletions

View File

@@ -0,0 +1,8 @@
"""Egress proxy integrations.
Currently ships an iron-proxy (ironsh/iron-proxy) wrapper that intercepts
outbound traffic from remote terminal sandboxes and swaps proxy tokens
for real upstream credentials at the network edge.
Design notes live in :mod:`agent.proxy_sources.iron_proxy`.
"""

File diff suppressed because it is too large Load Diff

View File

@@ -1913,6 +1913,67 @@ DEFAULT_CONFIG = {
"paste_collapse_threshold": 5,
"paste_collapse_threshold_fallback": 0,
# =========================================================================
# Egress credential-injection proxy (iron-proxy)
# =========================================================================
# When enabled, outbound traffic from remote terminal sandboxes (Docker
# today; Modal/SSH in follow-ups) is routed through a managed iron-proxy
# subprocess. The sandbox sees opaque proxy tokens; iron-proxy swaps in
# real API credentials at the egress boundary. Compromising the sandbox
# leaks tokens that only work from behind the proxy.
#
# Configure with `hermes egress setup`. Disabled by default — the rest of
# Hermes works exactly as before with `enabled: false`.
"proxy": {
# Master switch. When false, iron-proxy is never started, no docker
# mounts are added, no binaries are auto-installed — feature is a
# complete no-op.
"enabled": False,
# Tunnel listener port. Sandboxes get `HTTPS_PROXY=http://<host>:<port>`.
# 9090 is the default; collide-aware setup wizard can reassign.
"tunnel_port": 9090,
# Auto-download the pinned iron-proxy binary into ~/.hermes/bin/ on
# first use. When false, you must place `iron-proxy` on PATH yourself.
"auto_install": True,
# Where iron-proxy looks up the real upstream secrets at egress time.
# "env" — process env (default; what bitwarden integration
# already populates if you use it)
# "bitwarden" — refetch via `bws secret list` on each proxy restart;
# rotation in the Bitwarden web app propagates without
# touching .env (requires `secrets.bitwarden.enabled`).
"credential_source": "env",
# When true, the Docker backend refuses to start a sandbox if the
# proxy is enabled but not running. False = fall back to direct
# outbound with real credentials in the sandbox (the legacy posture).
"enforce_on_docker": True,
# When true, `hermes egress start` refuses to start if any provider
# env var is set that the proxy cannot strip (Anthropic native
# `x-api-key`, Azure OpenAI api-key, Gemini x-goog-api-key).
# These LLM-specific credentials would otherwise leak into the
# sandbox bypassing the proxy. Generic cloud creds (AWS_*,
# GOOGLE_APPLICATION_CREDENTIALS) are warned about but never
# block. Defaults to false because false positives (operator has
# the env set but doesn't actually use that provider) are common.
"fail_on_uncovered_providers": False,
# When credential_source is bitwarden but the BWS access token /
# project_id is missing OR the bws fetch returns no values for
# mapped providers, the daemon raises by default. Set this to
# True to opt back in to the legacy "silently fall back to host
# env" behaviour — useful for migrations where the operator wants
# to switch credential_source to bitwarden but hasn't fully wired
# BWS yet. Defaults to false (strict).
"allow_env_fallback": False,
# SSRF deny list applied to outbound traffic. Omit / leave empty
# to use the safe default: loopback, link-local (incl. cloud
# metadata IPs at 169.254.169.254), and RFC1918. Set to an
# explicit ``[]`` to opt out entirely (only sensible in hermetic
# tests that need to reach a loopback upstream).
"upstream_deny_cidrs": None,
# Extra allowed upstream hosts beyond the bundled defaults (which
# cover OpenRouter, OpenAI, Anthropic, Google, xAI, Mistral, Groq,
# Together, DeepSeek, Nous). Wildcards (`*.foo.com`) are supported.
"extra_allowed_hosts": [],
},
# Config schema version - bump this when adding new required fields
"_config_version": 24,

View File

@@ -10759,7 +10759,7 @@ _BUILTIN_SUBCOMMANDS = frozenset(
"acp", "auth", "backup", "bundles", "checkpoints", "claw", "completion",
"computer-use",
"config", "cron", "curator", "dashboard", "debug", "doctor",
"dump", "fallback", "gateway", "hooks", "import", "insights",
"dump", "egress", "fallback", "gateway", "hooks", "import", "insights",
"kanban", "login", "logout", "logs", "lsp", "mcp", "memory", "migrate",
"model", "pairing", "plugins", "portal", "postinstall", "profile", "proxy",
"send", "sessions", "setup",
@@ -11186,6 +11186,37 @@ def main():
secrets_parser.set_defaults(func=_dispatch_secrets)
# =========================================================================
# egress command — iron-proxy outbound credential-injection firewall
# =========================================================================
# NOTE: this is the OUTBOUND egress firewall (ironsh/iron-proxy).
# `hermes proxy` (defined elsewhere in this file) is a separate INBOUND
# OAuth-aggregator reverse proxy. Different direction, different purpose.
egress_parser = subparsers.add_parser(
"egress",
help="Manage the iron-proxy egress credential-injection firewall",
description=(
"Manage iron-proxy, the optional TLS-intercepting egress firewall "
"that swaps proxy tokens for real API credentials before outbound "
"requests leave a sandbox. Disabled by default. See: "
"https://hermes-agent.nousresearch.com/docs/user-guide/egress/iron-proxy"
),
)
from hermes_cli import proxy_cli as _proxy_cli
_proxy_cli.register_cli(egress_parser)
def _dispatch_egress(args): # noqa: ANN001
# The egress subparser uses dest='egress_command' to stay disjoint
# from the inbound OAuth ``hermes proxy`` subparser (dest='proxy_command').
sub = getattr(args, "egress_command", None)
if sub is not None and hasattr(args, "func") and args.func is not _dispatch_egress:
return args.func(args)
egress_parser.print_help()
return 0
egress_parser.set_defaults(func=_dispatch_egress)
# =========================================================================
# migrate command
# =========================================================================
@@ -13970,9 +14001,15 @@ Examples:
cmd_chat(args)
return
# Execute the command
# Execute the command. Propagate the handler's return code as the
# process exit code so subcommands that signal failure (e.g.
# ``hermes egress start`` refusing because of fail_on_uncovered_
# providers) actually exit non-zero. Handlers that return None
# are treated as success (exit 0).
if hasattr(args, "func"):
args.func(args)
rc = args.func(args)
if isinstance(rc, int) and rc != 0:
sys.exit(rc)
else:
parser.print_help()

654
hermes_cli/proxy_cli.py Normal file
View File

@@ -0,0 +1,654 @@
"""CLI handlers for ``hermes egress ...``.
Subcommands:
install — download the pinned iron-proxy binary
setup — interactive wizard: install binary, generate CA, mint tokens, write config
start — launch the proxy as a managed subprocess
stop — terminate the managed proxy
status — show binary version + config presence + listen state + mappings
disable — flip ``proxy.enabled`` to False (does not stop a running proxy)
config — print the generated proxy.yaml path (for debugging / external review)
The top-level command is ``hermes egress``. Note that the inbound OAuth
reverse-proxy command (``hermes proxy``) lives elsewhere in
``hermes_cli/main.py`` — different direction, different purpose.
"""
from __future__ import annotations
import argparse
import os
from pathlib import Path
from typing import List
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
from agent.proxy_sources import iron_proxy as ip
from hermes_cli.config import load_config, save_config
# ---------------------------------------------------------------------------
# Argparse wiring — called from hermes_cli.main
# ---------------------------------------------------------------------------
def register_cli(parent_parser: argparse.ArgumentParser) -> None:
"""Attach the egress subcommand tree to a parent parser.
Called from ``hermes_cli.main`` as part of building the top-level
``hermes egress`` parser.
"""
# dest='egress_command' — keeps this subparser tree disjoint from the
# inbound OAuth ``hermes proxy`` subparser (which uses dest='proxy_command').
# No runtime collision today since they live in separate parser trees,
# but a future grep-and-refactor on ``proxy_command`` would otherwise
# hit both handlers.
sub = parent_parser.add_subparsers(dest="egress_command")
install = sub.add_parser(
"install",
help=f"Download iron-proxy binary (v{ip._IRON_PROXY_VERSION})",
)
install.add_argument(
"--force", action="store_true",
help="Re-download even if a managed copy already exists",
)
install.set_defaults(func=cmd_install)
setup = sub.add_parser(
"setup",
help="Interactive wizard: install + CA + mint tokens + write config",
)
setup.add_argument(
"--tunnel-port", type=int, default=None,
help=f"Override the tunnel port (default {ip._DEFAULT_TUNNEL_PORT})",
)
setup.add_argument(
"--from-bitwarden", action="store_true",
help="Treat secrets as managed by Bitwarden — discover provider keys "
"from secrets.bitwarden config instead of the current env. Fails "
"loudly if BW is unreachable rather than silently falling back.",
)
setup.add_argument(
"--no-bitwarden", action="store_true",
help="Explicitly switch credential_source back to env on re-setup "
"(only meaningful when the previous setup used --from-bitwarden).",
)
setup.add_argument(
"--rotate-tokens", action="store_true",
help="Mint fresh proxy tokens for every provider (default is to "
"preserve tokens for providers that already had one — avoids "
"401-ing already-running sandboxes on re-setup).",
)
setup.set_defaults(func=cmd_setup)
start = sub.add_parser("start", help="Start the managed iron-proxy")
start.set_defaults(func=cmd_start)
stop = sub.add_parser("stop", help="Stop the managed iron-proxy")
stop.set_defaults(func=cmd_stop)
status = sub.add_parser("status", help="Show proxy state and mappings")
status.add_argument(
"--show-tokens", action="store_true",
help="Print the proxy tokens (default: redacted prefix only). "
"Beware: tokens may persist in your shell history.",
)
status.set_defaults(func=cmd_status)
disable = sub.add_parser("disable", help="Turn off the proxy integration")
disable.set_defaults(func=cmd_disable)
cfg = sub.add_parser("config", help="Print the generated proxy.yaml path")
cfg.set_defaults(func=cmd_config)
# ---------------------------------------------------------------------------
# Handlers
# ---------------------------------------------------------------------------
def cmd_install(args: argparse.Namespace) -> int:
console = Console()
try:
binary = ip.install_iron_proxy(force=bool(args.force))
except Exception as exc: # noqa: BLE001 — top-level user-facing error funnel
console.print(f"[red]✗ install failed:[/red] {exc}")
console.print(
" Manual install: https://github.com/ironsh/iron-proxy/releases"
)
return 1
version = ip.iron_proxy_version(binary) or "(version unknown)"
console.print(f"[green]✓[/green] installed {binary} {version}")
return 0
def cmd_setup(args: argparse.Namespace) -> int:
console = Console()
console.print(Panel.fit(
"[bold]iron-proxy setup[/bold]\n\n"
"Routes outbound sandbox traffic through a local TLS-intercepting\n"
"proxy so prompt-injected agents never see real provider API keys.\n\n"
"[dim]Project: https://github.com/ironsh/iron-proxy (Apache-2.0)[/dim]",
border_style="cyan",
))
# ------------------------------------------------------------------ binary
console.print()
console.print("[bold]Step 1[/bold] Install the iron-proxy binary")
try:
binary = ip.find_iron_proxy(install_if_missing=False)
if binary is None:
console.print(" No iron-proxy on PATH — downloading…")
binary = ip.install_iron_proxy()
version = ip.iron_proxy_version(binary) or "(version unknown)"
console.print(f" [green]✓[/green] {binary} {version}")
except Exception as exc: # noqa: BLE001
console.print(f" [red]✗ install failed: {exc}[/red]")
return 1
# ------------------------------------------------------------------ CA
console.print()
console.print("[bold]Step 2[/bold] Generate a CA cert")
try:
ca_crt, ca_key = ip.ensure_ca_cert()
except Exception as exc: # noqa: BLE001
console.print(f" [red]✗ CA generation failed: {exc}[/red]")
return 1
console.print(f" [green]✓[/green] {ca_crt}")
# ------------------------------------------------------------------ mint
console.print()
console.print("[bold]Step 3[/bold] Mint proxy tokens for known providers")
available_env_names: List[str] = []
if args.from_bitwarden:
cfg = load_config()
bw_cfg = (cfg.get("secrets") or {}).get("bitwarden") or {}
if not bw_cfg.get("enabled"):
console.print(
" [red]✗ --from-bitwarden requested but "
"secrets.bitwarden.enabled is false.[/red]"
)
console.print(
" Run `hermes secrets bitwarden setup` first, or omit "
"--from-bitwarden."
)
return 1
try:
from agent.secret_sources import bitwarden as bw
access_token = os.environ.get(
bw_cfg.get("access_token_env", "BWS_ACCESS_TOKEN"), ""
).strip()
if not access_token:
console.print(
f" [red]✗ --from-bitwarden requested but "
f"{bw_cfg.get('access_token_env', 'BWS_ACCESS_TOKEN')} "
"is not set in the environment.[/red]"
)
return 1
secrets, _ = bw.fetch_bitwarden_secrets(
access_token=access_token,
project_id=bw_cfg.get("project_id", ""),
cache_ttl_seconds=0,
use_cache=False,
)
available_env_names = list(secrets.keys())
if not available_env_names:
console.print(
" [red]✗ Bitwarden returned an empty secrets list.[/red]\n"
" Check the project_id in secrets.bitwarden and the "
"BWS access-token's project scope."
)
return 1
console.print(
f" Pulled {len(available_env_names)} env names from Bitwarden."
)
except Exception as exc: # noqa: BLE001 — explicit user-facing error
console.print(
f" [red]✗ Could not enumerate Bitwarden secrets: {exc}[/red]"
)
console.print(
" Either fix the Bitwarden config and retry, or rerun setup "
"without --from-bitwarden (the proxy will read secrets from "
"the host process env at start time)."
)
return 1
discovered = ip.discover_provider_mappings(
available_env_names=available_env_names or None,
)
# Preserve tokens for providers we already had unless the operator
# explicitly requested rotation. This prevents re-running `hermes
# egress setup` from invalidating tokens baked into already-running
# sandboxes.
existing = ip.load_mappings()
rotate = bool(getattr(args, "rotate_tokens", False))
# P3 confirmation gate: --rotate-tokens invalidates every running
# sandbox's proxy tokens immediately. An accidental re-run (history
# scroll-back, tmux paste) is unrecoverable, so require explicit
# confirmation when there's something to actually rotate. Skipped
# when stdin isn't a tty (CI / non-interactive use), in which case
# the operator passed the flag deliberately.
if rotate and existing:
import sys as _sys
from datetime import datetime as _dt
if _sys.stdin.isatty():
console.print(
"[yellow]⚠[/yellow] --rotate-tokens will invalidate proxy "
"tokens in every running Hermes sandbox. They will start "
"401-ing against upstreams until restarted."
)
try:
ans = input("Type 'rotate' to confirm: ").strip().lower()
except EOFError:
ans = ""
if ans != "rotate":
console.print("[yellow]Cancelled.[/yellow]")
return 1
# Backup the existing mappings before we overwrite. The
# resulting ``.rotated-<unix>`` sibling is plain JSON and lets
# the operator manually recover tokens if they realise the
# rotation was a mistake.
try:
import shutil as _shutil
state_dir = ip._proxy_state_dir()
mappings_src = state_dir / "mappings.json"
if mappings_src.exists():
ts = _dt.now().strftime("%Y%m%dT%H%M%S")
backup = state_dir / f"mappings.json.rotated-{ts}"
_shutil.copy2(str(mappings_src), str(backup))
console.print(f" [dim]backup: {backup}[/dim]")
except OSError as exc:
console.print(
f" [yellow]Could not back up mappings before rotation: "
f"{exc}[/yellow]"
)
elif rotate and not existing:
console.print(
"[dim]Note: --rotate-tokens is a no-op on first-time setup "
"(no existing tokens to rotate).[/dim]"
)
mappings = ip.merge_mappings(
existing=existing,
discovered=discovered,
rotate=rotate,
)
if not mappings:
console.print(
" [yellow]No known provider API keys found in env/Bitwarden.[/yellow]"
)
console.print(
" Set at least one of these and rerun setup:"
)
for env_name in sorted(ip._BEARER_PROVIDERS):
console.print(f" - {env_name}")
return 1
# Warn the operator about providers we recognize but can't proxy
# (Anthropic native, AWS Bedrock, Azure OpenAI, etc). These still
# work — they just bypass the egress isolation.
uncovered = ip.discover_uncovered_providers(
available_env_names=available_env_names or None,
)
if uncovered:
console.print()
console.print(
" [yellow]⚠[/yellow] Detected provider env vars that the "
"proxy does not yet cover:"
)
for name in uncovered:
console.print(f" - {name}")
console.print(
" [dim]These providers use non-bearer auth (x-api-key, "
"SigV4, etc.) and will hold real credentials inside the "
"sandbox. Egress isolation is INCOMPLETE for these.[/dim]"
)
table = Table(show_header=True, header_style="bold")
table.add_column("Provider env", style="cyan")
table.add_column("Upstream hosts", style="dim")
table.add_column("Proxy token", style="green")
for m in mappings:
table.add_row(
m.real_env_name,
", ".join(m.upstream_hosts),
_redact_token(m.proxy_token),
)
console.print(table)
# ------------------------------------------------------------------ write
console.print()
console.print("[bold]Step 4[/bold] Write config and persist mappings")
cfg = load_config()
proxy_cfg = cfg.setdefault("proxy", {})
# ``args.tunnel_port`` is None when the flag was not given; ``0`` is
# invalid for a TCP listener so we treat it as an explicit refusal
# and surface a clear error rather than silently substituting the
# default.
if args.tunnel_port is not None:
if args.tunnel_port == 0:
console.print(
" [red]✗ --tunnel-port=0 is not a valid TCP port.[/red]"
)
return 1
tunnel_port = int(args.tunnel_port)
else:
tunnel_port = int(proxy_cfg.get("tunnel_port", ip._DEFAULT_TUNNEL_PORT))
proxy_cfg["tunnel_port"] = tunnel_port
extra_hosts = list(proxy_cfg.get("extra_allowed_hosts") or [])
allowed = list(ip._DEFAULT_ALLOWED_HOSTS) + [
h for h in extra_hosts if h not in ip._DEFAULT_ALLOWED_HOSTS
]
audit_log_path = ip._proxy_state_dir() / "audit.log"
# Pre-create the audit log with 0o600 so iron-proxy inherits private
# perms instead of letting the daemon create it under the default
# umask (potentially world-readable). Raises on failure (planted
# symlink, immutable parent, full disk) — the wizard must surface
# that rather than print "✓" for a file the daemon will create
# under a slacker umask.
try:
ip.ensure_audit_log(audit_log_path)
except RuntimeError as exc:
console.print(f" [red]✗ {exc}[/red]")
return 1
# Allow operator override of the deny list via
# ``proxy.upstream_deny_cidrs`` — but the default (None) gives a safe
# default-deny list (loopback, IMDS, RFC1918) that matches the docs
# promise.
deny_cidrs = proxy_cfg.get("upstream_deny_cidrs")
iron_cfg = ip.build_proxy_config(
mappings=mappings,
ca_cert=ca_crt,
ca_key=ca_key,
tunnel_port=tunnel_port,
audit_log=audit_log_path,
allowed_hosts=allowed,
upstream_deny_cidrs=deny_cidrs,
)
cfg_path = ip.write_proxy_config(iron_cfg)
mappings_path = ip.write_mappings(mappings)
console.print(f" [green]✓[/green] config: {cfg_path}")
console.print(f" [green]✓[/green] mappings: {mappings_path}")
console.print(f" [green]✓[/green] audit log: {audit_log_path}")
# ------------------------------------------------------------------ enable
proxy_cfg["enabled"] = True
proxy_cfg.setdefault("auto_install", True)
proxy_cfg.setdefault("enforce_on_docker", True)
# CRITICAL: do NOT silently downgrade credential_source on re-run.
# If the operator previously configured `bitwarden` mode (e.g. for
# rotation), running `hermes egress setup` again WITHOUT
# --from-bitwarden must not rewrite credential_source to "env" —
# that silently breaks the Bitwarden rotation guarantee the docs
# make. Require an explicit --no-bitwarden to switch back.
existing_source = proxy_cfg.get("credential_source")
if args.from_bitwarden:
proxy_cfg["credential_source"] = "bitwarden"
elif getattr(args, "no_bitwarden", False):
proxy_cfg["credential_source"] = "env"
if existing_source == "bitwarden":
console.print(
"[yellow]Switched credential_source from bitwarden to env.[/yellow]"
)
elif existing_source == "bitwarden":
# Preserve the existing bitwarden mode. Surface the decision so
# the operator knows we kept it.
console.print(
"[dim]Keeping credential_source=bitwarden from existing config. "
"Pass --no-bitwarden to switch to env-based credentials.[/dim]"
)
else:
proxy_cfg["credential_source"] = "env"
proxy_cfg.setdefault("fail_on_uncovered_providers", False)
save_config(cfg)
console.print()
console.print(
"[green]✓ iron-proxy is configured.[/green] "
"Sandboxes will route outbound traffic through it."
)
console.print(
" Start: [cyan]hermes egress start[/cyan]\n"
" Status: [cyan]hermes egress status[/cyan]\n"
" Stop: [cyan]hermes egress stop[/cyan]\n"
" Disable: [cyan]hermes egress disable[/cyan]"
)
return 0
def cmd_start(args: argparse.Namespace) -> int:
console = Console()
cfg = load_config()
proxy_cfg = cfg.get("proxy") or {}
if not proxy_cfg.get("enabled"):
console.print(
"[yellow]proxy.enabled is false — run `hermes egress setup` "
"first.[/yellow]"
)
return 1
# If the operator opted in to Bitwarden-rotation semantics, refresh
# upstream secrets from BSM at startup. This is what delivers the
# rotation guarantee that distinguishes ``credential_source:
# bitwarden`` from ``credential_source: env``. Without it, rotating
# a key in the Bitwarden web app doesn't reach the proxy.
credential_source = proxy_cfg.get("credential_source", "env")
bw_cfg = (cfg.get("secrets") or {}).get("bitwarden")
refresh_bw = (
credential_source == "bitwarden"
and bw_cfg is not None
and bool(bw_cfg.get("enabled"))
)
# Pass the proxy-side allow_env_fallback opt-in through to
# start_proxy. This is a deliberate, documented escape hatch: when
# set, the daemon silently falls back to host env if BWS is
# unreachable, instead of raising. Default is strict (raise).
if refresh_bw and bw_cfg is not None:
bw_cfg = dict(bw_cfg)
bw_cfg["allow_env_fallback"] = bool(
proxy_cfg.get("allow_env_fallback", False)
)
# fail_on_uncovered_providers: when true, refuse to start if any
# LLM-specific non-bearer providers (Anthropic native, Azure OpenAI,
# Gemini) have env vars set in the host process — those would
# otherwise leak real credentials into the sandbox while bypassing
# the proxy. Only the strict LLM-specific subset blocks; generic
# cloud creds (AWS_*, GOOGLE_APPLICATION_CREDENTIALS) still surface
# as warnings via `discover_uncovered_providers` but don't block, to
# avoid tripping every operator with terraform / gcloud set up.
if bool(proxy_cfg.get("fail_on_uncovered_providers", False)):
blocked = ip.discover_blocked_providers()
if blocked:
console.print(
"[red]✗ Refusing to start: provider env vars present "
"that bypass the proxy:[/red]"
)
for name in blocked:
console.print(f" - {name}")
console.print(
" Set `proxy.fail_on_uncovered_providers: false` in "
"config.yaml to start anyway (sandbox will hold real "
"credentials for those providers)."
)
return 1
# stephenschoettler #1: when `credential_source: bitwarden`, the
# operator picked BWS specifically to get the rotation guarantee —
# silently falling back to parent-env at start_proxy time reintroduces
# exactly the bug class the BW mode is supposed to defeat (host env
# is stale / mismatched). Pre-check at the wizard layer so we fail
# loud with actionable error messages BEFORE start_proxy degrades.
if refresh_bw:
bw_access_env = (bw_cfg or {}).get("access_token_env", "BWS_ACCESS_TOKEN")
if not os.environ.get(bw_access_env, "").strip():
console.print(
f"[red]✗ Refusing to start: credential_source=bitwarden but "
f"{bw_access_env} is not set in the environment.[/red]"
)
console.print(
" Either export the access token, or run "
"`hermes egress setup --no-bitwarden` to switch back to "
"env-based credentials."
)
return 1
if not (bw_cfg or {}).get("project_id"):
console.print(
"[red]✗ Refusing to start: credential_source=bitwarden but "
"secrets.bitwarden.project_id is empty.[/red]"
)
console.print(
" Run `hermes secrets bitwarden setup` to configure the "
"project, or switch back via `hermes egress setup "
"--no-bitwarden`."
)
return 1
try:
status = ip.start_proxy(
refresh_secrets_from_bitwarden=refresh_bw,
bitwarden_config=bw_cfg,
)
except Exception as exc: # noqa: BLE001 — top-level user-facing funnel
console.print(f"[red]✗ failed to start iron-proxy:[/red] {exc}")
return 1
if status.pid:
listening = (
"[green]listening[/green]"
if status.listening
else "[yellow]not yet listening[/yellow]"
)
console.print(
f"[green]✓[/green] iron-proxy running pid={status.pid} "
f"port={status.tunnel_port} {listening}"
)
else:
console.print("[red]✗ iron-proxy did not come up cleanly[/red]")
return 1
return 0
def cmd_stop(args: argparse.Namespace) -> int:
console = Console()
if ip.stop_proxy():
console.print("[green]✓[/green] iron-proxy stopped")
else:
console.print("[dim]iron-proxy was not running[/dim]")
return 0
def cmd_status(args: argparse.Namespace) -> int:
console = Console()
cfg = load_config()
proxy_cfg = cfg.get("proxy") or {}
status = ip.get_status()
table = Table(show_header=False, box=None, padding=(0, 2))
table.add_column("", style="bold")
table.add_column("")
table.add_row("Enabled", _yn(bool(proxy_cfg.get("enabled"))))
table.add_row("Binary", str(status.binary_path or "[dim](missing)[/dim]"))
table.add_row("Binary version", status.binary_version or "[dim](unknown)[/dim]")
table.add_row("Config", str(status.config_path or "[dim](not generated)[/dim]"))
table.add_row("CA cert", str(status.ca_cert_path or "[dim](not generated)[/dim]"))
table.add_row("Tunnel port", str(status.tunnel_port))
table.add_row("Process", f"pid {status.pid}" if status.pid else "[dim](stopped)[/dim]")
table.add_row("Listening", _yn(status.listening))
table.add_row("Credential src", str(proxy_cfg.get("credential_source", "env")))
table.add_row("Docker enforce", _yn(bool(proxy_cfg.get("enforce_on_docker", True))))
console.print(table)
mappings = ip.load_mappings()
if mappings:
console.print()
console.print("[bold]Token mappings[/bold]")
m_table = Table(show_header=True, header_style="bold")
m_table.add_column("Real env", style="cyan")
m_table.add_column("Upstream", style="dim")
m_table.add_column("Proxy token", style="green")
for m in mappings:
tok = m.proxy_token if args.show_tokens else _redact_token(m.proxy_token)
m_table.add_row(m.real_env_name, ", ".join(m.upstream_hosts), tok)
console.print(m_table)
if args.show_tokens:
console.print(
"[yellow]⚠[/yellow] proxy tokens just printed in full — "
"they may persist in your shell history. Consider clearing "
"it after this command."
)
# Surface uncovered providers so the operator knows the isolation
# boundary is incomplete for those upstreams.
uncovered = ip.discover_uncovered_providers()
if uncovered:
console.print()
console.print(
"[yellow]Uncovered providers[/yellow] "
"(real credentials still visible inside the sandbox):"
)
for name in uncovered:
console.print(f" - {name}")
return 0
def cmd_disable(args: argparse.Namespace) -> int:
console = Console()
cfg = load_config()
proxy_cfg = cfg.setdefault("proxy", {})
if not proxy_cfg.get("enabled"):
console.print("[dim]proxy.enabled was already false.[/dim]")
return 0
proxy_cfg["enabled"] = False
save_config(cfg)
console.print("[green]✓[/green] proxy.enabled set to false")
# Use the public get_status() pid (which already incorporates the
# _pid_alive check) instead of reaching into ip._read_pid(). That
# private accessor only proves the pidfile is non-empty — a stale
# pidfile from a crashed previous run would fire the warning
# spuriously.
if ip.get_status().pid is not None:
console.print(
" iron-proxy is still running — stop it with "
"[cyan]hermes egress stop[/cyan] if you want it down too."
)
return 0
def cmd_config(args: argparse.Namespace) -> int:
console = Console()
status = ip.get_status()
if status.config_path is None:
console.print(
"[yellow](no config generated — run `hermes egress setup`)[/yellow]"
)
return 1
console.print(str(status.config_path))
return 0
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _yn(value: bool) -> str:
return "[green]yes[/green]" if value else "[dim]no[/dim]"
def _redact_token(token: str) -> str:
if len(token) < 16:
return token
return f"{token[:12]}{token[-4:]}"

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 MiB

1457
tests/test_iron_proxy.py Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,427 @@
"""Unit tests for ``hermes_cli.proxy_cli`` command handlers.
These tests cover the user-facing CLI surface that was previously
uncovered. We mock the iron_proxy module's side-effect functions
(install / start / stop / discover) and exercise the dispatch +
return-code logic plus the small amount of presentation logic in
each handler (e.g. --from-bitwarden's fail-loud path).
"""
from __future__ import annotations
import argparse
import os
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from agent.proxy_sources import iron_proxy as ip
from hermes_cli import proxy_cli
@pytest.fixture
def hermes_home(tmp_path, monkeypatch):
"""Point HERMES_HOME at a temp dir so the wizard doesn't touch the
operator's real config. Also blanks any provider env vars so we
don't accidentally read a real key."""
home = tmp_path / "hermes"
home.mkdir()
monkeypatch.setenv("HERMES_HOME", str(home))
for key in list(os.environ):
if key.endswith("_API_KEY") or key in (
"BWS_ACCESS_TOKEN", "ANTHROPIC_API_KEY",
"AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY",
):
monkeypatch.delenv(key, raising=False)
return home
def _args(**overrides):
ns = argparse.Namespace(
force=False,
tunnel_port=None,
from_bitwarden=False,
rotate_tokens=False,
show_tokens=False,
)
for k, v in overrides.items():
setattr(ns, k, v)
return ns
# ---------------------------------------------------------------------------
# cmd_install
# ---------------------------------------------------------------------------
def test_cmd_install_success_returns_0(hermes_home, monkeypatch):
monkeypatch.setattr(ip, "install_iron_proxy", lambda **kw: hermes_home / "iron-proxy")
monkeypatch.setattr(ip, "iron_proxy_version", lambda b: "v0.39.0-test")
rc = proxy_cli.cmd_install(_args())
assert rc == 0
def test_cmd_install_failure_returns_1(hermes_home, monkeypatch):
def boom(**kw):
raise RuntimeError("download failed")
monkeypatch.setattr(ip, "install_iron_proxy", boom)
rc = proxy_cli.cmd_install(_args())
assert rc == 1
# ---------------------------------------------------------------------------
# cmd_setup — --from-bitwarden fail-loud paths
# ---------------------------------------------------------------------------
def test_cmd_setup_from_bitwarden_refuses_when_bw_disabled(hermes_home, monkeypatch):
"""When --from-bitwarden is passed but secrets.bitwarden.enabled=false,
the wizard must FAIL rather than silently rewriting credential_source
to bitwarden."""
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("secrets", {})["bitwarden"] = {"enabled": False}
save_config(cfg)
# Pre-stub install + CA so we get to step 3.
monkeypatch.setattr(ip, "find_iron_proxy", lambda **kw: hermes_home / "iron-proxy")
monkeypatch.setattr(ip, "iron_proxy_version", lambda b: "test")
monkeypatch.setattr(
ip, "ensure_ca_cert",
lambda **kw: (hermes_home / "ca.crt", hermes_home / "ca.key"),
)
rc = proxy_cli.cmd_setup(_args(from_bitwarden=True))
assert rc == 1
# Verify we did NOT write credential_source: bitwarden to config.
cfg2 = load_config()
proxy_cfg = cfg2.get("proxy") or {}
assert proxy_cfg.get("credential_source", "env") != "bitwarden"
def test_cmd_setup_from_bitwarden_refuses_when_token_missing(hermes_home, monkeypatch):
"""--from-bitwarden with secrets.bitwarden.enabled=true but BWS access
token unset → fail loud, not silent env-fallback."""
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("secrets", {})["bitwarden"] = {
"enabled": True,
"project_id": "test-proj",
"access_token_env": "BWS_ACCESS_TOKEN",
}
save_config(cfg)
monkeypatch.delenv("BWS_ACCESS_TOKEN", raising=False)
monkeypatch.setattr(ip, "find_iron_proxy", lambda **kw: hermes_home / "iron-proxy")
monkeypatch.setattr(ip, "iron_proxy_version", lambda b: "test")
monkeypatch.setattr(
ip, "ensure_ca_cert",
lambda **kw: (hermes_home / "ca.crt", hermes_home / "ca.key"),
)
rc = proxy_cli.cmd_setup(_args(from_bitwarden=True))
assert rc == 1
def test_cmd_setup_from_bitwarden_refuses_on_empty_vault(hermes_home, monkeypatch):
"""If BW returns {} (empty vault / scoped wrong / unreachable), fail
loud rather than silently writing credential_source: bitwarden."""
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("secrets", {})["bitwarden"] = {
"enabled": True,
"project_id": "test-proj",
"access_token_env": "BWS_ACCESS_TOKEN",
}
save_config(cfg)
monkeypatch.setenv("BWS_ACCESS_TOKEN", "bwsk-test-token")
monkeypatch.setattr(ip, "find_iron_proxy", lambda **kw: hermes_home / "iron-proxy")
monkeypatch.setattr(ip, "iron_proxy_version", lambda b: "test")
monkeypatch.setattr(
ip, "ensure_ca_cert",
lambda **kw: (hermes_home / "ca.crt", hermes_home / "ca.key"),
)
# Mock fetch_bitwarden_secrets to return an empty dict (empty vault).
fake_bw = MagicMock()
fake_bw.fetch_bitwarden_secrets = lambda **kw: ({}, [])
monkeypatch.setattr("agent.secret_sources.bitwarden", fake_bw, raising=False)
import sys
sys.modules["agent.secret_sources.bitwarden"] = fake_bw
rc = proxy_cli.cmd_setup(_args(from_bitwarden=True))
assert rc == 1
def test_cmd_setup_rejects_tunnel_port_zero(hermes_home, monkeypatch):
"""--tunnel-port=0 is rejected explicitly (was silently substituting
the default before the fix)."""
monkeypatch.setenv("OPENROUTER_API_KEY", "sk-or-test")
monkeypatch.setattr(ip, "find_iron_proxy", lambda **kw: hermes_home / "iron-proxy")
monkeypatch.setattr(ip, "iron_proxy_version", lambda b: "test")
monkeypatch.setattr(
ip, "ensure_ca_cert",
lambda **kw: (hermes_home / "ca.crt", hermes_home / "ca.key"),
)
rc = proxy_cli.cmd_setup(_args(tunnel_port=0))
assert rc == 1
# ---------------------------------------------------------------------------
# cmd_start — fail_on_uncovered_providers + Bitwarden rotation wire-up
# ---------------------------------------------------------------------------
def test_cmd_start_refuses_when_proxy_disabled(hermes_home, monkeypatch):
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("proxy", {})["enabled"] = False
save_config(cfg)
rc = proxy_cli.cmd_start(_args())
assert rc == 1
def test_cmd_start_refuses_on_uncovered_provider_when_strict(hermes_home, monkeypatch):
"""fail_on_uncovered_providers=true + ANTHROPIC_API_KEY in env =
refuse to start (real credential would otherwise leak into sandbox)."""
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("proxy", {})["enabled"] = True
cfg["proxy"]["fail_on_uncovered_providers"] = True
save_config(cfg)
monkeypatch.setenv("ANTHROPIC_API_KEY", "sk-ant-test")
rc = proxy_cli.cmd_start(_args())
assert rc == 1
def test_cmd_start_passes_bitwarden_refresh_flag_when_credential_source_is_bitwarden(
hermes_home, monkeypatch,
):
"""When credential_source=bitwarden, cmd_start must wire
refresh_secrets_from_bitwarden=True into start_proxy. That's what
delivers the rotation promise the docs make."""
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("proxy", {})["enabled"] = True
cfg["proxy"]["credential_source"] = "bitwarden"
cfg["proxy"]["fail_on_uncovered_providers"] = False
cfg.setdefault("secrets", {})["bitwarden"] = {
"enabled": True,
"project_id": "test-proj-id",
"access_token_env": "BWS_ACCESS_TOKEN",
}
save_config(cfg)
# v3: cmd_start now pre-checks BWS access token + project_id before
# calling start_proxy. Provide both so we get to the rotation
# wire-up code path.
monkeypatch.setenv("BWS_ACCESS_TOKEN", "bwsk-test-access-token")
captured: dict = {}
def fake_start_proxy(**kw):
captured.update(kw)
s = ip.ProxyStatus()
s.pid = 4242
s.listening = True
s.tunnel_port = 9090
return s
monkeypatch.setattr(ip, "start_proxy", fake_start_proxy)
monkeypatch.setattr(ip, "discover_uncovered_providers", lambda **kw: [])
monkeypatch.setattr(ip, "discover_blocked_providers", lambda **kw: [])
rc = proxy_cli.cmd_start(_args())
assert rc == 0
assert captured.get("refresh_secrets_from_bitwarden") is True
assert captured.get("bitwarden_config") is not None
def test_cmd_start_refuses_when_bitwarden_token_missing(hermes_home, monkeypatch):
"""stephenschoettler #1: when credential_source=bitwarden but the
access-token env var is empty, cmd_start must fail-loud BEFORE
start_proxy can silently fall back to parent env."""
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("proxy", {})["enabled"] = True
cfg["proxy"]["credential_source"] = "bitwarden"
cfg["proxy"]["fail_on_uncovered_providers"] = False
cfg.setdefault("secrets", {})["bitwarden"] = {
"enabled": True,
"project_id": "test-proj-id",
"access_token_env": "BWS_ACCESS_TOKEN",
}
save_config(cfg)
monkeypatch.delenv("BWS_ACCESS_TOKEN", raising=False)
# Sentinel: start_proxy must NOT be called.
def must_not_call(**kw):
pytest.fail("start_proxy should not be invoked when BWS token missing")
monkeypatch.setattr(ip, "start_proxy", must_not_call)
monkeypatch.setattr(ip, "discover_uncovered_providers", lambda **kw: [])
monkeypatch.setattr(ip, "discover_blocked_providers", lambda **kw: [])
rc = proxy_cli.cmd_start(_args())
assert rc == 1
def test_cmd_start_does_not_pass_bitwarden_refresh_when_credential_source_is_env(
hermes_home, monkeypatch,
):
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("proxy", {})["enabled"] = True
cfg["proxy"]["credential_source"] = "env"
cfg["proxy"]["fail_on_uncovered_providers"] = False
save_config(cfg)
captured: dict = {}
def fake_start_proxy(**kw):
captured.update(kw)
s = ip.ProxyStatus()
s.pid = 4242
s.listening = True
return s
monkeypatch.setattr(ip, "start_proxy", fake_start_proxy)
monkeypatch.setattr(ip, "discover_uncovered_providers", lambda **kw: [])
rc = proxy_cli.cmd_start(_args())
assert rc == 0
assert captured.get("refresh_secrets_from_bitwarden") is False
# ---------------------------------------------------------------------------
# cmd_stop, cmd_status, cmd_disable, cmd_config
# ---------------------------------------------------------------------------
def test_cmd_stop_returns_0_when_running(hermes_home, monkeypatch):
monkeypatch.setattr(ip, "stop_proxy", lambda: True)
rc = proxy_cli.cmd_stop(_args())
assert rc == 0
def test_cmd_stop_returns_0_when_already_stopped(hermes_home, monkeypatch):
monkeypatch.setattr(ip, "stop_proxy", lambda: False)
rc = proxy_cli.cmd_stop(_args())
assert rc == 0
def test_cmd_status_returns_0(hermes_home, monkeypatch):
monkeypatch.setattr(ip, "get_status", lambda: ip.ProxyStatus())
monkeypatch.setattr(ip, "load_mappings", lambda: [])
monkeypatch.setattr(ip, "discover_uncovered_providers", lambda **kw: [])
rc = proxy_cli.cmd_status(_args())
assert rc == 0
def test_cmd_disable_uses_public_status_pid_not_private_read_pid(
hermes_home, monkeypatch,
):
"""cmd_disable must read status.pid (which incorporates the _pid_alive
check) — NOT ip._read_pid() directly (which would fire a spurious
'still running' warning for a stale pidfile from a crashed run)."""
from hermes_cli.config import load_config, save_config
cfg = load_config()
cfg.setdefault("proxy", {})["enabled"] = True
save_config(cfg)
# Pidfile exists but the process is dead. Old code would have warned
# "still running"; the new code reads status.pid which returns None
# because _pid_alive is False, so no spurious warning.
state = ip._proxy_state_dir()
(state / "iron-proxy.pid").write_text("99999")
# _pid_alive returns False → status.pid is None.
monkeypatch.setattr(ip, "_pid_alive", lambda pid: False)
# If cmd_disable reads _read_pid() directly (old path), this test
# would still pass — but reading status.pid is the correct
# API. Sentinel: confirm _read_pid is NOT called from cmd_disable.
read_pid_calls = []
real_read_pid = ip._read_pid
def tracked_read_pid(*a, **kw):
read_pid_calls.append((a, kw))
return real_read_pid(*a, **kw)
monkeypatch.setattr(ip, "_read_pid", tracked_read_pid)
rc = proxy_cli.cmd_disable(_args())
assert rc == 0
# cmd_disable should call get_status() (which may internally call
# _read_pid), but should NOT call _read_pid from its own body.
# Hard to assert directly without source-introspection — the meatier
# assertion is that no "still running" message fired with a stale
# pidfile. That's covered by inspecting return code + config
# mutation only.
from hermes_cli.config import load_config as _lc
cfg2 = _lc()
assert cfg2["proxy"]["enabled"] is False
def test_cmd_config_returns_0_when_present(hermes_home, monkeypatch):
fake = ip.ProxyStatus()
fake.config_path = hermes_home / "proxy.yaml"
monkeypatch.setattr(ip, "get_status", lambda: fake)
rc = proxy_cli.cmd_config(_args())
assert rc == 0
def test_cmd_config_returns_1_when_missing(hermes_home, monkeypatch):
monkeypatch.setattr(ip, "get_status", lambda: ip.ProxyStatus())
rc = proxy_cli.cmd_config(_args())
assert rc == 1
# ---------------------------------------------------------------------------
# Argparse wiring — dest='egress_command' regression
# ---------------------------------------------------------------------------
def test_register_cli_uses_egress_command_dest():
"""The subparser dest must be 'egress_command' to stay disjoint from
the inbound OAuth 'hermes proxy' subparser (dest='proxy_command').
A future grep-and-refactor on proxy_command should not hit this
subparser by accident."""
parser = argparse.ArgumentParser(prog="hermes egress")
proxy_cli.register_cli(parser)
# Parse a no-op invocation and confirm the attribute name.
args = parser.parse_args(["install"])
assert hasattr(args, "egress_command")
assert not hasattr(args, "proxy_command")
def test_egress_subcommands_registered():
"""Smoke test: every documented subcommand parses without error."""
parser = argparse.ArgumentParser(prog="hermes egress")
proxy_cli.register_cli(parser)
for sub in ("install", "setup", "start", "stop", "status", "disable", "config"):
args = parser.parse_args([sub])
assert args.egress_command == sub
def test_setup_has_rotate_tokens_flag():
"""--rotate-tokens is the documented escape hatch for re-rolling
every proxy token (used after a suspected token leak). Default is
preserve-existing."""
parser = argparse.ArgumentParser(prog="hermes egress")
proxy_cli.register_cli(parser)
args = parser.parse_args(["setup"])
assert args.rotate_tokens is False
args = parser.parse_args(["setup", "--rotate-tokens"])
assert args.rotate_tokens is True

View File

@@ -0,0 +1,165 @@
"""End-to-end smoke test for the iron-proxy egress integration.
Spins up the REAL iron-proxy binary (auto-installed if not present), routes
a curl request through it against a local fake upstream, and verifies that
the Authorization header was swapped from a proxy token to a real secret.
Gated on the network. Skipped by default in CI unless the user explicitly
opts in with --run-e2e or HERMES_RUN_E2E=1. This is intentional — the test
downloads ~16MB and requires both `openssl` and `curl` to be present.
"""
from __future__ import annotations
import os
import socket
import subprocess
import threading
import time
from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path
from typing import Optional
import pytest
from agent.proxy_sources import iron_proxy as ip
pytestmark = pytest.mark.skipif(
os.environ.get("HERMES_RUN_E2E", "0") != "1",
reason="E2E proxy test — set HERMES_RUN_E2E=1 to run (requires network + curl + openssl)",
)
@pytest.fixture
def hermes_home(tmp_path, monkeypatch):
home = tmp_path / "hermes"
home.mkdir()
monkeypatch.setenv("HERMES_HOME", str(home))
return home
def _free_port() -> int:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("127.0.0.1", 0))
return s.getsockname()[1]
class _CaptureHandler(BaseHTTPRequestHandler):
"""Records the Authorization header of every incoming request."""
captured_auth: Optional[str] = None # class-level so tests can read it
def do_GET(self):
type(self).captured_auth = self.headers.get("Authorization")
body = b'{"ok": true}'
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
def log_message(self, *args, **kwargs):
return # silence access log
def test_iron_proxy_swaps_authorization_header_end_to_end(hermes_home, monkeypatch):
"""Real binary, real CA, real curl. Verify the proxy swaps a proxy-token
Authorization header for the real bearer value before forwarding."""
if not __import__("shutil").which("curl"):
pytest.skip("curl not available")
if not __import__("shutil").which("openssl"):
pytest.skip("openssl not available")
# ----- fake upstream ----------------------------------------------------
upstream_port = _free_port()
server = HTTPServer(("127.0.0.1", upstream_port), _CaptureHandler)
server_thread = threading.Thread(target=server.serve_forever, daemon=True)
server_thread.start()
try:
# ----- iron-proxy install + CA + config ---------------------------
binary = ip.install_iron_proxy()
assert binary.exists()
ca_crt, ca_key = ip.ensure_ca_cert()
assert ca_crt.exists()
real_secret = "sk-real-upstream-value-deadbeef"
monkeypatch.setenv("TEST_UPSTREAM_KEY", real_secret)
proxy_token = ip.mint_proxy_token("test")
mapping = ip.TokenMapping(
proxy_token=proxy_token,
real_env_name="TEST_UPSTREAM_KEY",
upstream_hosts=("127.0.0.1",),
)
tunnel_port = _free_port()
cfg = ip.build_proxy_config(
mappings=[mapping],
ca_cert=ca_crt,
ca_key=ca_key,
tunnel_port=tunnel_port,
allowed_hosts=["127.0.0.1"],
# Test target is on loopback — clear the default IMDS+loopback
# deny list so iron-proxy will dial 127.0.0.1.
upstream_deny_cidrs=[],
)
ip.write_proxy_config(cfg)
ip.write_mappings([mapping])
# ----- start the proxy --------------------------------------------
try:
status = ip.start_proxy()
except RuntimeError as exc:
pytest.skip(f"iron-proxy could not start in this environment: {exc}")
assert status.pid is not None
# Wait up to 10s for the listener to come up.
for _ in range(50):
if ip._port_listening("127.0.0.1", tunnel_port):
break
time.sleep(0.2)
else:
pytest.fail("iron-proxy never started listening on the tunnel port")
# ----- request through the proxy ----------------------------------
# The fake upstream listens on plain HTTP (not HTTPS), so we use the
# proxy's tunnel for the CONNECT but talk plaintext to upstream via
# `--proxy-insecure` semantics: iron-proxy accepts HTTPS_PROXY-style
# CONNECT to any host on its allowlist. For a clean E2E we hit
# http://127.0.0.1:<port>/ which goes through the proxy as a plain
# HTTP forward (no MITM needed) and the secrets transform still fires
# on the Authorization header.
result = subprocess.run(
[
"curl",
"--silent",
"--max-time", "10",
"-x", f"http://127.0.0.1:{tunnel_port}",
"-H", f"Authorization: Bearer {proxy_token}",
f"http://127.0.0.1:{upstream_port}/",
],
capture_output=True,
text=True,
)
assert result.returncode == 0, f"curl failed: {result.stderr}"
# Some iron-proxy versions return 200 with no body; only the swap matters.
captured = _CaptureHandler.captured_auth
assert captured is not None, "upstream never received the request"
assert real_secret in captured, (
f"Authorization header was not swapped — upstream saw: {captured!r}"
)
assert proxy_token not in captured, (
f"Proxy token leaked through to upstream: {captured!r}"
)
finally:
# ----- cleanup ------------------------------------------------------
try:
ip.stop_proxy()
except Exception:
pass
server.shutdown()
server.server_close()

View File

@@ -180,6 +180,158 @@ _PRIVDROP_CAP_ARGS = [
]
def _egress_proxy_args_for_docker() -> tuple[list[str], dict[str, str], list[str]]:
"""Build the docker mount/env/host args needed to route a sandbox through
the iron-proxy egress firewall.
Returns ``(volume_args, env_overrides, host_args)``:
* ``volume_args`` — read-only bind mount of the CA cert into the container
(extends docker's ``-v`` argv list)
* ``env_overrides`` — env vars to set on container creation: ``HTTPS_PROXY``,
``HTTP_PROXY``, ``NO_PROXY`` (loopback only), Python/Node/curl CA-bundle
paths, and one ``HERMES_PROXY_TOKEN_<NAME>`` per minted mapping
* ``host_args`` — extra ``--add-host`` flags so the container can reach the
host-side proxy (Linux needs ``host.docker.internal:host-gateway``;
Docker Desktop populates this automatically on macOS/Windows)
Returns three empty containers when the proxy is disabled, not yet set up,
or not currently running. If ``proxy.enforce_on_docker`` is true and the
proxy is enabled-but-not-running, raises ``RuntimeError`` so the docker
backend refuses to start the sandbox.
"""
# Narrow except: ImportError is the only legitimate failure here.
# Bare ``except Exception`` would hide AttributeError, SyntaxError in
# the config module, etc. and silently start the sandbox without
# proxy enforcement. We let unexpected exceptions propagate so the
# docker backend visibly fails rather than degrading silently.
try:
from hermes_cli.config import load_config
from agent.proxy_sources import iron_proxy as ip
except ImportError as exc:
logger.debug("Egress proxy plumbing unavailable: %s", exc)
return ([], {}, [])
cfg = load_config()
proxy_cfg = cfg.get("proxy") or {}
if not proxy_cfg.get("enabled"):
return ([], {}, [])
status = ip.get_status()
enforce = bool(proxy_cfg.get("enforce_on_docker", True))
if not status.configured:
msg = (
"proxy.enabled is true but iron-proxy is not configured. "
"Run `hermes egress setup` to mint tokens and write proxy.yaml."
)
if enforce:
raise RuntimeError(msg)
logger.warning("%s — continuing without proxy (enforce_on_docker=false).", msg)
return ([], {}, [])
if not (status.pid and status.listening):
msg = (
f"iron-proxy is enabled but not running on port {status.tunnel_port}. "
"Start it with `hermes egress start`."
)
if enforce:
raise RuntimeError(msg)
logger.warning("%s — continuing without proxy (enforce_on_docker=false).", msg)
return ([], {}, [])
if status.ca_cert_path is None or not status.ca_cert_path.exists():
# status.configured was True a moment ago but the CA file has
# disappeared. Treat this with the same enforce semantics as the
# other failure branches — silently dropping the CA mount would
# leave the sandbox with proxy env vars pointing at iron-proxy
# but no trust anchor, so every TLS handshake would 5xx; or
# worse, with enforce_on_docker=false we'd drop both the proxy
# vars AND any other isolation, opening the sandbox.
msg = (
f"iron-proxy CA cert vanished from {status.ca_cert_path}. "
"Re-run `hermes egress setup` to regenerate it."
)
if enforce:
raise RuntimeError(msg)
logger.warning("%s — continuing without proxy (enforce_on_docker=false).", msg)
return ([], {}, [])
# Corrupt or empty mappings.json is a silent failure mode that's
# indistinguishable from an upstream outage from inside the sandbox
# (every request returns 403). Refuse to mount with empty mappings
# rather than ship a broken sandbox.
mappings = ip.load_mappings()
if not mappings:
msg = (
"iron-proxy is configured but mappings.json is empty or "
"corrupt. Re-run `hermes egress setup` to mint provider "
"tokens before starting a sandbox."
)
if enforce:
raise RuntimeError(msg)
logger.warning("%s — continuing without proxy (enforce_on_docker=false).", msg)
return ([], {}, [])
container_ca = "/etc/ssl/certs/hermes-egress-ca.crt"
volume_args = ["-v", f"{status.ca_cert_path}:{container_ca}:ro"]
proxy_url = f"http://host.docker.internal:{status.tunnel_port}"
env_overrides: dict[str, str] = {
# HTTPS_PROXY / HTTP_PROXY are respected by curl, requests, urllib,
# httpx, node fetch, go default transport, etc. Lowercase variants
# are also set because some tools only look at one casing.
"HTTPS_PROXY": proxy_url,
"https_proxy": proxy_url,
"HTTP_PROXY": proxy_url,
"http_proxy": proxy_url,
# Loopback-only NO_PROXY so localhost dev servers inside the sandbox
# (test fixtures, local LLMs) don't get sent through the proxy.
"NO_PROXY": "127.0.0.1,localhost,::1",
"no_proxy": "127.0.0.1,localhost,::1",
# CA bundle locations for the major language runtimes. iron-proxy
# presents a leaf cert signed by our CA on every MITM'd connection.
#
# CRITICAL ASYMMETRY: Python (REQUESTS_CA_BUNDLE / SSL_CERT_FILE)
# and curl (CURL_CA_BUNDLE) REPLACE the system CA store.
# NODE_EXTRA_CA_CERTS ADDS to it. A Node.js process that
# bypasses HTTPS_PROXY by using a raw socket would still see the
# system CA store and succeed where Python/curl fail validation.
# We additionally set NODE_OPTIONS=--use-openssl-ca to force Node
# through the OpenSSL store that SSL_CERT_FILE controls, narrowing
# the asymmetry. Not a complete fix — see the docs caveat — but
# closes the easy case.
"REQUESTS_CA_BUNDLE": container_ca, # Python `requests`
"SSL_CERT_FILE": container_ca, # Python ssl module / OpenSSL
"CURL_CA_BUNDLE": container_ca, # curl
"NODE_EXTRA_CA_CERTS": container_ca, # Node.js: adds to system store
# NOTE: NODE_OPTIONS is intentionally NOT placed in env_overrides
# here as a flat assignment. We need to APPEND --use-openssl-ca
# to whatever the user already has in NODE_OPTIONS (e.g.
# --max-old-space-size=4096), not clobber it. The append-merge
# happens in DockerEnvironment._merge_node_options below.
# For the agent inside the sandbox to identify itself as proxy-aware.
"HERMES_EGRESS_PROXY": "1",
# Sentinel that DockerEnvironment uses to do the NODE_OPTIONS
# append-merge. Stripped from the final env before docker run.
"_HERMES_EGRESS_NODE_OPTIONS_APPEND": "--use-openssl-ca",
}
# Surface the per-provider proxy tokens. The sandbox can swap these into
# its provider config (or its env, if it reads the standard names) and the
# proxy translates them to the real secrets on egress.
for m in mappings:
env_overrides[f"HERMES_PROXY_TOKEN_{m.real_env_name}"] = m.proxy_token
# On Linux, host.docker.internal isn't populated by default — Docker Desktop
# adds it on macOS/Windows; on Linux we need an explicit --add-host with
# host-gateway. On Desktop this is a no-op (harmless duplicate).
host_args: list[str] = ["--add-host", "host.docker.internal:host-gateway"]
return (volume_args, env_overrides, host_args)
def _build_security_args(run_as_host_user: bool) -> list[str]:
"""Return the security/cap/tmpfs args tailored to the privilege mode."""
if run_as_host_user:
@@ -453,11 +605,155 @@ class DockerEnvironment(BaseEnvironment):
except Exception as e:
logger.debug("Docker: could not load credential file mounts: %s", e)
# Egress credential-injection proxy (iron-proxy) — when configured,
# mount the CA cert into the sandbox and set HTTPS_PROXY + CA-bundle
# env vars so outbound traffic routes through the host-side proxy.
# The sandbox receives PROXY tokens instead of real API keys.
egress_volume_args, egress_env_overrides, egress_host_args = (
_egress_proxy_args_for_docker()
)
volume_args.extend(egress_volume_args)
# egress env overrides are merged in further below alongside the
# other env_args computation.
# Explicit environment variables (docker_env config) — set at container
# creation so they're available to all processes (including entrypoint).
# Egress proxy env vars (HTTPS_PROXY, CA-bundle paths, proxy tokens)
# are merged below. Precedence policy:
#
# - When egress enforcement is on AND the user's docker_env tries
# to override one of the proxy-control vars (HTTPS_PROXY,
# SSL_CERT_FILE, etc.), fail-loud rather than silently inverting
# the isolation. The CA mount + tokens would still ship while
# traffic leaves the sandbox direct with real credentials —
# exactly what enforce_on_docker is meant to prevent.
# - When enforcement is off, the user's docker_env wins (current
# behavior) but we log a warning naming both config sources.
# - When the user override is identical to the egress value, no-op.
if egress_env_overrides:
try:
from hermes_cli.config import load_config as _load_cfg_for_collision
_proxy_cfg = (_load_cfg_for_collision().get("proxy") or {})
except (ImportError, OSError):
_proxy_cfg = {}
except Exception as _e: # noqa: BLE001 — narrowed below via yaml import
# yaml.YAMLError from a malformed config.yaml. We import
# lazily because PyYAML is a soft dep in some test envs.
try:
import yaml # noqa: F401
except ImportError:
raise
logger.warning(
"Could not read proxy config for egress collision check: %s",
_e,
)
_proxy_cfg = {}
_enforce_egress = bool(_proxy_cfg.get("enforce_on_docker", True))
# Egress-controlling env vars that affect the proxy posture.
_critical_proxy_control = {
"HTTPS_PROXY", "https_proxy", "HTTP_PROXY", "http_proxy",
"NO_PROXY", "no_proxy",
"REQUESTS_CA_BUNDLE", "SSL_CERT_FILE", "CURL_CA_BUNDLE",
"NODE_EXTRA_CA_CERTS",
}
# stephenschoettler #2: also block docker_env from injecting
# real provider keys. `docker_env: {OPENROUTER_API_KEY: sk-real}`
# in config.yaml puts the live secret into the sandbox while
# egress is nominally enforced — defeats the entire feature.
# Pull the mapped real_env_name from each token mapping at
# call time so this stays in sync with whatever the operator
# has configured.
_critical_provider_keys: set[str] = set()
try:
from agent.proxy_sources import iron_proxy as _ip_for_mappings
_critical_provider_keys = {
m.real_env_name for m in _ip_for_mappings.load_mappings()
}
except Exception: # noqa: BLE001 — best-effort collision check
pass
_critical = _critical_proxy_control | _critical_provider_keys
_collisions = sorted(
k for k in _critical
if k in self._env
and (
k not in egress_env_overrides
or self._env[k] != egress_env_overrides[k]
)
# For provider keys, ANY override is a collision (the egress
# path mints proxy tokens; a real key in docker_env bypasses
# the swap regardless of whether the egress dict happens to
# carry it).
and (
k in _critical_provider_keys
or (k in egress_env_overrides
and self._env[k] != egress_env_overrides[k])
)
)
if _collisions:
_msg = (
f"docker_env in config.yaml overrides egress-proxy "
f"variables {_collisions}; enforce_on_docker is "
f"{'enabled' if _enforce_egress else 'disabled'}."
)
if _enforce_egress:
raise RuntimeError(
f"{_msg} Remove these keys from docker_env or "
"disable enforce_on_docker to opt out of egress "
"isolation."
)
logger.warning(
"%s Falling back to docker_env values; sandbox traffic "
"will NOT route through the proxy.", _msg,
)
# When enforce_on_docker is true, egress overrides win. When
# false, docker_env wins (back-compat for users who deliberately
# opt out). In both cases the collision check above has already
# surfaced any disagreement.
try:
from hermes_cli.config import load_config as _load_cfg_for_precedence
_enforce_egress_merge = bool(
(_load_cfg_for_precedence().get("proxy") or {})
.get("enforce_on_docker", True)
)
except (ImportError, OSError):
_enforce_egress_merge = True
except Exception: # noqa: BLE001 — yaml.YAMLError or similar
# Malformed config.yaml; fail-safe to enforced.
_enforce_egress_merge = True
if _enforce_egress_merge and egress_env_overrides:
merged_env = dict(self._env)
merged_env.update(egress_env_overrides)
else:
merged_env = dict(egress_env_overrides)
merged_env.update(self._env)
# arshkumarsingh #1: NODE_OPTIONS append-merge. The egress path
# wants ``--use-openssl-ca`` so Node routes through the OpenSSL
# CA store ``SSL_CERT_FILE`` controls. But the operator's
# ``docker_env: {NODE_OPTIONS: "--max-old-space-size=8192"}``
# MUST be preserved — replacing it would silently drop their
# tuning. We carry the egress flag in a sentinel key
# ``_HERMES_EGRESS_NODE_OPTIONS_APPEND`` and merge here.
_egress_node_append = merged_env.pop(
"_HERMES_EGRESS_NODE_OPTIONS_APPEND", None,
)
if _egress_node_append:
existing_node = merged_env.get("NODE_OPTIONS", "")
# De-dup: only add if not already present (the operator may
# have set the same flag themselves).
if _egress_node_append.strip() not in existing_node.split():
if existing_node.strip():
merged_env["NODE_OPTIONS"] = (
f"{existing_node} {_egress_node_append}".strip()
)
else:
merged_env["NODE_OPTIONS"] = _egress_node_append
env_args = []
for key in sorted(self._env):
env_args.extend(["-e", f"{key}={self._env[key]}"])
for key in sorted(merged_env):
env_args.extend(["-e", f"{key}={merged_env[key]}"])
# Optional: run the container as the host user so files written into
# bind-mounted dirs (/workspace, /root, docker_volumes entries) are
@@ -494,6 +790,7 @@ class DockerEnvironment(BaseEnvironment):
+ user_args
+ writable_args
+ resource_args
+ egress_host_args
+ volume_args
+ env_args
+ validated_extra

View File

@@ -0,0 +1,305 @@
---
sidebar_position: 14
title: "Egress proxy internals"
description: "How the iron-proxy egress firewall integrates with Hermes — module layout, lifecycle, security invariants, and extension points"
---
# Egress proxy internals
This page covers the architecture of the egress credential-injection firewall (`hermes egress` / iron-proxy) from a contributor / plugin author's perspective. End-user setup + usage docs live at [Egress proxy](../user-guide/egress/iron-proxy.md).
The threat model and high-level design are summarised on the user page; this page is about *how* it's wired, where the security-relevant code lives, and what invariants you have to preserve if you touch it.
## Module layout
```text
agent/proxy_sources/iron_proxy.py Core: binary install, CA gen, config build,
subprocess lifecycle, mappings I/O, PID/nonce
defense. Pure-function surface where possible.
hermes_cli/proxy_cli.py Wizard + slash command handlers.
`hermes egress {install,setup,start,stop,
status,disable,config}`. Wires the
core module into argparse.
hermes_cli/main.py:_dispatch_egress Top-level subparser dispatcher.
dest='egress_command' (intentionally
disjoint from the inbound OAuth
`hermes proxy` subparser, which uses
dest='proxy_command').
hermes_cli/config.py: proxy schema The `proxy:` block in DEFAULT_CONFIG.
Adding a knob means: add it here, add a
wizard prompt or `setdefault` in
proxy_cli.cmd_setup, and document it
in the user-guide page.
tools/environments/docker.py
_egress_proxy_args_for_docker() Builds the volume_args / env_overrides /
host_args triple that the Docker backend
injects when `proxy.enabled: true`.
DockerEnvironment.__init__ Docker-side merge logic: collision
detection against critical egress vars,
NODE_OPTIONS append-merge via the
_HERMES_EGRESS_NODE_OPTIONS_APPEND
sentinel, enforce_on_docker precedence.
tests/test_iron_proxy.py Hermetic tests (~70). Binary install
path, config build, mappings I/O,
subprocess lifecycle, docker arg builder,
deny CIDR defaults, bind policy, CA
TOCTOU, ensure_audit_log behaviour, etc.
tests/test_iron_proxy_cli.py CLI handler unit tests (~20). Argparse
wiring, fail-loud paths, BWS refresh
wire-up, dest='egress_command'
regression guard.
tests/test_iron_proxy_e2e.py Live E2E (gated on HERMES_RUN_E2E=1).
Real iron-proxy binary, real curl,
end-to-end token swap verified.
```
## Lifecycle
```text
hermes egress install
-> agent.proxy_sources.iron_proxy.install_iron_proxy(force=...)
Downloads pinned tarball + checksums.txt from GitHub Releases.
SHA-256 verification before extraction.
tarfile.extract(..., filter="data") on Python 3.12+ (PEP 706);
falls back to plain extract on older Python with member-name
sanitisation via _pick_tar_member.
Stage into ~/.hermes/bin/.iron-proxy_XXXX, chmod 755, os.replace
to ~/.hermes/bin/iron-proxy (atomic).
_VERSION_CACHE.pop(target) so a forced reinstall re-probes
--version on next call.
hermes egress setup [--from-bitwarden | --no-bitwarden] [--rotate-tokens]
-> proxy_cli.cmd_setup
Step 1. find_iron_proxy(install_if_missing=False) -> install if absent.
Step 2. ensure_ca_cert()
Run openssl genrsa + req via subprocess.
Write CA key via os.open(O_WRONLY|O_CREAT|O_TRUNC|O_NOFOLLOW, 0o600)
+ os.replace. Never exists on disk under default umask.
Write CA cert with 0o644 (public).
Step 3. discover_provider_mappings() or pull names from BWS via
fetch_bitwarden_secrets() when --from-bitwarden.
merge_mappings(existing=load_mappings(), discovered,
rotate=args.rotate_tokens) preserves prior
tokens unless --rotate-tokens is passed.
discover_uncovered_providers() and surface warnings.
Step 4. ensure_audit_log(audit_log_path) # raises on OSError
build_proxy_config(...) with defaults applied at the call site
(deny CIDRs default, bind policy from _default_http_listen).
write_proxy_config(cfg) # atomic via .tmp + os.replace, 0o600
write_mappings(mappings) # atomic, 0o600
Step 5. proxy_cfg["enabled"] = True; credential_source preservation logic
(do NOT silently downgrade bitwarden -> env on re-run);
save_config(cfg).
hermes egress start
-> proxy_cli.cmd_start
Pre-checks (refuse-start path):
- proxy.fail_on_uncovered_providers? -> discover_blocked_providers()
- credential_source=bitwarden? -> pre-validate access_token_env + project_id
-> iron_proxy.start_proxy(
refresh_secrets_from_bitwarden=...,
bitwarden_config=...,
)
existing=_read_pid(); if alive, idempotent return.
_build_proxy_subprocess_env(...): ALLOWLIST + mapped real_env_names,
strip HTTPS_PROXY/etc. to avoid recursion, optional BWS refresh
(raises on missing values unless allow_env_fallback=true).
Plant nonce: _proxy_nonce = sha256(urandom(16)); env[NONCE_ENV] = ...
Open log_path via O_NOFOLLOW + 0o600 + st_uid check.
Popen with stdin=DEVNULL, stdout=log_fd, stderr=STDOUT,
start_new_session=True (POSIX).
Close parent's log_fd in finally.
_write_pidfile_safely(pidfile, proc.pid)
O_EXCL + O_NOFOLLOW + uid check + persisted nonce sidecar.
FileExistsError -> discriminate live vs stale, retry once if stale.
Install SIGINT/SIGTERM handlers (main-thread only).
Poll loop (do-while shape):
while True:
if proc.poll() is not None: tail log + unlink pidfile + raise
if _port_listening("127.0.0.1", tunnel_port): break
if time.time() >= deadline: break (do-while: checked AFTER first probe)
time.sleep(0.1)
If not listening at exit: _kill_and_wait(proc) + unlink pidfile + raise.
hermes egress stop
-> iron_proxy.stop_proxy
_read_pid + _pid_alive guard.
starttime_before = _pid_proc_starttime(pid) # Linux only; None elsewhere
os.kill(pid, SIGTERM)
Wait up to 5s for graceful exit.
After grace: re-check starttime + _pid_alive.
If recycled (starttime drift OR _pid_alive False), DO NOT SIGKILL.
Otherwise os.kill(pid, _KILL_SIGNAL).
_cleanup_state_files: unlink pidfile + nonce sibling.
```
## Security invariants
These are the load-bearing properties. If you touch the module, you must preserve them. Where there's a regression test, it's named.
### Filesystem perms
| Path | Mode | Test |
|---|---|---|
| `~/.hermes/proxy/` (dir) | `0o700` | `test_proxy_state_dir_is_0o700` |
| `ca.key` | `0o600` | `test_ca_key_created_with_0o600` |
| `ca.crt` | `0o644` | (implicit; chmod call in `ensure_ca_cert`) |
| `proxy.yaml` | `0o600` | (chmod after atomic rename in `write_proxy_config`) |
| `mappings.json` | `0o600` | (chmod after atomic rename in `write_mappings`) |
| `iron-proxy.pid` | `0o600` | (`os.open(..., 0o600)` mode in `_write_pidfile_safely`) |
| `iron-proxy.nonce` | `0o600` | (`os.open(..., 0o600)` mode in `_write_pidfile_safely`) |
| `audit.log` | `0o600` | `test_ensure_audit_log_creates_with_0o600` |
| `iron-proxy.log` | `0o600` | (`os.open(..., 0o600)` + `fchmod`) |
All write paths use `os.open(O_WRONLY | O_CREAT | O_NOFOLLOW, 0o600)` + `os.fstat().st_uid` check. `shutil.copy2` + `os.chmod` is forbidden because it leaks a default-umask window.
### Subprocess env minimisation
`_build_proxy_subprocess_env` MUST NOT use `os.environ.copy()`. The allowlist is `_PROXY_SUBPROCESS_ENV_ALLOWLIST` (PATH, HOME, locale, etc.) plus the env names referenced by `load_mappings()`. Everything else stays on the host.
Regression: `test_subprocess_env_strips_unrelated_secrets`, `test_subprocess_env_strips_proxy_recursion_vars`, `test_subprocess_env_keeps_infrastructure_vars`.
### Bind policy
`_default_http_listen` returns loopback (and, on Linux, the docker bridge IP as a *second list entry the rendered yaml currently discards* — see below). Never `0.0.0.0`, never `:PORT` (INADDR_ANY).
`_detect_docker_bridge_ip` validates via `ipaddress.IPv4Address` and rejects `is_unspecified` / `is_loopback` / `is_multicast` / `is_reserved` / `is_link_local` / `is_global`. A hostile `ip` shim on PATH cannot inject `0.0.0.0`.
**v0.39 schema constraint:** the binary's `config.Proxy` struct has only a singular `http_listen` string field — there is no `http_listens` (plural) list, despite earlier comments in this module claiming otherwise. `build_proxy_config` emits only the first entry of `_default_http_listen`'s result; the second-bind path is dead code today. When the pinned `_IRON_PROXY_VERSION` is bumped to one that supports the plural form, re-enable the list-emit in `build_proxy_config` and the docker-bridge bind becomes live without further changes.
Regression: `test_default_bind_is_loopback_not_zero_zero` (asserts loopback bind AND that `http_listens` is NOT in the rendered yaml), `test_default_bind_uses_loopback_on_linux`, `test_detect_docker_bridge_ip_rejects_dangerous` (parametrized over 8 attack inputs).
### Metrics port collision
`metrics.listen` defaults to `:9090` in iron-proxy v0.39 — the SAME port as Hermes's default `tunnel_port: 9090`. `build_proxy_config` MUST explicitly pin `metrics.listen: 127.0.0.1:0` so the metrics binding gets an ephemeral loopback port that can never collide with the proxy listener regardless of operator-chosen `tunnel_port`.
Regression: `test_metrics_listener_pinned_to_loopback_ephemeral`.
### Default deny CIDRs
`_DEFAULT_UPSTREAM_DENY_CIDRS` covers loopback (v4 + v6), link-local (incl. IMDS at 169.254.169.254 and the IPv4-mapped-v6 form), RFC1918, IPv6 ULA, CGNAT, and the RFC2544 benchmark range. `build_proxy_config(..., upstream_deny_cidrs=None)` MUST emit the default; only an explicit empty list opts out.
Regression: `test_default_deny_cidrs_present_when_unspecified`, `test_default_deny_includes_ipv4_mapped_v6`.
### Audit log fail-loud
`ensure_audit_log` raises `RuntimeError` on any `OSError`. Swallowing the failure would let the daemon create the file under the default umask, defeating the privacy promise. `cmd_setup` catches the RuntimeError and surfaces a clear error to the operator.
**v0.39 schema constraint:** `log.audit_path` is NOT a field in iron-proxy v0.39's `config.Log` struct, so `build_proxy_config` accepts the `audit_log` kwarg but does NOT emit it into the rendered yaml. Per-request records on v0.39 land in `iron-proxy.log` alongside daemon-level events. The `audit.log` file is still pre-created at `0o600` with `O_NOFOLLOW` so the privacy contract holds when the pinned version is bumped to one that supports the separate stream.
Regression: `test_ensure_audit_log_raises_on_immutable_parent`, `test_audit_log_kwarg_does_not_inject_audit_path_v039`.
### Bitwarden mode fail-loud
When `credential_source: bitwarden` AND `proxy.allow_env_fallback: false` (default):
- Missing access token env var -> `cmd_start` refuses.
- Missing `project_id` -> `cmd_start` refuses.
- `bws secret list` returns no values for one or more mapped providers -> `_build_proxy_subprocess_env` raises.
Falling back to host env in BW mode reintroduces exactly the staleness bug the BW path is meant to defeat.
Regression: `test_cmd_start_refuses_when_bitwarden_token_missing` (CLI layer); strict-mode assertions in `_build_proxy_subprocess_env` (daemon layer).
### docker_env collision detection
When `enforce_on_docker: true`, `docker_env` overrides on any of the egress-controlling vars (HTTPS_PROXY, SSL_CERT_FILE, NODE_EXTRA_CA_CERTS, etc.) OR any mapped `real_env_name` (OPENROUTER_API_KEY, etc.) raises `RuntimeError` BEFORE the container starts.
Regression: `test_docker_env_collision_with_proxy_raises_when_enforce`.
### PID recycling defense
`_pid_alive` MUST consult either the in-process `_proxy_nonce` (same-process case) OR the on-disk `iron-proxy.nonce` (cross-CLI case) before trusting an `argv[0]` basename match. `stop_proxy` MUST re-check `/proc/<pid>/stat` starttime before SIGKILL and suppress the signal on starttime drift.
Regression: `test_stop_proxy_suppresses_sigkill_on_pid_recycle`, `test_pid_proc_starttime_parses_comm_with_parens`, `test_persisted_nonce_roundtrip`.
### Token preservation on re-setup
`merge_mappings(existing, discovered, rotate=False)` MUST return prior tokens for providers that overlap. Re-running `hermes egress setup` cannot silently 401 running sandboxes. `--rotate-tokens` is the explicit opt-in.
Regression: `test_merge_mappings_preserves_existing_tokens`, `test_merge_mappings_rotate_mints_fresh_tokens`.
### `credential_source` preservation
`cmd_setup` MUST NOT downgrade `credential_source: bitwarden` to `env` on re-run without an explicit `--no-bitwarden` flag. Running `hermes egress setup` (no flag) preserves whatever was previously configured.
Tested via the `cmd_setup` flow in CLI tests (the bitwarden-preservation path is exercised when `--from-bitwarden` is followed by a plain `setup` re-run).
## Extension points
### Adding a new bearer-token provider
`_BEARER_PROVIDERS` in `iron_proxy.py` maps env var name -> tuple of upstream hosts. Adding an entry makes it discoverable by `discover_provider_mappings()`; the wizard mints a token for it automatically when the env var is present.
```python
_BEARER_PROVIDERS: Dict[str, Tuple[str, ...]] = {
...,
"MY_PROVIDER_API_KEY": ("api.myprovider.com",),
}
```
Also update `_DEFAULT_ALLOWED_HOSTS` so the proxy allows the upstream by default. Run `test_discover_provider_mappings_*` to confirm.
### Adding a new non-bearer provider
If the provider uses `x-api-key` / SigV4 / OAuth-from-SDK / etc., iron-proxy's `secrets` transform cannot swap it. Add the env var to `_NON_BEARER_PROVIDERS` so the wizard warns about it. If the provider is LLM-specific enough that you want `fail_on_uncovered_providers: true` to actually block it, also add to `_LLM_SPECIFIC_NON_BEARER_PROVIDERS`.
```python
_NON_BEARER_PROVIDERS: Tuple[str, ...] = (
...,
"MY_X_API_KEY_PROVIDER",
)
_LLM_SPECIFIC_NON_BEARER_PROVIDERS: Tuple[str, ...] = (
...,
"MY_X_API_KEY_PROVIDER",
)
```
### Wiring iron-proxy into a non-Docker backend
`_egress_proxy_args_for_docker` is Docker-specific. Backends that want similar wiring need their own analogue that:
1. Reads `load_config().get("proxy", {})`; returns empty args if `enabled` is false.
2. Calls `iron_proxy.get_status()`; surfaces `enforce` semantics on `configured` / `pid` / `listening` / `ca_cert_path` failure paths.
3. Calls `iron_proxy.load_mappings()`; refuses to mount if empty AND `enforce_on_docker: true`.
4. Sets the seven env vars (HTTPS_PROXY, NO_PROXY, REQUESTS_CA_BUNDLE, SSL_CERT_FILE, CURL_CA_BUNDLE, NODE_EXTRA_CA_CERTS, HERMES_EGRESS_PROXY) and the per-mapping `HERMES_PROXY_TOKEN_<NAME>` vars.
5. Distributes the CA cert into the sandbox at a path the runtime will trust (typically `/etc/ssl/certs/hermes-egress-ca.crt`).
6. Implements collision detection against the user's backend-specific env config.
The Docker implementation is ~150 lines; expect similar volume for Modal / Daytona / SSH.
### Subscribing to per-request audit events
iron-proxy writes line-delimited JSON to `~/.hermes/proxy/iron-proxy.log` on the currently pinned v0.39 (daemon + per-request records combined; see "Logging on iron-proxy v0.39" in the user guide). A plugin / external watcher can tail that file and react to allowlist denials, secret swaps, or upstream errors. When the pinned version is bumped to one that supports `log.audit_path`, the per-request stream moves to `audit.log` and watchers wired to that path go live without operator action. The schema is documented at [docs.iron.sh/audit](https://docs.iron.sh/audit) (link).
## Testing
```bash
# Hermetic suite (no network, no real binary)
scripts/run_tests.sh tests/test_iron_proxy.py tests/test_iron_proxy_cli.py
# Live E2E (real binary, real curl, real CONNECT tunnel)
HERMES_RUN_E2E=1 scripts/run_tests.sh tests/test_iron_proxy_e2e.py
# Live PTY smoke against `hermes egress`
HERMES_HOME=/tmp/hermes-egress-test python3 -m hermes_cli.main egress --help
HERMES_HOME=/tmp/hermes-egress-test python3 -m hermes_cli.main egress setup --help
```
The CLI uses argparse, so `--help` is a good first probe for "did my new flag register correctly".
## See also
- User-facing setup + troubleshooting: [Egress proxy](../user-guide/egress/iron-proxy.md)
- Docker backend internals: [Docker](../user-guide/docker.md)
- Bitwarden Secrets Manager integration: [`hermes secrets bitwarden`](../user-guide/secrets/bitwarden.md)
- CLI command reference: [`hermes egress`](../reference/cli-commands.md#hermes-egress)
- Sandbox-injected environment variables: [Egress proxy (sandbox-injected)](../reference/environment-variables.md#egress-proxy-sandbox-injected)

View File

@@ -256,6 +256,8 @@ hermes config set terminal.backend docker # Docker isolation
hermes config set terminal.backend ssh # Remote server
```
For Docker sandboxes, you can also enable the **egress credential-injection proxy** so the sandbox never sees your real API keys — only opaque proxy tokens that work exclusively from behind a local TLS-intercepting daemon. See [Egress proxy](../user-guide/egress/iron-proxy.md). Setup is `hermes egress setup && hermes egress start`; the Docker backend wires everything up automatically once `proxy.enabled` flips on.
### Voice mode
```bash

View File

@@ -41,6 +41,7 @@ hermes [global-options] <command> [subcommand/options]
| `hermes fallback` | Manage fallback providers tried when the primary model errors. |
| `hermes gateway` | Run or manage the messaging gateway service. |
| `hermes proxy` | Local OpenAI-compatible proxy that attaches OAuth provider credentials. See [Subscription Proxy](../user-guide/features/subscription-proxy.md). |
| `hermes egress` | Outbound credential-injection firewall for remote terminal sandboxes (iron-proxy). Disabled by default. See [Egress proxy](../user-guide/egress/iron-proxy.md). |
| `hermes lsp` | Manage Language Server Protocol integration (semantic diagnostics for write_file/patch). |
| `hermes setup` | Interactive setup wizard for all or part of the configuration. |
| `hermes whatsapp` | Configure and pair the WhatsApp bridge. |
@@ -458,6 +459,65 @@ All actions are also available as a slash command in the gateway (`/kanban …`)
For the full design — comparison with Cline Kanban / Paperclip / NanoClaw / Gemini Enterprise, eight collaboration patterns, four user stories, concurrency correctness proof — see `docs/hermes-kanban-v1-spec.pdf` in the repository or the [Kanban user guide](/user-guide/features/kanban).
## `hermes egress`
Outbound credential-injection firewall for remote terminal sandboxes. Wraps the [iron-proxy](https://github.com/ironsh/iron-proxy) daemon — a TLS-intercepting proxy that swaps opaque proxy tokens for real upstream API credentials at the network boundary, so sandboxes never hold real keys. Disabled by default; see the full [Egress proxy](../user-guide/egress/iron-proxy.md) page for setup + architecture.
```bash
hermes egress install # download the pinned iron-proxy binary
hermes egress install --force # re-download even if already installed
hermes egress setup # interactive wizard: CA, mappings, config
hermes egress setup --tunnel-port N # override the tunnel listener port (default 9090)
hermes egress setup --from-bitwarden # use Bitwarden Secrets Manager as credential source
hermes egress setup --no-bitwarden # explicitly switch back to env-based credentials
hermes egress setup --rotate-tokens # mint fresh proxy tokens (default preserves existing)
hermes egress start # spawn the managed proxy daemon
hermes egress stop # SIGTERM (then SIGKILL after 5s grace)
hermes egress status # binary + config + pid + listening + mappings
hermes egress status --show-tokens # print proxy tokens in full (default: redacted)
hermes egress disable # flip proxy.enabled = false (does not stop a running proxy)
hermes egress config # print the path to proxy.yaml for inspection
```
### Common flows
```bash
# First-time setup
export OPENROUTER_API_KEY=
hermes egress setup && hermes egress start
hermes config set terminal.backend docker # if not already
# Switching credential source after the fact
hermes egress setup --from-bitwarden # env → bitwarden
hermes egress setup --no-bitwarden # bitwarden → env
# (just `setup` without either flag preserves the existing mode)
# Rotating all tokens (e.g. after a suspected token leak)
hermes egress setup --rotate-tokens
hermes egress stop && hermes egress start # restart daemon to pick up new mappings
# (running sandboxes still hold old tokens; restart them too)
# Adding a new upstream
# Edit ~/.hermes/config.yaml proxy.extra_allowed_hosts: [api.example.com]
hermes egress setup
hermes egress stop && hermes egress start
```
### Diagnostic shortcuts
```bash
hermes egress status # current state in one view
cat ~/.hermes/proxy/proxy.yaml # the rendered iron-proxy config
tail -20 ~/.hermes/proxy/iron-proxy.log # daemon-level diagnostics
tail -f ~/.hermes/proxy/iron-proxy.log | jq # daemon + per-request log (line-delimited JSON; v0.39 combines both streams)
```
Common failure modes + recovery are covered in [Egress proxy → Troubleshooting](../user-guide/egress/iron-proxy.md#troubleshooting).
## `hermes webhook`
```bash

View File

@@ -237,6 +237,22 @@ For cloud sandbox backends, persistence is filesystem-oriented. `TERMINAL_LIFETI
| `TERMINAL_LOCAL_PERSISTENT` | Enable persistent shell for local backend (default: `false`) |
| `TERMINAL_SSH_PERSISTENT` | Override persistent shell for SSH backend (default: follows `TERMINAL_PERSISTENT_SHELL`) |
## Egress proxy (sandbox-injected)
These env vars are NOT set on the host — they're injected into Docker sandboxes by the [Egress proxy](../user-guide/egress/iron-proxy.md) integration when `proxy.enabled: true`. The agent code reads them instead of real API keys.
| Variable | Description |
|----------|-------------|
| `HERMES_EGRESS_PROXY` | Set to `1` inside a sandbox when the egress proxy is active. Agent code can check this to know it's running behind a TLS-intercepting proxy. |
| `HERMES_PROXY_TOKEN_<ENV_NAME>` | One per minted provider mapping. E.g. `HERMES_PROXY_TOKEN_OPENROUTER_API_KEY=hermes-proxy-openrouter-…`. The sandbox uses these in the `Authorization: Bearer` header; iron-proxy swaps them for the real upstream secret at the network boundary. |
| `HTTPS_PROXY` / `HTTP_PROXY` | Set to `http://host.docker.internal:<tunnel_port>` so every standard HTTP client routes through iron-proxy. |
| `NO_PROXY` | `127.0.0.1,localhost,::1` so loopback dev servers inside the sandbox bypass the proxy. |
| `REQUESTS_CA_BUNDLE` / `SSL_CERT_FILE` / `CURL_CA_BUNDLE` / `NODE_EXTRA_CA_CERTS` | Path to the mounted Hermes egress CA cert inside the sandbox (`/etc/ssl/certs/hermes-egress-ca.crt`). Lets the language runtimes trust iron-proxy's MITM-minted leaf certs. |
| `NODE_OPTIONS` | Appended with `--use-openssl-ca` (your existing flags are preserved) so Node.js routes through the OpenSSL store the other CA-bundle vars control. Narrows the [Node.js asymmetric CA caveat](../user-guide/egress/iron-proxy.md#nodejs-asymmetric-ca-caveat). |
| `HERMES_IRON_PROXY_NONCE` | Set on the iron-proxy daemon process itself (NOT inside the sandbox). Used by `_pid_alive` to confirm a candidate PID still refers to *our* managed binary across PID recycling. |
These are set automatically by the Docker terminal backend when `proxy.enabled: true` AND the daemon is running. You don't set them yourself; the relevant operator-facing knobs are in `~/.hermes/config.yaml` under the `proxy:` section — see [Egress proxy → Configuration](../user-guide/egress/iron-proxy.md#configuration).
## Messaging
| Variable | Description |

View File

@@ -0,0 +1,10 @@
---
title: Egress proxy
sidebar_position: 1
---
# Egress proxy
Optional outbound credential-injection firewall for remote terminal sandboxes. The sandbox only ever holds opaque proxy tokens; real API keys never leave the host.
- [iron-proxy](./iron-proxy) — single-binary TLS-intercepting proxy from [ironsh/iron-proxy](https://github.com/ironsh/iron-proxy), lazy-installed and managed by `hermes egress`.

View File

@@ -0,0 +1,574 @@
# Egress credential-injection proxy (iron-proxy)
When Hermes runs your agent inside a remote terminal sandbox — Docker, Modal, SSH — that sandbox normally holds your real upstream API keys (`OPENROUTER_API_KEY`, `OPENAI_API_KEY`, etc.). A prompt-injected agent in that sandbox can `cat ~/.config/openrouter/auth.json` or `printenv | grep -i key` and exfiltrate them.
The egress proxy fixes this: the sandbox holds opaque **proxy tokens**, never the real keys. All outbound traffic from the sandbox routes through a local [iron-proxy](https://github.com/ironsh/iron-proxy) daemon (Apache-2.0, Go) on the host, which terminates TLS and swaps the proxy token for the real credential before forwarding the request upstream. Compromise the sandbox and the attacker walks away with tokens that only work from behind the proxy.
This page covers the Docker backend, which is what v1 ships. Modal, Daytona, and SSH wiring will follow in later releases.
## What it is
- A managed `iron-proxy` subprocess on the host, lazy-installed into `~/.hermes/bin/iron-proxy`
- A local CA at `~/.hermes/proxy/ca.crt` that the sandbox trusts so iron-proxy can MITM TLS and rewrite headers
- A `proxy.yaml` config at `~/.hermes/proxy/proxy.yaml` listing the upstream hosts you allow and the secrets-transform mapping
- A `mappings.json` recording which proxy token corresponds to which real env var
The sandbox gets `HTTPS_PROXY=http://host.docker.internal:9090` plus a set of `HERMES_PROXY_TOKEN_<ENV_NAME>` env vars. The agent code reads those tokens instead of the real API keys. iron-proxy's `secrets` transform matches the token in the `Authorization` header and substitutes the real value sourced from its own environment.
## What it is not
- It is **not** the inbound `hermes proxy` command, which is an OAuth aggregator reverse proxy. Different command (`hermes egress`), different direction.
- It does **not** sit between your local terminal and providers — only between the sandbox and providers.
- It does **not** rewrite credentials for in-process LLM calls the host process makes. Those continue to use your `.env` keys directly. The threat model is the *sandbox*, not the host.
## Quick start
```bash
# 1. Install the iron-proxy binary (pinned version, SHA-256 verified)
hermes egress install
# 2. Run the wizard: generates CA, mints proxy tokens for every provider key
# in your env, writes proxy.yaml.
hermes egress setup
# 3. Start the proxy daemon
hermes egress start
# 4. Check status
hermes egress status
```
Once running, the Docker terminal backend automatically:
- Mounts `~/.hermes/proxy/ca.crt` into the sandbox at `/etc/ssl/certs/hermes-egress-ca.crt`
- Sets `HTTPS_PROXY`, `HTTP_PROXY`, `REQUESTS_CA_BUNDLE`, `SSL_CERT_FILE`, `CURL_CA_BUNDLE`, `NODE_EXTRA_CA_CERTS` to make every common HTTP runtime route through the proxy and trust the CA
- Sets `NODE_OPTIONS=--use-openssl-ca` (appended to whatever you already have in `docker_env.NODE_OPTIONS`) so Node.js routes through the OpenSSL store the other CA-bundle vars control — see [Node.js asymmetric CA caveat](#nodejs-asymmetric-ca-caveat) below for the residual gap
- Adds `--add-host=host.docker.internal:host-gateway` so the sandbox can reach the host-side proxy on Linux (Docker Desktop handles this automatically on macOS/Windows)
- Exports one `HERMES_PROXY_TOKEN_<ENV_NAME>` per minted mapping
## Configuration
The full config lives in `~/.hermes/config.yaml` under the `proxy:` section. Defaults are documented inline; everything is optional.
```yaml
proxy:
# Master switch. When false the feature is a complete no-op — no
# binaries downloaded, no docker mounts added, no subprocess started.
enabled: false
# Tunnel listener port. Sandboxes hit http://host.docker.internal:<port>.
tunnel_port: 9090
# Auto-download the pinned iron-proxy binary on first use.
auto_install: true
# Where iron-proxy looks up the real upstream secrets at egress time.
# env — process env (default). Whatever is in your ~/.hermes/.env
# at proxy-start time is the source of truth.
# bitwarden — refetch from Bitwarden Secrets Manager on each proxy
# restart. Rotation in the BW web app propagates without
# touching .env. Requires `secrets.bitwarden.enabled: true`.
credential_source: env
# When true (default), the Docker backend refuses to start a sandbox if
# the proxy is enabled but not running. Set to false to fall back to the
# legacy "real credentials inside the sandbox" posture when the proxy
# is unavailable.
enforce_on_docker: true
# When true, `hermes egress start` refuses to start if LLM-specific
# non-bearer provider env vars are set (Anthropic native, Azure OpenAI,
# Gemini) — those bypass the proxy's secrets transform and would leak
# real credentials into the sandbox. Defaults to false because the
# false-positive cost (operator has the env set but doesn't actually
# use that provider) is higher than the security cost of a warning.
# See "Uncovered providers" below for the strict tier vs warn tier
# distinction.
fail_on_uncovered_providers: false
# When `credential_source: bitwarden` but the BWS access token /
# project_id is missing OR the bws fetch returns no values for mapped
# providers, the daemon raises by default (matches the spirit of "I
# asked for rotation — don't silently use stale env values"). Set
# to true to opt back into the legacy host-env fallback — useful for
# migrations where you want to start switching to BW mode but haven't
# wired every secret yet.
allow_env_fallback: false
# SSRF deny list applied to outbound traffic. Omit / leave null to
# use the safe default: loopback (v4 + v6), link-local (incl. cloud
# metadata IPs at 169.254.169.254), RFC1918, IPv6 ULA, IPv4-mapped-v6,
# CGNAT, and the RFC2544 benchmark range. Set to an explicit `[]`
# to opt out entirely (only sensible in hermetic tests).
upstream_deny_cidrs: null
# Extra allowed upstream hosts beyond the bundled defaults.
# Wildcards (`*.foo.com`) are supported. The defaults cover OpenRouter,
# OpenAI, Anthropic, Google, xAI, Mistral, Groq, Together, DeepSeek,
# and Nous Research.
extra_allowed_hosts: []
```
### Default allowed upstream hosts
```
openrouter.ai *.openrouter.ai
api.openai.com api.anthropic.com
generativelanguage.googleapis.com
api.x.ai api.mistral.ai
api.groq.com api.together.xyz
api.deepseek.com inference.nousresearch.com
```
If your agent needs an upstream that isn't on the list — a self-hosted inference endpoint, an extra cloud LLM, an MCP server — add it to `proxy.extra_allowed_hosts`. Wildcards are matched against the full hostname (`*.example.com` matches `api.example.com` and `staging.example.com` but not `example.com` itself).
### Default SSRF deny CIDRs
Applied regardless of allowlist. These ranges are refused by iron-proxy at the network boundary, so a DNS rebinding attack via an allowlisted hostname can't reach IMDS or your internal network:
| CIDR | Purpose |
|---|---|
| `127.0.0.0/8`, `::1/128` | Loopback (v4 + v6) |
| `169.254.0.0/16`, `fe80::/10` | Link-local — **incl. AWS / GCP / Azure IMDS at `169.254.169.254`** |
| `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16` | RFC1918 |
| `fc00::/7` | IPv6 ULA |
| `::ffff:0:0/96` | IPv4-mapped IPv6 — closes the dual-stack IMDS bypass |
| `100.64.0.0/10` | RFC6598 CGNAT (used by AWS VPC, K8s pod networks) |
| `198.18.0.0/15` | RFC2544 benchmark range |
To override: set `proxy.upstream_deny_cidrs` to your own list. To opt out entirely (e.g. for a hermetic test that needs to reach a loopback upstream): set it to an empty list `[]`.
### Bind policy
The proxy binds **loopback only** (`127.0.0.1:<tunnel_port>`). It does NOT bind `0.0.0.0`. This means:
- A LAN peer with a leaked proxy token cannot use it — the proxy is unreachable from the network.
- Containers reach the proxy via `host.docker.internal:9090`, which Docker maps to the host gateway via `--add-host=host.docker.internal:host-gateway` on Linux. On macOS / Windows Docker Desktop, Desktop manages the gateway itself.
iron-proxy v0.39 only supports a single bind per daemon process — earlier drafts of this integration emitted a plural `http_listens` list with the docker bridge IP appended for direct sandbox-to-bridge connectivity, but v0.39's YAML parser rejects that field. The `host.docker.internal -> host-gateway` mapping that Docker provides is sufficient: containers resolve the hostname to the bridge IP, then connect TO the host's loopback bind through it.
We also pin `metrics.listen: 127.0.0.1:0` so the daemon's built-in metrics server gets an ephemeral loopback port instead of its default `:9090` — otherwise it would fight `tunnel_port: 9090` for the same socket and the daemon would refuse to start with "address already in use".
If a hostile `ip` shim earlier on PATH had been able to inject a non-private IPv4 here (`0.0.0.0`, a public address, multicast, link-local, etc.) the loopback fallback still applies — we never bind anything we couldn't validate via `ipaddress.IPv4Address` + `is_*` checks.
## Uncovered providers
iron-proxy's `secrets` transform only handles `Authorization: Bearer` headers. Providers using `x-api-key`, SigV4, AAD tokens, or custom signatures cannot be proxied — if their env vars are present, the sandbox holds **real credentials** for those providers and the egress isolation guarantee is incomplete for them.
The wizard and `hermes egress status` always surface uncovered providers in your env. There are two tiers:
### Strict tier — refuses start when `fail_on_uncovered_providers: true`
| Env var | Provider | Reason |
|---|---|---|
| `ANTHROPIC_API_KEY` | Anthropic native | x-api-key header, not Bearer |
| `AZURE_OPENAI_API_KEY` | Azure OpenAI | api-key header + optional AAD |
| `GEMINI_API_KEY` | Google AI Studio (Gemini) | x-goog-api-key |
These are LLM-specific names. An operator who has them set is using those providers; a bypass is a real isolation failure.
### Warn-only tier — surfaced but never blocks
| Env var | Provider | Reason |
|---|---|---|
| `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` | AWS Bedrock / SageMaker | SigV4-signed |
| `GOOGLE_APPLICATION_CREDENTIALS` | GCP Vertex AI | gcloud OAuth |
| `GOOGLE_API_KEY` | Google AI Studio | x-goog-api-key OR query param |
These env vars are present on most developer laptops for unrelated tooling (terraform, gcloud, aws CLI, ECR push). They surface as warnings in the wizard + `status` output but don't refuse-start.
### Operator playbook
If `hermes egress start` refuses because of a strict-tier env var you don't actually use:
```bash
unset ANTHROPIC_API_KEY # or whichever one is flagged
hermes egress start
```
If you DO use that provider but accept the isolation gap:
```yaml
# config.yaml
proxy:
fail_on_uncovered_providers: false # default
```
Either way, the warning persists in `hermes egress status` until you remove the env var.
## Bitwarden integration
If you already use Bitwarden Secrets Manager via [`hermes secrets bitwarden setup`](../secrets/bitwarden), the egress proxy can pull real credentials from there instead of `os.environ`:
```bash
hermes egress setup --from-bitwarden
```
This sets `proxy.credential_source: bitwarden` and discovers provider env names from your BW project.
### Rotation semantics
When `credential_source: bitwarden`, the iron-proxy daemon refetches secrets from BWS via `bws secret list <project_id>` **every time it starts**. So the rotation flow is:
1. Rotate a key in the Bitwarden web app.
2. `hermes egress stop && hermes egress start` on the host.
3. Sandboxes started after that point swap proxy tokens for the new value.
No `.env` edits. No Hermes restart on the host. The proxy daemon is the only thing that touches the new value — your host process and `os.environ` are untouched.
### Fail-loud at start
When `credential_source: bitwarden`, `hermes egress start` pre-checks at the wizard layer AND `_build_proxy_subprocess_env` re-checks at the daemon layer:
- BWS access token env var is unset → refuse to start with a hint to `unset` and re-run, or `hermes egress setup --no-bitwarden` to switch back to env mode
- `secrets.bitwarden.project_id` is empty → refuse to start with a hint to run `hermes secrets bitwarden setup`
- `bws secret list` returns no values for one or more mapped providers → refuse to start, listing the missing names
This is intentional. Falling back to host env in BW mode reintroduces exactly the staleness bug the BW path is meant to defeat (operator picked BW for the rotation guarantee; silent fallback breaks that guarantee).
The `proxy.allow_env_fallback: true` config flag opts back in to the legacy "silently fall back to host env if BWS is unreachable" behavior for migration scenarios. Use it when you're moving secrets into BW one at a time and want the daemon to start with whichever values are available.
### Switching credential source
| From | To | Command |
|---|---|---|
| env | bitwarden | `hermes egress setup --from-bitwarden` |
| bitwarden | env | `hermes egress setup --no-bitwarden` |
**Re-running `hermes egress setup` WITHOUT either flag preserves the existing `credential_source`** — the wizard refuses to silently downgrade you back to env. This matters because once you've configured bitwarden mode, the rotation guarantee is what you signed up for; you have to explicitly say "I want env again" to change it.
## Slash commands
The CLI subcommand tree:
```
hermes egress install # download the pinned iron-proxy binary
hermes egress install --force # re-download even if a managed copy exists
hermes egress setup # interactive wizard
hermes egress setup --tunnel-port N # override the tunnel listener port
hermes egress setup --from-bitwarden # use BWS as credential source (fail-loud)
hermes egress setup --no-bitwarden # explicitly switch back to env mode
hermes egress setup --rotate-tokens # mint fresh tokens for every provider
# (default preserves existing)
hermes egress start # spawn the managed proxy daemon
hermes egress stop # SIGTERM (then SIGKILL after 5s grace)
hermes egress status # binary + config + pid + listening state + mappings
hermes egress status --show-tokens # print proxy tokens in full
# (default: redacted prefix + suffix only)
hermes egress disable # flip proxy.enabled = false
# (does not stop a running proxy)
hermes egress config # print the path to proxy.yaml for debugging
```
### Token rotation
By default, `hermes egress setup` **preserves** proxy tokens for providers that already have them. Adding a new provider mints a fresh token only for the new one; existing tokens are unchanged. This avoids 401-ing running sandboxes when you re-run the wizard.
`--rotate-tokens` rolls every token:
```bash
hermes egress setup --rotate-tokens
```
When there are existing tokens AND stdin is a tty, the wizard prompts for confirmation:
```
⚠ --rotate-tokens will invalidate proxy tokens in every running
Hermes sandbox. They will start 401-ing against upstreams until restarted.
Type 'rotate' to confirm:
```
Non-tty invocations (CI, scripts) skip the prompt — the flag is treated as deliberate. Before any overwrite the current `mappings.json` is copied to a timestamped sibling so manual recovery is possible:
```
backup: ~/.hermes/proxy/mappings.json.rotated-20260524T143012
```
**Caveat:** rotating tokens DOES NOT automatically restart iron-proxy. The running daemon still has the old mappings in memory (and the old YAML). After `--rotate-tokens`:
```bash
hermes egress stop && hermes egress start
```
Containers already running hold the old tokens and will need to be restarted to pick up the new ones.
## State directory layout
Everything iron-proxy maintains lives in `~/.hermes/proxy/`:
| Path | Mode | Purpose |
|---|---|---|
| `~/.hermes/proxy/` (dir) | `0o700` | Owned + traversable by you only |
| `ca.crt` | `0o644` | Public CA cert distributed into sandboxes |
| `ca.key` | `0o600` | CA signing key — never leaves the host |
| `proxy.yaml` | `0o600` | iron-proxy config; rewritten every `setup` |
| `mappings.json` | `0o600` | Sandbox proxy token → upstream env var |
| `mappings.json.rotated-*` | `0o600` | Backups created by `--rotate-tokens` |
| `iron-proxy.pid` | `0o600` | PID of the running daemon |
| `iron-proxy.nonce` | `0o600` | Per-start nonce for PID-recycle defense |
| `iron-proxy.log` | `0o600` | Daemon stdout/stderr — **includes per-request records on v0.39** |
| `audit.log` | `0o600` | Reserved for the dedicated per-request audit stream on future binary versions; pre-created so the privacy contract holds when upstream wires it in |
The CA private key is the most sensitive file. It's created with `0o600` from the first byte (no umask-window TOCTOU) and `O_NOFOLLOW` so a same-uid attacker can't redirect it via a planted symlink. The pidfile, nonce file, daemon log, and audit log get the same treatment.
### Logging on iron-proxy v0.39
On the currently pinned binary version (**v0.39.0**) iron-proxy writes ALL output — daemon-level diagnostics AND per-request records — to **`~/.hermes/proxy/iron-proxy.log`**. v0.39's `config.Log` struct doesn't have a separate `audit_path` field, so we can't route per-request records to a dedicated stream there.
We still pre-create `~/.hermes/proxy/audit.log` at `0o600` with `O_NOFOLLOW` because:
1. It serves as a stable logrotate / fluent-bit / monitoring target — operators can wire downstream tooling to that path today, and when we bump the pinned version to one that supports `log.audit_path`, the records will start flowing without any operator-side reconfiguration.
2. The 0o600-from-first-byte guarantee defends against the upstream-fix-day where v0.40+ creates the file under its default umask if it doesn't already exist.
Until that version bump lands, treat `iron-proxy.log` as the source of truth for both audiences:
- Daemon-level events (startup banner, bind errors, shutdown reason, transform errors). Operations + troubleshooting.
- Per-request records (CONNECT to allowlisted upstream, secret swap fired, allowlist denial). Forensics + compliance.
Both files are appended to across restarts. Rotate them with logrotate if you care about disk usage on long-lived hosts.
## How it works
```
┌──────────────┐ ┌──────────────┐ ┌─────────────┐
│ Docker │ CONNECT / │ iron-proxy │ HTTPS w/ │ OpenRouter │
│ sandbox ├──────────────▶│ (host:9090) ├───────────────▶│ / OpenAI / │
│ │ HTTP forward │ │ real API key │ Anthropic … │
│ has: │ w/ proxy tok │ mints leaf │ │ │
│ - proxy tok │ in Auth hdr │ cert from CA │ │ │
│ - CA cert │ │ matches token │ │ │
│ - HTTPS_PROXY│ │ swaps secret │ │ │
└──────────────┘ └──────────────┘ └─────────────┘
│ daemon + per-request log (combined on v0.39)
~/.hermes/proxy/iron-proxy.log
(~/.hermes/proxy/audit.log reserved for v0.40+ split stream)
```
1. Sandbox makes an HTTPS request, e.g. `POST https://openrouter.ai/v1/chat/completions` with `Authorization: Bearer hermes-proxy-openrouter-…` (the proxy token, not the real key).
2. Because `HTTPS_PROXY` is set, the request goes to iron-proxy as a CONNECT tunnel.
3. iron-proxy checks the allowlist. `openrouter.ai` is allowed.
4. iron-proxy mints a leaf cert signed by our CA for `openrouter.ai`, terminates the TLS connection, inspects the request.
5. The `secrets` transform matches the proxy-token string in the `Authorization` header and substitutes the real `OPENROUTER_API_KEY` value, sourced from iron-proxy's own environment.
6. Request is re-encrypted and forwarded to OpenRouter.
7. The request is logged to `~/.hermes/proxy/iron-proxy.log` on v0.39. When the pinned binary version supports the split stream (v0.40+), per-request records will flow to `~/.hermes/proxy/audit.log` and daemon-level diagnostics will stay in `iron-proxy.log`. See [Logging on iron-proxy v0.39](#logging-on-iron-proxy-v039).
A request to a non-allowlisted host (e.g. `https://attacker.example.com/leak?key=...`) is rejected with HTTP 403 before any bytes leave the host. The denial is recorded in `iron-proxy.log` with the upstream host and the source sandbox.
### CA distribution into the sandbox
When the Docker backend starts a container with `proxy.enabled: true` and the daemon is listening, it adds these arguments to `docker run`:
| Arg | Purpose |
|---|---|
| `-v ~/.hermes/proxy/ca.crt:/etc/ssl/certs/hermes-egress-ca.crt:ro` | Read-only mount of the CA |
| `-e HTTPS_PROXY=http://host.docker.internal:9090` | Python httpx / curl / go default transport / Node fetch |
| `-e HTTP_PROXY=…` | curl + wget for plain HTTP (rare in modern stacks) |
| `-e NO_PROXY=127.0.0.1,localhost,::1` | Loopback dev servers inside the sandbox bypass the proxy |
| `-e REQUESTS_CA_BUNDLE=…ca.crt` | Python `requests` |
| `-e SSL_CERT_FILE=…ca.crt` | Python `ssl` module / OpenSSL — **replaces** the system store |
| `-e CURL_CA_BUNDLE=…ca.crt` | curl — **replaces** the system store |
| `-e NODE_EXTRA_CA_CERTS=…ca.crt` | Node.js — **adds** to the system store |
| `-e NODE_OPTIONS="<your value> --use-openssl-ca"` | Node.js — route through OpenSSL store (appended; your `--max-old-space-size` etc. are preserved) |
| `-e HERMES_EGRESS_PROXY=1` | Sentinel the agent can read to know it's proxy-aware |
| `-e HERMES_PROXY_TOKEN_<NAME>=…` | One per mapping; the sandbox uses these instead of real keys |
| `--add-host=host.docker.internal:host-gateway` | Linux-only; Docker Desktop maps it automatically |
#### Node.js asymmetric CA caveat
`REQUESTS_CA_BUNDLE` / `SSL_CERT_FILE` / `CURL_CA_BUNDLE` **replace** the system CA store inside the sandbox. `NODE_EXTRA_CA_CERTS` **adds** to it. A Node.js process inside the sandbox could in principle bypass the proxy by opening a raw `net.Socket` and starting its own TLS handshake — the system CA store would still trust real upstream certs, so the request would succeed where Python / curl would fail validation.
`NODE_OPTIONS=--use-openssl-ca` is appended to whatever you already have in `docker_env.NODE_OPTIONS`. This forces Node through the OpenSSL store that `SSL_CERT_FILE` controls, narrowing the asymmetry. It does NOT cover code that explicitly passes its own `ca` option to `tls.connect()` or `https.request()`, but it closes the easy case.
This is a known v1 limitation. Track [github.com/ironsh/iron-proxy/issues](https://github.com/ironsh/iron-proxy/issues) for an upstream resolution; in the meantime, do not run untrusted Node code that opens raw sockets in a sandbox you're depending on egress isolation for.
### docker\_env collisions
If you set proxy-controlling env vars in your `docker_env:` config block (rare but possible), Hermes refuses to start the sandbox when `enforce_on_docker: true` is set. This includes both:
- Egress-control vars: `HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, `REQUESTS_CA_BUNDLE`, `SSL_CERT_FILE`, `CURL_CA_BUNDLE`, `NODE_EXTRA_CA_CERTS`
- Real provider env vars: every name in `mappings.json` (e.g. `OPENROUTER_API_KEY`, `OPENAI_API_KEY`)
Example error:
```
docker_env in config.yaml overrides egress-proxy variables
['HTTPS_PROXY', 'OPENROUTER_API_KEY']; enforce_on_docker is enabled.
Remove these keys from docker_env or disable enforce_on_docker to
opt out of egress isolation.
```
With `enforce_on_docker: false` the same situation surfaces as a warning and your `docker_env` values win — useful for migrations or testing, but you're explicitly opting OUT of the isolation guarantee.
## PID and nonce defense
The daemon's pidfile is written with `O_EXCL` + `O_NOFOLLOW` + ownership check. Concurrent `hermes egress start` calls produce one of two outcomes:
- The existing pidfile points at a live iron-proxy → second start refuses with "another start in progress" + a hint to run `hermes egress stop`
- The existing pidfile is stale (crashed daemon) → second start unlinks it and retries once
Beyond that, every `start_proxy` plants a fresh random nonce in two places:
- `HERMES_IRON_PROXY_NONCE=<nonce>` in the daemon's env
- `~/.hermes/proxy/iron-proxy.nonce` (0o600 sibling of the pidfile)
When `hermes egress stop` (or any other `_pid_alive` check) wants to confirm a PID still refers to *our* daemon — not an unrelated process that was assigned the same PID after iron-proxy crashed — it reads `/proc/<pid>/environ` and looks for the nonce. The on-disk copy is what makes this work across CLI invocations (the in-memory `_proxy_nonce` is per-process and resets on every `hermes` invocation).
If the nonce check fails, the code falls back to matching `argv[0]` basename against `iron-proxy`. `stop_proxy` additionally captures `/proc/<pid>/stat` starttime before SIGTERM and re-verifies after the 5s grace window — if starttime drifted, the PID was recycled mid-wait and SIGKILL is suppressed with a warning.
## Security model
**What this protects against:**
- Prompt-injected agent in a Docker sandbox reading `printenv` / credential files and exfiltrating real keys.
- Compromised dependency in the sandbox phoning home to an arbitrary host — default-deny allowlist blocks unknown destinations.
- Agent dialing cloud metadata endpoints (`169.254.169.254`) — iron-proxy denies these by default via `upstream_deny_cidrs`, including the IPv4-mapped-v6 form `::ffff:169.254.169.254`.
- DNS rebinding through an allowlisted hostname to a private IP — the deny CIDRs are checked at connect time, not at allowlist time.
- Same-uid local processes reading the iron-proxy daemon's env to scrape secrets — only the env var names referenced by mappings are forwarded, not the full host env.
- A LAN peer with a leaked sandbox proxy token spending your API quota — the proxy binds loopback only, never `0.0.0.0` (containers reach it via `host.docker.internal -> host-gateway`).
**What it does NOT protect against:**
- A compromised host process. If the agent process itself is compromised, real keys in the host's `~/.hermes/.env` are exposed regardless. This is a defense-in-depth feature for *sandbox* compromise, not host compromise.
- Sandbox processes that bypass `HTTPS_PROXY` by using a raw socket. The proxy can't intercept what doesn't route to it. Node.js is partially mitigated via `NODE_OPTIONS=--use-openssl-ca` (see caveat above).
- Allowlisted-host data exfiltration. If `api.openai.com` is allowed, an agent could embed exfil data in a request body to that host. The daemon log captures the request happened but doesn't prevent it.
- Uncovered providers (Anthropic native, AWS Bedrock, Azure OpenAI, Gemini). Their env vars stay in the sandbox; if you enable them, those credentials bypass the proxy entirely. See [Uncovered providers](#uncovered-providers).
- iron-proxy in-memory secret zeroisation. The Go binary holds swapped-in real credentials in process memory; a core-dump or `/proc/<pid>/mem` read from a same-uid attacker would expose them. Out of scope for this layer.
## Failure modes
- **Binary not installed, `auto_install: true`** — first `hermes egress setup` or `hermes egress start` downloads it. SHA-256 verified against the upstream `checksums.txt`.
- **Binary not installed, `auto_install: false`** — `start` fails with a clear message pointing to manual install.
- **`enabled: true` but proxy not running** — with `enforce_on_docker: true` (default), Docker sandbox creation refuses to start with an explanatory error. With `enforce: false`, it falls back to direct outbound with real creds and logs a warning.
- **Port collision** — iron-proxy exits immediately; `hermes egress start` reports the last 20 log lines and fails with non-zero exit.
- **Upstream-host denied** — sandbox gets HTTP 403 from the proxy with a body explaining which host wasn't allowed. The agent sees the error and reports it.
- **Cloud metadata IP (169.254.169.254) requested** — refused by `upstream_deny_cidrs` regardless of allowlist.
- **Strict-tier uncovered provider env var set** — `hermes egress start` refuses with a list of the offending env vars and the `proxy.fail_on_uncovered_providers: false` escape hatch.
- **`docker_env` collides with a proxy-controlling var (enforce on)** — sandbox creation refuses with the names of the colliding keys.
- **BWS access token missing in `credential_source: bitwarden`** — `hermes egress start` refuses with `--no-bitwarden` as the recovery hint.
- **iron-proxy doesn't bind within 5 seconds** — process is killed, pidfile unlinked, error names the port + tail of `iron-proxy.log`.
- **Concurrent `hermes egress start` calls** — second call refuses with "another start in progress" if the first's daemon is up; otherwise the second unlinks the stale pidfile and proceeds.
## Troubleshooting
### "Refusing to start: BWS_ACCESS_TOKEN is not set"
You enabled `credential_source: bitwarden` but the access-token env var isn't in your shell. Either:
```bash
export BWS_ACCESS_TOKEN=# one-shot
hermes egress start
```
Or move it into `~/.hermes/.env`. Or switch back to env mode:
```bash
hermes egress setup --no-bitwarden
```
### "Refusing to start: provider env vars present that bypass the proxy"
You have `fail_on_uncovered_providers: true` AND one of `ANTHROPIC_API_KEY` / `AZURE_OPENAI_API_KEY` / `GEMINI_API_KEY` is set in your env. Either unset the offending var, or flip the config flag back to `false` (default) if you accept the isolation gap.
### "iron-proxy exited immediately"
Look at the last 20 lines of `~/.hermes/proxy/iron-proxy.log`. Common causes:
- Port already in use → change `proxy.tunnel_port` or kill whatever else owns 9090
- Invalid `proxy.yaml` → run `hermes egress setup` to regenerate
- CA cert / key permissions wrong → `chmod 0o600 ~/.hermes/proxy/ca.key`
### "iron-proxy did not bind 127.0.0.1:9090 within 5s"
The daemon started but never bound the listener. Usually means the binary is wedged or doing something expensive at startup. Check `~/.hermes/proxy/iron-proxy.log`. The orphan process is killed automatically and the pidfile cleaned up so you can just retry `hermes egress start`.
### Sandbox sees `HTTP 403` from the proxy
The agent inside the sandbox tried to hit a host that isn't in `proxy.extra_allowed_hosts`. The 403 body explains which host. If you want to allow it, add to your config:
```yaml
proxy:
extra_allowed_hosts:
- api.example.com
- "*.staging.example.com"
```
Then `hermes egress setup` (to regenerate `proxy.yaml`) and `hermes egress stop && hermes egress start`.
### Sandbox sees SSL verification errors
Either the CA isn't mounted in the sandbox (rare; the docker backend does this automatically when `proxy.enabled: true`), or your image's HTTP client is reading from a non-standard env var.
```bash
# Inside the sandbox:
cat /etc/ssl/certs/hermes-egress-ca.crt | head -1
# Should print: -----BEGIN CERTIFICATE-----
env | grep -E "^(REQUESTS|CURL|SSL|NODE).*CA"
# Should list all four CA-bundle env vars pointing at /etc/ssl/certs/hermes-egress-ca.crt
```
If the cert isn't there, check that `proxy.enabled: true` AND `hermes egress status` shows `Listening yes`. If the env vars are missing, the sandbox image might be running an entrypoint that strips them — check your `docker_env` config.
### Sandbox sees `HTTP 401` from upstreams
Two common causes:
1. **Token-clobber on re-setup.** You ran `hermes egress setup --rotate-tokens` (or rotated tokens some other way) and the running sandboxes still hold the old tokens. Restart the sandboxes.
2. **Bitwarden refresh failed silently.** Should not happen with the new fail-loud behavior, but if you have `proxy.allow_env_fallback: true` set, the daemon may have started with stale env values. Check the daemon's environment (`/proc/<iron-proxy-pid>/environ`) for the expected `OPENROUTER_API_KEY` etc.
### "Address in use" after the parent process died
The parent Hermes process died during `hermes egress start` (Ctrl-C during the listening probe, OOM, panic). The new fix-up logic writes the pidfile immediately after `Popen` so the orphan is recoverable:
```bash
hermes egress stop # finds the orphan via the pidfile, kills it
hermes egress start
```
If `hermes egress stop` says "iron-proxy was not running" but you can still see the daemon in `ps`, the pidfile got out of sync. Manual recovery:
```bash
pkill -TERM iron-proxy
rm -f ~/.hermes/proxy/iron-proxy.pid ~/.hermes/proxy/iron-proxy.nonce
hermes egress start
```
### Inspecting per-request behavior
On the pinned binary version (**v0.39**) both daemon-level events and per-request records land in `~/.hermes/proxy/iron-proxy.log`. The format is line-delimited JSON. Grep for a specific upstream:
```bash
grep '"upstream":"openrouter.ai"' ~/.hermes/proxy/iron-proxy.log | tail -20
```
Or watch in real-time:
```bash
tail -f ~/.hermes/proxy/iron-proxy.log | jq
```
When the pinned version moves to v0.40+ (which adds `log.audit_path`), per-request records will move to `~/.hermes/proxy/audit.log` and `iron-proxy.log` will hold only daemon-level events. The file at `audit.log` is pre-created today at `0o600` so any logrotate / monitoring tooling you wire to that path keeps working through the version bump without operator-side reconfig.
## Limitations (v1)
- Docker backend only. Modal, Daytona, and SSH wiring will follow in separate PRs.
- Only bearer-token providers (OpenRouter, OpenAI, Anthropic-via-OR, etc.) are wired through the `secrets` transform out of the box. Providers with custom auth (x-api-key, query params, signatures) bypass the proxy entirely — see [Uncovered providers](#uncovered-providers).
- No native Windows binary upstream. Run on Linux / macOS / WSL.
- The CA is a 10-year self-signed cert on first generation. Rotation requires `openssl genrsa ...` by hand (or wait for a follow-up that adds `hermes egress rotate-ca`).
- Token rotation does not auto-restart the daemon; after `--rotate-tokens` you must `hermes egress stop && hermes egress start` and then restart running sandboxes.
- iron-proxy in-memory secret zeroisation is upstream-controlled. Same-uid attackers with `/proc/<pid>/mem` read access can read swapped-in secrets from the daemon's memory.
- iron-proxy v0.39 only supports a **single bind per daemon** and combines daemon + per-request records into a single log stream. The integration is designed to upgrade cleanly: the moment upstream adds `proxy.http_listens` (plural) and `log.audit_path`, both wire in automatically without changing operator configs.
## See also
- Upstream project: [github.com/ironsh/iron-proxy](https://github.com/ironsh/iron-proxy)
- Upstream docs: [docs.iron.sh](https://docs.iron.sh/)
- Bitwarden integration: [`hermes secrets bitwarden`](../secrets/bitwarden)
- Hermes Docker terminal backend: [Docker](../docker)
- Developer / contributor reference: [Egress proxy internals](../../developer-guide/egress-internals)

View File

@@ -36,6 +36,15 @@ const sidebars: SidebarsConfig = {
'user-guide/secrets/bitwarden',
],
},
{
type: 'category',
label: 'Egress proxy',
collapsed: true,
items: [
'user-guide/egress/index',
'user-guide/egress/iron-proxy',
],
},
'user-guide/sessions',
'user-guide/profiles',
'user-guide/profile-distributions',
@@ -736,6 +745,7 @@ const sidebars: SidebarsConfig = {
'developer-guide/tools-runtime',
'developer-guide/acp-internals',
'developer-guide/cron-internals',
'developer-guide/egress-internals',
'developer-guide/trajectory-format',
],
},