Files
hermes-agent/website/docs/user-guide/features/built-in-plugins.md
Teknium 447d800b81 docs: add observability/langfuse to built-in-plugins + env-vars reference (#16929)
Documents the langfuse plugin shipped in #16917:
- website/docs/user-guide/features/built-in-plugins.md: new
  observability/langfuse section (setup wizard vs manual, hook-by-hook
  behaviour, verify / optional tuning / disable)
- website/docs/reference/environment-variables.md: Langfuse Observability
  subsection under Tool APIs listing the 3 required + 5 optional env vars,
  with a back-link to the built-in-plugins page

Validated: ascii-guard clean, npm run build succeeds, #observabilitylangfuse
anchor resolves.

Co-authored-by: teknium1 <teknium@users.noreply.github.com>
2026-04-28 01:57:52 -07:00

8.9 KiB
Raw Blame History

sidebar_position, sidebar_label, title, description
sidebar_position sidebar_label title description
12 Built-in Plugins Built-in Plugins Plugins shipped with Hermes Agent that run automatically via lifecycle hooks — disk-cleanup and friends

Built-in Plugins

Hermes ships a small set of plugins bundled with the repository. They live under <repo>/plugins/<name>/ and load automatically alongside user-installed plugins in ~/.hermes/plugins/. They use the same plugin surface as third-party plugins — hooks, tools, slash commands — just maintained in-tree.

See the Plugins page for the general plugin system, and Build a Hermes Plugin to write your own.

How discovery works

The PluginManager scans four sources, in order:

  1. Bundled<repo>/plugins/<name>/ (what this page documents)
  2. User~/.hermes/plugins/<name>/
  3. Project./.hermes/plugins/<name>/ (requires HERMES_ENABLE_PROJECT_PLUGINS=1)
  4. Pip entry pointshermes_agent.plugins

On name collision, later sources win — a user plugin named disk-cleanup would replace the bundled one.

plugins/memory/ and plugins/context_engine/ are deliberately excluded from bundled scanning. Those directories use their own discovery paths because memory providers and context engines are single-select providers configured through hermes memory setup / context.engine in config.

Bundled plugins are opt-in

Bundled plugins ship disabled. Discovery finds them (they appear in hermes plugins list and the interactive hermes plugins UI), but none load until you explicitly enable them:

hermes plugins enable disk-cleanup

Or via ~/.hermes/config.yaml:

plugins:
  enabled:
    - disk-cleanup

This is the same mechanism user-installed plugins use. Bundled plugins are never auto-enabled — not on fresh install, not for existing users upgrading to a newer Hermes. You always opt in explicitly.

To turn a bundled plugin off again:

hermes plugins disable disk-cleanup
# or: remove it from plugins.enabled in config.yaml

Currently shipped

disk-cleanup

Auto-tracks and removes ephemeral files created during sessions — test scripts, temp outputs, cron logs, stale chrome profiles — without requiring the agent to remember to call a tool.

How it works:

Hook Behaviour
post_tool_call When write_file / terminal / patch creates a file matching test_*, tmp_*, or *.test.* inside HERMES_HOME or /tmp/hermes-*, track it silently as test / temp / cron-output.
on_session_end If any test files were auto-tracked during the turn, run the safe quick cleanup and log a one-line summary. Stays silent otherwise.

Deletion rules:

Category Threshold Confirmation
test every session end Never
temp >7 days since tracked Never
cron-output >14 days since tracked Never
empty dirs under HERMES_HOME always Never
research >30 days, beyond 10 newest Always (deep only)
chrome-profile >14 days since tracked Always (deep only)
files >500 MB never auto Always (deep only)

Slash command/disk-cleanup available in both CLI and gateway sessions:

/disk-cleanup status                     # breakdown + top-10 largest
/disk-cleanup dry-run                    # preview without deleting
/disk-cleanup quick                      # run safe cleanup now
/disk-cleanup deep                       # quick + list items needing confirmation
/disk-cleanup track <path> <category>    # manual tracking
/disk-cleanup forget <path>              # stop tracking (does not delete)

State — everything lives at $HERMES_HOME/disk-cleanup/:

File Contents
tracked.json Tracked paths with category, size, and timestamp
tracked.json.bak Atomic-write backup of the above
cleanup.log Append-only audit trail of every track / skip / reject / delete

Safety — cleanup only ever touches paths under HERMES_HOME or /tmp/hermes-*. Windows mounts (/mnt/c/...) are rejected. Well-known top-level state dirs (logs/, memories/, sessions/, cron/, cache/, skills/, plugins/, disk-cleanup/ itself) are never removed even when empty — a fresh install does not get gutted on first session end.

Enabling: hermes plugins enable disk-cleanup (or check the box in hermes plugins).

Disabling again: hermes plugins disable disk-cleanup.

observability/langfuse

Traces Hermes turns, LLM calls, and tool invocations to Langfuse — an open-source LLM observability platform. One span per turn, one generation per API call, one tool observation per tool call. Usage totals, per-type token counts, and cost estimates come out of Hermes' canonical agent.usage_pricing numbers, so the Langfuse dashboard sees the same breakdown (input / output / cache_read_input_tokens / cache_creation_input_tokens / reasoning_tokens) that appears in hermes logs.

The plugin is fail-open: no SDK installed, no credentials, or a transient Langfuse error — all turn into a silent no-op in the hook. The agent loop is never impacted.

Setup (interactive — recommended):

hermes tools          # → Langfuse Observability → Cloud or Self-Hosted

The wizard collects your keys, pip installs the langfuse SDK, and adds observability/langfuse to plugins.enabled for you. Restart Hermes and the next turn ships a trace.

Setup (manual):

pip install langfuse
hermes plugins enable observability/langfuse

Then put the credentials in ~/.hermes/.env:

HERMES_LANGFUSE_PUBLIC_KEY=pk-lf-...
HERMES_LANGFUSE_SECRET_KEY=sk-lf-...
HERMES_LANGFUSE_BASE_URL=https://cloud.langfuse.com   # or your self-hosted URL

How it works:

Hook Behaviour
pre_api_request / pre_llm_call Open (or reuse) a per-turn root span "Hermes turn". Start a generation child observation for this API call with serialized recent messages as input.
post_api_request / post_llm_call Close the generation, attach usage_details, cost_details, finish_reason, assistant output + tool calls. If no tool calls and non-empty content, close the turn.
pre_tool_call Start a tool child observation with sanitized args.
post_tool_call Close the tool observation with sanitized result. read_file payloads get summarized (head + tail + omitted-line count) so a huge file read stays under HERMES_LANGFUSE_MAX_CHARS.

Session grouping keys off the Hermes session ID (or task ID for sub-agents) via langfuse.propagate_attributes, so everything in a single hermes chat session lives under one Langfuse session.

Verify:

hermes plugins list                 # observability/langfuse should show "enabled"
hermes chat -q "hello"              # check the Langfuse UI for a "Hermes turn" trace

Optional tuning (in .env):

Variable Default Purpose
HERMES_LANGFUSE_ENV Environment tag on traces (production, staging, …)
HERMES_LANGFUSE_RELEASE Release/version tag
HERMES_LANGFUSE_SAMPLE_RATE 1.0 Sampling rate passed to the SDK (0.01.0)
HERMES_LANGFUSE_MAX_CHARS 12000 Per-field truncation for message content / tool args / tool results
HERMES_LANGFUSE_DEBUG false Verbose plugin logging to agent.log

Hermes-prefixed and standard SDK env vars (LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_BASE_URL) are both accepted — Hermes-prefixed wins when both are set.

Performance: the Langfuse client is cached after the first hook call. If credentials or SDK are missing, that decision is also cached — subsequent hooks fast-return without re-checking env vars or reloading config.

Disabling: hermes plugins disable observability/langfuse. The plugin module is still discovered, but no module code runs until you re-enable.

Adding a bundled plugin

Bundled plugins are written exactly like any other Hermes plugin — see Build a Hermes Plugin. The only differences are:

  • Directory lives at <repo>/plugins/<name>/ instead of ~/.hermes/plugins/<name>/
  • Manifest source is reported as bundled in hermes plugins list
  • User plugins with the same name override the bundled version

A plugin is a good candidate for bundling when:

  • It has no optional dependencies (or they're already pip install .[all] deps)
  • The behaviour benefits most users and is opt-out rather than opt-in
  • The logic ties into lifecycle hooks that the agent would otherwise have to remember to invoke
  • It complements a core capability without expanding the model-visible tool surface

Counter-examples — things that should stay as user-installable plugins, not bundled: third-party integrations with API keys, niche workflows, large dependency trees, anything that would meaningfully change agent behaviour by default.