mirror of
https://github.com/NousResearch/hermes-agent.git
synced 2026-05-03 09:17:09 +08:00
Broad drift audit against origin/main (b52b63396).
Reference pages (most user-visible drift):
- slash-commands: add /busy, /curator, /footer, /indicator, /redraw, /steer
that were missing; drop non-existent /terminal-setup; fix /q footnote
(resolves to /queue, not /quit); extend CLI-only list with all 24
CLI-only commands in the registry
- cli-commands: add dedicated sections for hermes curator / fallback /
hooks (new subcommands not previously documented); remove stale
hermes honcho standalone section (the plugin registers dynamically
via hermes memory); list curator/fallback/hooks in top-level table;
fix completion to include fish
- toolsets-reference: document the real 52-toolset count; split browser
vs browser-cdp; add discord / discord_admin / spotify / yuanbao;
correct hermes-cli tool count from 36 to 38; fix misleading claim
that hermes-homeassistant adds tools (it's identical to hermes-cli)
- tools-reference: bump tool count 55 -> 68; add 7 Spotify, 5 Yuanbao,
2 Discord toolsets; move browser_cdp/browser_dialog to their own
browser-cdp toolset section
- environment-variables: add 40+ user-facing HERMES_* vars that were
undocumented (--yolo, --accept-hooks, --ignore-*, inference model
override, agent/stream/checkpoint timeouts, OAuth trace, per-platform
batch tuning for Telegram/Discord/Matrix/Feishu/WeCom, cron knobs,
gateway restart/connect timeouts); dedupe the Cron Scheduler section;
replace stale QQ_SANDBOX with QQ_PORTAL_HOST
User-guide (top level):
- cli.md: compression preserves last 20 turns, not 4 (protect_last_n: 20)
- configuration.md: display.platforms is the canonical per-platform
override key; tool_progress_overrides is deprecated and auto-migrated
- profiles.md: model.default is the config key, not model.model
- sessions.md: CLI/TUI session IDs use 6-char hex, gateway uses 8
- checkpoints-and-rollback.md: destructive-command list now matches
_DESTRUCTIVE_PATTERNS (adds rmdir, cp, install, dd)
- docker.md: the container runs as non-root hermes (UID 10000) via
gosu; fix install command (uv pip); add missing --insecure on the
dashboard compose example (required for non-loopback bind)
- security.md: systemctl danger pattern also matches 'restart'
- index.md: built-in tool count 47 -> 68
- integrations/index.md: 6 STT providers, 8 memory providers
- integrations/providers.md: drop fictional dashscope/qwen aliases
Features:
- overview.md: 9 image models (not 8), 9 TTS providers (not 5),
8 memory providers (Supermemory was missing)
- tool-gateway.md: 9 image models
- tools.md: extend common-toolsets list with search / messaging /
spotify / discord / debugging / safe
- fallback-providers.md: add 6 real providers from PROVIDER_REGISTRY
(lmstudio, kimi-coding-cn, stepfun, alibaba-coding-plan,
tencent-tokenhub, azure-foundry)
- plugins.md: Available Hooks table now includes on_session_finalize,
on_session_reset, subagent_stop
- built-in-plugins.md: add the 7 bundled plugins the page didn't
mention (spotify, google_meet, three image_gen providers, two
dashboard examples)
- web-dashboard.md: add --insecure and --tui flags
- cron.md: hermes cron create takes positional schedule/prompt, not
flags
Messaging:
- telegram.md: TELEGRAM_WEBHOOK_SECRET is now REQUIRED when
TELEGRAM_WEBHOOK_URL is set (gateway refuses to start without it
per GHSA-3vpc-7q5r-276h). Biggest user-visible drift in the batch.
- discord.md: HERMES_DISCORD_TEXT_BATCH_SPLIT_DELAY_SECONDS default
is 2.0, not 0.1
- dingtalk.md: document DINGTALK_REQUIRE_MENTION /
FREE_RESPONSE_CHATS / MENTION_PATTERNS / HOME_CHANNEL /
ALLOW_ALL_USERS that the adapter supports
- bluebubbles.md: drop fictional BLUEBUBBLES_SEND_READ_RECEIPTS env
var; the setting lives in platforms.bluebubbles.extra only
- qqbot.md: drop dead QQ_SANDBOX; add real QQ_PORTAL_HOST and
QQ_GROUP_ALLOWED_USERS
- wecom-callback.md: replace 'hermes gateway start' (service-only)
with 'hermes gateway' for first-time setup
Developer-guide:
- architecture.md: refresh tool/toolset counts (61/52), terminal
backend count (7), line counts for run_agent.py (~13.7k), cli.py
(~11.5k), main.py (~10.4k), setup.py (~3.5k), gateway/run.py
(~12.2k), mcp_tool.py (~3.1k); add yuanbao adapter, bump platform
adapter count 18 -> 20
- agent-loop.md: run_agent.py line count 10.7k -> 13.7k
- tools-runtime.md: add vercel_sandbox backend
- adding-tools.md: remove stale 'Discovery import added to
model_tools.py' checklist item (registry auto-discovery)
- adding-platform-adapters.md: mark send_typing / get_chat_info as
concrete base methods; only connect/disconnect/send are abstract
- acp-internals.md: ACP sessions now persist to SessionDB
(~/.hermes/state.db); acp.run_agent call uses
use_unstable_protocol=True
- cron-internals.md: gateway runs scheduler in a dedicated background
thread via _start_cron_ticker, not on a maintenance cycle; locking
is cross-process via fcntl.flock (Unix) / msvcrt.locking (Windows)
- gateway-internals.md: gateway/run.py ~12k lines
- provider-runtime.md: cron DOES support fallback (run_job reads
fallback_providers from config)
- session-storage.md: SCHEMA_VERSION = 11 (not 9); add migrations
10 and 11 (trigram FTS, inline-mode FTS5 re-index); add
api_call_count column to Sessions DDL; document messages_fts_trigram
and state_meta in the architecture tree
- context-compression-and-caching.md: remove the obsolete 'context
pressure warnings' section (warnings were removed for causing
models to give up early)
- context-engine-plugin.md: compress() signature now includes
focus_topic param
- extending-the-cli.md: _build_tui_layout_children signature now
includes model_picker_widget; add to default layout
Also fixed three pre-existing broken links/anchors the build warned
about (docker.md -> api-server.md, yuanbao.md -> cron-jobs.md and
tips#background-tasks, nix-setup.md -> #container-aware-cli).
Regenerated per-skill pages via website/scripts/generate-skill-docs.py
so catalog tables and sidebar are consistent with current SKILL.md
frontmatter.
docusaurus build: clean, no broken links or anchors.
361 lines
10 KiB
Markdown
361 lines
10 KiB
Markdown
---
|
|
title: "Test Driven Development — TDD: enforce RED-GREEN-REFACTOR, tests before code"
|
|
sidebar_label: "Test Driven Development"
|
|
description: "TDD: enforce RED-GREEN-REFACTOR, tests before code"
|
|
---
|
|
|
|
{/* This page is auto-generated from the skill's SKILL.md by website/scripts/generate-skill-docs.py. Edit the source SKILL.md, not this page. */}
|
|
|
|
# Test Driven Development
|
|
|
|
TDD: enforce RED-GREEN-REFACTOR, tests before code.
|
|
|
|
## Skill metadata
|
|
|
|
| | |
|
|
|---|---|
|
|
| Source | Bundled (installed by default) |
|
|
| Path | `skills/software-development/test-driven-development` |
|
|
| Version | `1.1.0` |
|
|
| Author | Hermes Agent (adapted from obra/superpowers) |
|
|
| License | MIT |
|
|
| Tags | `testing`, `tdd`, `development`, `quality`, `red-green-refactor` |
|
|
| Related skills | [`systematic-debugging`](/docs/user-guide/skills/bundled/software-development/software-development-systematic-debugging), [`writing-plans`](/docs/user-guide/skills/bundled/software-development/software-development-writing-plans), [`subagent-driven-development`](/docs/user-guide/skills/bundled/software-development/software-development-subagent-driven-development) |
|
|
|
|
## Reference: full SKILL.md
|
|
|
|
:::info
|
|
The following is the complete skill definition that Hermes loads when this skill is triggered. This is what the agent sees as instructions when the skill is active.
|
|
:::
|
|
|
|
# Test-Driven Development (TDD)
|
|
|
|
## Overview
|
|
|
|
Write the test first. Watch it fail. Write minimal code to pass.
|
|
|
|
**Core principle:** If you didn't watch the test fail, you don't know if it tests the right thing.
|
|
|
|
**Violating the letter of the rules is violating the spirit of the rules.**
|
|
|
|
## When to Use
|
|
|
|
**Always:**
|
|
- New features
|
|
- Bug fixes
|
|
- Refactoring
|
|
- Behavior changes
|
|
|
|
**Exceptions (ask the user first):**
|
|
- Throwaway prototypes
|
|
- Generated code
|
|
- Configuration files
|
|
|
|
Thinking "skip TDD just this once"? Stop. That's rationalization.
|
|
|
|
## The Iron Law
|
|
|
|
```
|
|
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
|
|
```
|
|
|
|
Write code before the test? Delete it. Start over.
|
|
|
|
**No exceptions:**
|
|
- Don't keep it as "reference"
|
|
- Don't "adapt" it while writing tests
|
|
- Don't look at it
|
|
- Delete means delete
|
|
|
|
Implement fresh from tests. Period.
|
|
|
|
## Red-Green-Refactor Cycle
|
|
|
|
### RED — Write Failing Test
|
|
|
|
Write one minimal test showing what should happen.
|
|
|
|
**Good test:**
|
|
```python
|
|
def test_retries_failed_operations_3_times():
|
|
attempts = 0
|
|
def operation():
|
|
nonlocal attempts
|
|
attempts += 1
|
|
if attempts < 3:
|
|
raise Exception('fail')
|
|
return 'success'
|
|
|
|
result = retry_operation(operation)
|
|
|
|
assert result == 'success'
|
|
assert attempts == 3
|
|
```
|
|
Clear name, tests real behavior, one thing.
|
|
|
|
**Bad test:**
|
|
```python
|
|
def test_retry_works():
|
|
mock = MagicMock()
|
|
mock.side_effect = [Exception(), Exception(), 'success']
|
|
result = retry_operation(mock)
|
|
assert result == 'success' # What about retry count? Timing?
|
|
```
|
|
Vague name, tests mock not real code.
|
|
|
|
**Requirements:**
|
|
- One behavior per test
|
|
- Clear descriptive name ("and" in name? Split it)
|
|
- Real code, not mocks (unless truly unavoidable)
|
|
- Name describes behavior, not implementation
|
|
|
|
### Verify RED — Watch It Fail
|
|
|
|
**MANDATORY. Never skip.**
|
|
|
|
```bash
|
|
# Use terminal tool to run the specific test
|
|
pytest tests/test_feature.py::test_specific_behavior -v
|
|
```
|
|
|
|
Confirm:
|
|
- Test fails (not errors from typos)
|
|
- Failure message is expected
|
|
- Fails because the feature is missing
|
|
|
|
**Test passes immediately?** You're testing existing behavior. Fix the test.
|
|
|
|
**Test errors?** Fix the error, re-run until it fails correctly.
|
|
|
|
### GREEN — Minimal Code
|
|
|
|
Write the simplest code to pass the test. Nothing more.
|
|
|
|
**Good:**
|
|
```python
|
|
def add(a, b):
|
|
return a + b # Nothing extra
|
|
```
|
|
|
|
**Bad:**
|
|
```python
|
|
def add(a, b):
|
|
result = a + b
|
|
logging.info(f"Adding {a} + {b} = {result}") # Extra!
|
|
return result
|
|
```
|
|
|
|
Don't add features, refactor other code, or "improve" beyond the test.
|
|
|
|
**Cheating is OK in GREEN:**
|
|
- Hardcode return values
|
|
- Copy-paste
|
|
- Duplicate code
|
|
- Skip edge cases
|
|
|
|
We'll fix it in REFACTOR.
|
|
|
|
### Verify GREEN — Watch It Pass
|
|
|
|
**MANDATORY.**
|
|
|
|
```bash
|
|
# Run the specific test
|
|
pytest tests/test_feature.py::test_specific_behavior -v
|
|
|
|
# Then run ALL tests to check for regressions
|
|
pytest tests/ -q
|
|
```
|
|
|
|
Confirm:
|
|
- Test passes
|
|
- Other tests still pass
|
|
- Output pristine (no errors, warnings)
|
|
|
|
**Test fails?** Fix the code, not the test.
|
|
|
|
**Other tests fail?** Fix regressions now.
|
|
|
|
### REFACTOR — Clean Up
|
|
|
|
After green only:
|
|
- Remove duplication
|
|
- Improve names
|
|
- Extract helpers
|
|
- Simplify expressions
|
|
|
|
Keep tests green throughout. Don't add behavior.
|
|
|
|
**If tests fail during refactor:** Undo immediately. Take smaller steps.
|
|
|
|
### Repeat
|
|
|
|
Next failing test for next behavior. One cycle at a time.
|
|
|
|
## Why Order Matters
|
|
|
|
**"I'll write tests after to verify it works"**
|
|
|
|
Tests written after code pass immediately. Passing immediately proves nothing:
|
|
- Might test the wrong thing
|
|
- Might test implementation, not behavior
|
|
- Might miss edge cases you forgot
|
|
- You never saw it catch the bug
|
|
|
|
Test-first forces you to see the test fail, proving it actually tests something.
|
|
|
|
**"I already manually tested all the edge cases"**
|
|
|
|
Manual testing is ad-hoc. You think you tested everything but:
|
|
- No record of what you tested
|
|
- Can't re-run when code changes
|
|
- Easy to forget cases under pressure
|
|
- "It worked when I tried it" ≠ comprehensive
|
|
|
|
Automated tests are systematic. They run the same way every time.
|
|
|
|
**"Deleting X hours of work is wasteful"**
|
|
|
|
Sunk cost fallacy. The time is already gone. Your choice now:
|
|
- Delete and rewrite with TDD (high confidence)
|
|
- Keep it and add tests after (low confidence, likely bugs)
|
|
|
|
The "waste" is keeping code you can't trust.
|
|
|
|
**"TDD is dogmatic, being pragmatic means adapting"**
|
|
|
|
TDD IS pragmatic:
|
|
- Finds bugs before commit (faster than debugging after)
|
|
- Prevents regressions (tests catch breaks immediately)
|
|
- Documents behavior (tests show how to use code)
|
|
- Enables refactoring (change freely, tests catch breaks)
|
|
|
|
"Pragmatic" shortcuts = debugging in production = slower.
|
|
|
|
**"Tests after achieve the same goals — it's spirit not ritual"**
|
|
|
|
No. Tests-after answer "What does this do?" Tests-first answer "What should this do?"
|
|
|
|
Tests-after are biased by your implementation. You test what you built, not what's required. Tests-first force edge case discovery before implementing.
|
|
|
|
## Common Rationalizations
|
|
|
|
| Excuse | Reality |
|
|
|--------|---------|
|
|
| "Too simple to test" | Simple code breaks. Test takes 30 seconds. |
|
|
| "I'll test after" | Tests passing immediately prove nothing. |
|
|
| "Tests after achieve same goals" | Tests-after = "what does this do?" Tests-first = "what should this do?" |
|
|
| "Already manually tested" | Ad-hoc ≠ systematic. No record, can't re-run. |
|
|
| "Deleting X hours is wasteful" | Sunk cost fallacy. Keeping unverified code is technical debt. |
|
|
| "Keep as reference, write tests first" | You'll adapt it. That's testing after. Delete means delete. |
|
|
| "Need to explore first" | Fine. Throw away exploration, start with TDD. |
|
|
| "Test hard = design unclear" | Listen to the test. Hard to test = hard to use. |
|
|
| "TDD will slow me down" | TDD faster than debugging. Pragmatic = test-first. |
|
|
| "Manual test faster" | Manual doesn't prove edge cases. You'll re-test every change. |
|
|
| "Existing code has no tests" | You're improving it. Add tests for the code you touch. |
|
|
|
|
## Red Flags — STOP and Start Over
|
|
|
|
If you catch yourself doing any of these, delete the code and restart with TDD:
|
|
|
|
- Code before test
|
|
- Test after implementation
|
|
- Test passes immediately on first run
|
|
- Can't explain why test failed
|
|
- Tests added "later"
|
|
- Rationalizing "just this once"
|
|
- "I already manually tested it"
|
|
- "Tests after achieve the same purpose"
|
|
- "Keep as reference" or "adapt existing code"
|
|
- "Already spent X hours, deleting is wasteful"
|
|
- "TDD is dogmatic, I'm being pragmatic"
|
|
- "This is different because..."
|
|
|
|
**All of these mean: Delete code. Start over with TDD.**
|
|
|
|
## Verification Checklist
|
|
|
|
Before marking work complete:
|
|
|
|
- [ ] Every new function/method has a test
|
|
- [ ] Watched each test fail before implementing
|
|
- [ ] Each test failed for expected reason (feature missing, not typo)
|
|
- [ ] Wrote minimal code to pass each test
|
|
- [ ] All tests pass
|
|
- [ ] Output pristine (no errors, warnings)
|
|
- [ ] Tests use real code (mocks only if unavoidable)
|
|
- [ ] Edge cases and errors covered
|
|
|
|
Can't check all boxes? You skipped TDD. Start over.
|
|
|
|
## When Stuck
|
|
|
|
| Problem | Solution |
|
|
|---------|----------|
|
|
| Don't know how to test | Write the wished-for API. Write the assertion first. Ask the user. |
|
|
| Test too complicated | Design too complicated. Simplify the interface. |
|
|
| Must mock everything | Code too coupled. Use dependency injection. |
|
|
| Test setup huge | Extract helpers. Still complex? Simplify the design. |
|
|
|
|
## Hermes Agent Integration
|
|
|
|
### Running Tests
|
|
|
|
Use the `terminal` tool to run tests at each step:
|
|
|
|
```python
|
|
# RED — verify failure
|
|
terminal("pytest tests/test_feature.py::test_name -v")
|
|
|
|
# GREEN — verify pass
|
|
terminal("pytest tests/test_feature.py::test_name -v")
|
|
|
|
# Full suite — verify no regressions
|
|
terminal("pytest tests/ -q")
|
|
```
|
|
|
|
### With delegate_task
|
|
|
|
When dispatching subagents for implementation, enforce TDD in the goal:
|
|
|
|
```python
|
|
delegate_task(
|
|
goal="Implement [feature] using strict TDD",
|
|
context="""
|
|
Follow test-driven-development skill:
|
|
1. Write failing test FIRST
|
|
2. Run test to verify it fails
|
|
3. Write minimal code to pass
|
|
4. Run test to verify it passes
|
|
5. Refactor if needed
|
|
6. Commit
|
|
|
|
Project test command: pytest tests/ -q
|
|
Project structure: [describe relevant files]
|
|
""",
|
|
toolsets=['terminal', 'file']
|
|
)
|
|
```
|
|
|
|
### With systematic-debugging
|
|
|
|
Bug found? Write failing test reproducing it. Follow TDD cycle. The test proves the fix and prevents regression.
|
|
|
|
Never fix bugs without a test.
|
|
|
|
## Testing Anti-Patterns
|
|
|
|
- **Testing mock behavior instead of real behavior** — mocks should verify interactions, not replace the system under test
|
|
- **Testing implementation details** — test behavior/results, not internal method calls
|
|
- **Happy path only** — always test edge cases, errors, and boundaries
|
|
- **Brittle tests** — tests should verify behavior, not structure; refactoring shouldn't break them
|
|
|
|
## Final Rule
|
|
|
|
```
|
|
Production code → test exists and failed first
|
|
Otherwise → not TDD
|
|
```
|
|
|
|
No exceptions without the user's explicit permission.
|