Files
hermes-agent/tests/cron/test_cron_no_agent.py
Teknium 3db6b9cc87 feat(cron): add no_agent mode for script-only cron jobs (watchdog pattern) (#19709)
* feat(cron): add no_agent mode for script-only cron jobs (watchdog pattern)

Adds a no_agent=True option to the cronjob system. When enabled, the
scheduler runs the attached script on schedule and delivers its stdout
directly to the job's target — no LLM, no agent loop, no token spend.
This is the classic bash-watchdog pattern (memory alert every 5 min,
disk alert every 15 min, CI ping) reimplemented as a first-class Hermes
primitive instead of a systemd timer + curl + bot token triplet living
outside the system.

## What

  hermes cron create "every 5m" \
    --no-agent \
    --script memory-watchdog.sh \
    --deliver telegram \
    --name memory-watchdog

Agent tool:

  cronjob(action='create',
          schedule='every 5m',
          script='memory-watchdog.sh',
          no_agent=True,
          deliver='telegram')

Semantics:
- Script stdout (trimmed) → delivered verbatim as the message
- Empty stdout          → silent tick (no delivery; watchdog pattern)
- wakeAgent=false gate  → silent tick (same gate LLM jobs use)
- Non-zero exit/timeout → delivered as an error alert
                          (broken watchdogs shouldn't fail silently)
- No LLM ever invoked; no tokens spent; no provider fallback applied

## Implementation

cron/jobs.py
  * create_job gains no_agent: bool = False
  * prompt becomes Optional (no_agent jobs don't need one)
  * Validation: no_agent=True requires a script at create time
  * Field roundtrips via load_jobs / save_jobs / update_job

cron/scheduler.py
  * run_job: new short-circuit branch at the top that runs the script,
    wraps its output into the (success, doc, final_response, error)
    tuple downstream delivery already expects, and returns before any
    AIAgent import or construction
  * _run_job_script: picks interpreter by extension — .sh/.bash run
    under /bin/bash, anything else under sys.executable (Python).
    Shell support unlocks the bash-watchdog pattern without wrapping
    scripts in Python. Extension is explicit; we deliberately do NOT
    trust the file's own shebang. Path-containment guard (scripts dir)
    unchanged.

tools/cronjob_tools.py
  * Schema: new no_agent boolean property with clear trigger guidance
  * cronjob() accepts no_agent and validates mode-specific shape:
    - no_agent=True requires script; prompt/skills optional
    - no_agent=False keeps the existing 'prompt or skill required' rule
  * update path rejects flipping no_agent=True on a job without a script
  * _format_job surfaces no_agent in list output
  * Handler lambda forwards no_agent from tool args

hermes_cli/main.py, hermes_cli/cron.py
  * 'hermes cron create --no-agent' and edit's --no-agent / --agent
    pair for toggling at CLI parity with the agent tool
  * Existing --script help text updated to describe both modes
  * List / create / edit output now shows 'Mode: no-agent (...)' when set

## Tests

tests/cron/test_cron_no_agent.py — 18 tests covering:
  * create_job: no_agent shape, validation, field persistence
  * update_job: flag roundtrip across reload
  * cronjob tool: schema validation, update toggling, mode-specific
    requirements, prompt-relaxation rule
  * run_job short-circuit:
    - success path delivers stdout verbatim
    - empty stdout → SILENT_MARKER (no delivery downstream)
    - wakeAgent=false gate → silent
    - script failure → error alert
    - run_job does NOT import AIAgent (verified via mock)
  * _run_job_script:
    - .sh executes via bash (no shebang required)
    - .bash executes via bash
    - .py still runs via sys.executable (regression)
    - path-traversal still blocked (security regression)

All 18 new tests pass. 341/342 pre-existing cron tests still pass; the
one failure (test_script_empty_output_noted) was already broken on main
and is unrelated to this change.

## Docs

website/docs/guides/cron-script-only.md — new dedicated guide covering
the watchdog pattern, interpreter rules, delivery mapping, worked
examples (memory / disk alerts), and the comparison table vs hermes send,
regular LLM cron jobs, and OS-level cron.

website/docs/user-guide/features/cron.md — new 'No-agent mode' section
in the cron feature reference, cross-linked to the guide.

website/docs/guides/automate-with-cron.md — new tip box pointing users
to no-agent mode when they don't need LLM reasoning.

## Compatibility

- Existing jobs: unchanged. no_agent defaults to False, existing code
  paths untouched until the flag is set.
- Schema additive only; older jobs.json without the field load fine
  via .get() with False default.
- New CLI flags are opt-in and don't alter existing flag behavior.

* fix(cron): lazy-import AIAgent + SessionDB so no_agent ticks pay zero

The unconditional `from run_agent import AIAgent` + SessionDB() init at
the top of run_job() meant every no_agent tick still paid the full agent
module load cost (~300ms + transitive imports + DB open) even though it
never touched any of that machinery.

Move both to live under the default (LLM) path, after the no_agent
short-circuit has returned. Now a no_agent tick's sys.modules stays
clean — verified end-to-end:

    assert 'run_agent' not in sys.modules  # before
    run_job(no_agent_job)
    assert 'run_agent' not in sys.modules  # after

The existing mock-based unit test (test_run_job_no_agent_never_invokes_aiagent)
kept passing because patch() replaces the class AFTER import; the leak
was only visible via real subprocess-style verification. End-to-end
demo confirmed: agent calls cronjob(no_agent=True) → script runs →
stdout delivered → no LLM machinery loaded.

* docs(cron): tighten no_agent tool schema — defaults, silent semantics, pick rule

Previous description buried the important bits in one long sentence.
Agents could plausibly miss three things an LLM-facing schema should
make unmissable:

1. What the default is — now first sentence + JSON Schema `default: false`
2. What 'silent run' actually means for the user — now spelled out:
   'nothing is sent to the user and they won't see anything happened'
3. When to pick True vs False — now a concrete decision rule with
   examples on both sides (watchdogs/metrics/pollers → True;
   summarize/draft/pick/rephrase → False)

Also adds explicit 'prompt and skills are ignored when True' since the
agent could otherwise still pass them out of habit.

No behavior change — schema text only.
2026-05-04 12:31:01 -07:00

333 lines
11 KiB
Python

"""Tests for cronjob no_agent mode — script-driven jobs that skip the LLM.
Covers:
* ``create_job(no_agent=True)`` shape, validation, and serialization.
* ``cronjob(action='create', no_agent=True)`` tool-level validation.
* ``cronjob(action='update')`` flipping no_agent on/off.
* ``scheduler.run_job`` short-circuit path: success/silent/failure.
* Shell script support in ``_run_job_script`` (.sh runs via bash).
"""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import patch
import pytest
@pytest.fixture
def hermes_env(tmp_path, monkeypatch):
"""Isolate HERMES_HOME for each test so jobs/scripts don't leak."""
home = tmp_path / ".hermes"
home.mkdir()
(home / "scripts").mkdir()
(home / "cron").mkdir()
monkeypatch.setenv("HERMES_HOME", str(home))
# Reload modules that cache get_hermes_home() at import time.
import importlib
import hermes_constants
importlib.reload(hermes_constants)
import cron.jobs
importlib.reload(cron.jobs)
import cron.scheduler
importlib.reload(cron.scheduler)
return home
# ---------------------------------------------------------------------------
# create_job / update_job: data-layer semantics
# ---------------------------------------------------------------------------
def test_create_job_no_agent_requires_script(hermes_env):
from cron.jobs import create_job
with pytest.raises(ValueError, match="no_agent=True requires a script"):
create_job(prompt=None, schedule="every 5m", no_agent=True)
def test_create_job_no_agent_stores_field(hermes_env):
from cron.jobs import create_job
script_path = hermes_env / "scripts" / "watchdog.sh"
script_path.write_text("#!/bin/bash\necho hi\n")
job = create_job(
prompt=None,
schedule="every 5m",
script="watchdog.sh",
no_agent=True,
deliver="local",
)
assert job["no_agent"] is True
assert job["script"] == "watchdog.sh"
# Prompt can be empty/None for no_agent jobs.
assert job["prompt"] in (None, "")
def test_create_job_default_is_not_no_agent(hermes_env):
from cron.jobs import create_job
job = create_job(prompt="say hi", schedule="every 5m", deliver="local")
assert job.get("no_agent") is False
def test_update_job_roundtrips_no_agent_flag(hermes_env):
from cron.jobs import create_job, update_job, get_job
script_path = hermes_env / "scripts" / "w.sh"
script_path.write_text("echo hi\n")
job = create_job(prompt=None, schedule="every 5m", script="w.sh", no_agent=True, deliver="local")
update_job(job["id"], {"no_agent": False})
reloaded = get_job(job["id"])
assert reloaded["no_agent"] is False
update_job(job["id"], {"no_agent": True})
reloaded = get_job(job["id"])
assert reloaded["no_agent"] is True
# ---------------------------------------------------------------------------
# cronjob tool: API-layer validation
# ---------------------------------------------------------------------------
def test_cronjob_tool_create_no_agent_without_script_errors(hermes_env):
from tools.cronjob_tools import cronjob
result = json.loads(
cronjob(action="create", schedule="every 5m", no_agent=True, deliver="local")
)
assert result.get("success") is False
assert "no_agent=True requires a script" in result.get("error", "")
def test_cronjob_tool_create_no_agent_with_script_succeeds(hermes_env):
from tools.cronjob_tools import cronjob
script_path = hermes_env / "scripts" / "alert.sh"
script_path.write_text("#!/bin/bash\necho alert\n")
result = json.loads(
cronjob(
action="create",
schedule="every 5m",
script="alert.sh",
no_agent=True,
deliver="local",
)
)
assert result.get("success") is True
assert result["job"]["no_agent"] is True
assert result["job"]["script"] == "alert.sh"
def test_cronjob_tool_update_toggles_no_agent(hermes_env):
from tools.cronjob_tools import cronjob
script_path = hermes_env / "scripts" / "w.sh"
script_path.write_text("echo hi\n")
created = json.loads(
cronjob(
action="create",
schedule="every 5m",
script="w.sh",
no_agent=True,
deliver="local",
)
)
job_id = created["job_id"]
off = json.loads(cronjob(action="update", job_id=job_id, no_agent=False, prompt="run"))
assert off["success"] is True
assert off["job"].get("no_agent") in (False, None)
on = json.loads(cronjob(action="update", job_id=job_id, no_agent=True))
assert on["success"] is True
assert on["job"]["no_agent"] is True
def test_cronjob_tool_update_no_agent_without_script_errors(hermes_env):
"""Flipping no_agent=True on a job that has no script must fail."""
from tools.cronjob_tools import cronjob
created = json.loads(
cronjob(action="create", schedule="every 5m", prompt="do a thing", deliver="local")
)
job_id = created["job_id"]
result = json.loads(cronjob(action="update", job_id=job_id, no_agent=True))
assert result.get("success") is False
assert "without a script" in result.get("error", "")
def test_cronjob_tool_create_does_not_require_prompt_when_no_agent(hermes_env):
"""The 'prompt or skill required' rule is relaxed for no_agent jobs."""
from tools.cronjob_tools import cronjob
script_path = hermes_env / "scripts" / "w.sh"
script_path.write_text("echo hi\n")
result = json.loads(
cronjob(
action="create",
schedule="every 5m",
script="w.sh",
no_agent=True,
deliver="local",
)
)
assert result.get("success") is True
# ---------------------------------------------------------------------------
# scheduler.run_job: short-circuit behavior
# ---------------------------------------------------------------------------
def test_run_job_no_agent_success_returns_script_stdout(hermes_env):
"""Happy path: script exits 0 with output, delivered verbatim."""
from cron.jobs import create_job
from cron.scheduler import run_job
script_path = hermes_env / "scripts" / "alert.sh"
script_path.write_text("#!/bin/bash\necho 'RAM 92% on host'\n")
job = create_job(
prompt=None, schedule="every 5m", script="alert.sh", no_agent=True, deliver="local"
)
success, doc, final_response, error = run_job(job)
assert success is True
assert error is None
assert "RAM 92% on host" in final_response
assert "RAM 92% on host" in doc
def test_run_job_no_agent_empty_output_is_silent(hermes_env):
"""Empty stdout → SILENT_MARKER, which suppresses delivery downstream."""
from cron.jobs import create_job
from cron.scheduler import run_job, SILENT_MARKER
script_path = hermes_env / "scripts" / "quiet.sh"
script_path.write_text("#!/bin/bash\n# nothing to say\n")
job = create_job(
prompt=None, schedule="every 5m", script="quiet.sh", no_agent=True, deliver="local"
)
success, doc, final_response, error = run_job(job)
assert success is True
assert error is None
assert final_response == SILENT_MARKER
def test_run_job_no_agent_wake_gate_is_silent(hermes_env):
"""wakeAgent=false gate in stdout triggers a silent run."""
from cron.jobs import create_job
from cron.scheduler import run_job, SILENT_MARKER
script_path = hermes_env / "scripts" / "gated.sh"
script_path.write_text('#!/bin/bash\necho \'{"wakeAgent": false}\'\n')
job = create_job(
prompt=None, schedule="every 5m", script="gated.sh", no_agent=True, deliver="local"
)
success, doc, final_response, error = run_job(job)
assert success is True
assert final_response == SILENT_MARKER
def test_run_job_no_agent_script_failure_delivers_error(hermes_env):
"""Non-zero exit → success=False, error alert is the delivered message."""
from cron.jobs import create_job
from cron.scheduler import run_job
script_path = hermes_env / "scripts" / "broken.sh"
script_path.write_text("#!/bin/bash\necho oops >&2\nexit 3\n")
job = create_job(
prompt=None, schedule="every 5m", script="broken.sh", no_agent=True, deliver="local"
)
success, doc, final_response, error = run_job(job)
assert success is False
assert error is not None
assert "oops" in final_response or "exited with code 3" in final_response
assert "Cron watchdog" in final_response # alert header
def test_run_job_no_agent_never_invokes_aiagent(hermes_env):
"""no_agent jobs must NOT import/construct the AIAgent."""
from cron.jobs import create_job
script_path = hermes_env / "scripts" / "alert.sh"
script_path.write_text("#!/bin/bash\necho alert\n")
job = create_job(
prompt=None, schedule="every 5m", script="alert.sh", no_agent=True, deliver="local"
)
with patch("run_agent.AIAgent") as ai_mock:
from cron.scheduler import run_job
run_job(job)
ai_mock.assert_not_called()
# ---------------------------------------------------------------------------
# _run_job_script: shell-script support
# ---------------------------------------------------------------------------
def test_run_job_script_shell_script_runs_via_bash(hermes_env):
""".sh files should execute under /bin/bash even without a shebang line."""
from cron.scheduler import _run_job_script
script_path = hermes_env / "scripts" / "shelly.sh"
# No shebang — relies on the interpreter-by-extension rule.
script_path.write_text('echo "shell: $BASH_VERSION" | head -c 7\n')
ok, output = _run_job_script("shelly.sh")
assert ok is True
assert output.startswith("shell:")
def test_run_job_script_bash_extension_also_runs_via_bash(hermes_env):
from cron.scheduler import _run_job_script
script_path = hermes_env / "scripts" / "thing.bash"
script_path.write_text('printf "via bash\\n"\n')
ok, output = _run_job_script("thing.bash")
assert ok is True
assert output == "via bash"
def test_run_job_script_python_still_runs_via_python(hermes_env):
"""Regression: .py files must keep running via sys.executable."""
from cron.scheduler import _run_job_script
script_path = hermes_env / "scripts" / "py.py"
script_path.write_text("import sys\nprint(f'python {sys.version_info.major}')\n")
ok, output = _run_job_script("py.py")
assert ok is True
assert output.startswith("python ")
def test_run_job_script_path_traversal_still_blocked(hermes_env):
"""Security regression: shell-script support must NOT loosen containment."""
from cron.scheduler import _run_job_script
# Absolute path outside the scripts dir should be rejected.
ok, output = _run_job_script("/etc/passwd")
assert ok is False
assert "Blocked" in output or "outside" in output