diff --git a/skills/creative/comfyui/SKILL.md b/skills/creative/comfyui/SKILL.md index 91975bfbcb7..e488e2c581e 100644 --- a/skills/creative/comfyui/SKILL.md +++ b/skills/creative/comfyui/SKILL.md @@ -1,9 +1,10 @@ --- name: comfyui -description: "Generate images, video, and audio with ComfyUI — install, launch, manage nodes/models, run workflows with parameter injection. Uses the official comfy-cli for lifecycle and direct REST API for execution." -version: 4.1.0 -requires: ComfyUI (local or Comfy Cloud); comfy-cli (pip install comfy-cli) -author: [kshitijk4poor, alt-glitch] +description: "Generate images, video, and audio with ComfyUI — install, launch, manage nodes/models, run workflows with parameter injection. Uses the official comfy-cli for lifecycle and direct REST/WebSocket API for execution." +version: 5.0.0 +requires_runtime: ComfyUI (local, Comfy Desktop, or Comfy Cloud) +requires_tooling: comfy-cli (auto-installed via pipx/uvx by setup script) +author: [kshitijk4poor, alt-glitch, audit-v5] license: MIT platforms: [macos, linux, windows] prerequisites: @@ -17,6 +18,9 @@ metadata: - image-generation - stable-diffusion - flux + - sd3 + - wan-video + - hunyuan-video - creative - generative-ai - video-generation @@ -26,325 +30,331 @@ metadata: # ComfyUI -Generate images, video, and audio through ComfyUI using the official `comfy-cli` for -setup/management and direct REST API calls for workflow execution. +Generate images, video, audio, and 3D content through ComfyUI using the +official `comfy-cli` for setup/lifecycle and direct REST/WebSocket API +for workflow execution. -**Reference files in this skill:** +## What's in this skill -- `references/official-cli.md` — comfy-cli command reference (install, launch, nodes, models) -- `references/rest-api.md` — ComfyUI REST API endpoints (local + cloud) -- `references/workflow-format.md` — workflow JSON format, common node types, parameter mapping +**Reference docs (`references/`):** -**Scripts in this skill:** +- `official-cli.md` — every `comfy ...` command, with flags +- `rest-api.md` — REST + WebSocket endpoints (local + cloud), payload schemas +- `workflow-format.md` — API-format JSON, common node types, param mapping -- `scripts/hardware_check.py` — detect GPU/VRAM/Apple Silicon, decide local vs Comfy Cloud -- `scripts/comfyui_setup.sh` — full setup automation (hardware check + install + launch + verify) -- `scripts/extract_schema.py` — reads workflow JSON, outputs which parameters are controllable -- `scripts/run_workflow.py` — injects user args, submits workflow, monitors progress, downloads outputs -- `scripts/check_deps.py` — checks if required custom nodes and models are installed +**Scripts (`scripts/`):** + +| Script | Purpose | +|--------|---------| +| `_common.py` | Shared HTTP, cloud routing, node catalogs (don't run directly) | +| `hardware_check.py` | Probe GPU/VRAM/disk → recommend local vs Comfy Cloud | +| `comfyui_setup.sh` | Hardware check + comfy-cli + ComfyUI install + launch + verify | +| `extract_schema.py` | Read a workflow → list controllable params + model deps | +| `check_deps.py` | Check workflow against running server → list missing nodes/models | +| `auto_fix_deps.py` | Run check_deps then `comfy node install` / `comfy model download` | +| `run_workflow.py` | Inject params, submit, monitor, download outputs (HTTP or WS) | +| `run_batch.py` | Submit a workflow N times with sweeps, parallel up to your tier | +| `ws_monitor.py` | Real-time WebSocket viewer for executing jobs (live progress) | +| `health_check.py` | Verification checklist runner — comfy-cli + server + models + smoke test | +| `fetch_logs.py` | Pull traceback / status messages for a given prompt_id | + +**Example workflows (`workflows/`):** SD 1.5, SDXL, Flux Dev, SDXL img2img, +SDXL inpaint, ESRGAN upscale, AnimateDiff video, Wan T2V. See +`workflows/README.md`. ## When to Use -- User asks to generate images with Stable Diffusion, SDXL, Flux, or other diffusion models -- User wants to run a specific ComfyUI workflow +- User asks to generate images with Stable Diffusion, SDXL, Flux, SD3, etc. +- User wants to run a specific ComfyUI workflow file - User wants to chain generative steps (txt2img → upscale → face restore) - User needs ControlNet, inpainting, img2img, or other advanced pipelines - User asks to manage ComfyUI queue, check models, or install custom nodes -- User wants video/audio generation via AnimateDiff, Hunyuan, AudioCraft, etc. +- User wants video/audio/3D generation via AnimateDiff, Hunyuan, Wan, AudioCraft, etc. ## Architecture: Two Layers ``` ┌─────────────────────────────────────────────────────┐ -│ Layer 1: comfy-cli (official) │ -│ Setup, lifecycle, nodes, models │ -│ comfy install / launch / stop / node / model │ +│ Layer 1: comfy-cli (official lifecycle tool) │ +│ Setup, server lifecycle, custom nodes, models │ +│ → comfy install / launch / stop / node / model │ └─────────────────────────┬───────────────────────────┘ │ ┌─────────────────────────▼───────────────────────────┐ -│ Layer 2: REST API + skill scripts │ +│ Layer 2: REST/WebSocket API + skill scripts │ │ Workflow execution, param injection, monitoring │ -│ POST /api/prompt, GET /api/view, WebSocket │ -│ scripts/run_workflow.py, extract_schema.py │ +│ POST /api/prompt, GET /api/view, WS /ws │ +│ → run_workflow.py, run_batch.py, ws_monitor.py │ └─────────────────────────────────────────────────────┘ ``` -**Why two layers?** The official CLI handles installation and server management excellently -but has minimal workflow execution support (just raw file submission, no param injection, -no structured output). The REST API fills that gap — the scripts in this skill handle the -param injection, execution monitoring, and output download that the CLI doesn't do. +**Why two layers?** The official CLI is excellent for installation and server +management but has minimal workflow execution support. The REST/WS API fills +that gap — the scripts handle param injection, execution monitoring, and +output download that the CLI doesn't do. ## Quick Start -### Detect Environment +### Detect environment ```bash # What's available? command -v comfy >/dev/null 2>&1 && echo "comfy-cli: installed" curl -s http://127.0.0.1:8188/system_stats 2>/dev/null && echo "server: running" -# Can this machine actually run ComfyUI locally? (GPU/VRAM/Apple Silicon check) +# Can this machine run ComfyUI locally? (GPU/VRAM/disk check) python3 scripts/hardware_check.py ``` -If nothing is installed, go to **Setup & Onboarding** below — but always run the -hardware check first, before picking an install path. -If the server is already running, skip to **Core Workflow**. +If nothing is installed, see **Setup & Onboarding** below — but always run the +hardware check first. + +### One-line health check + +```bash +python3 scripts/health_check.py +# → JSON: comfy_cli on PATH? server reachable? at least one checkpoint? smoke-test passes? +``` ## Core Workflow -### Step 1: Get a Workflow +### Step 1: Get a workflow JSON in API format -Users provide workflow JSON files. These come from: -- ComfyUI web editor → "Save (API Format)" button -- Community downloads (civitai, Reddit, Discord) -- The `scripts/` directory of this skill (example workflows) +Workflows must be in API format (each node has `class_type`). They come from: -**The workflow must be in API format** (node IDs as keys with `class_type`). -If user has editor format (has `nodes[]` and `links[]` at top level), they -need to re-export using "Save (API Format)" in the ComfyUI web editor. +- ComfyUI web UI → **Workflow → Export (API)** (newer UI) or + the legacy "Save (API Format)" button (older UI) +- This skill's `workflows/` directory (ready-to-run examples) +- Community downloads (civitai, Reddit, Discord) — usually editor format, + must be loaded into ComfyUI then re-exported -### Step 2: Understand What's Controllable +Editor format (top-level `nodes` and `links` arrays) is **not directly +executable**. The scripts detect this and tell you to re-export. + +### Step 2: See what's controllable ```bash +python3 scripts/extract_schema.py workflow_api.json --summary-only +# → {"parameter_count": 12, "has_negative_prompt": true, "has_seed": true, ...} + python3 scripts/extract_schema.py workflow_api.json +# → full schema with parameters, model deps, embedding refs ``` -Output (JSON): -```json -{ - "parameters": { - "prompt": {"node_id": "6", "field": "text", "type": "string", "value": "a cat"}, - "negative_prompt": {"node_id": "7", "field": "text", "type": "string", "value": "bad quality"}, - "seed": {"node_id": "3", "field": "seed", "type": "int", "value": 42}, - "steps": {"node_id": "3", "field": "steps", "type": "int", "value": 20}, - "width": {"node_id": "5", "field": "width", "type": "int", "value": 512}, - "height": {"node_id": "5", "field": "height", "type": "int", "value": 512} - } -} -``` +### Step 3: Run with parameters -### Step 3: Run with Parameters - -**Local:** ```bash +# Local (defaults to http://127.0.0.1:8188) python3 scripts/run_workflow.py \ --workflow workflow_api.json \ - --args '{"prompt": "a beautiful sunset over mountains", "seed": 123, "steps": 30}' \ + --args '{"prompt": "a beautiful sunset over mountains", "seed": -1, "steps": 30}' \ --output-dir ./outputs -``` -**Cloud:** -```bash +# Cloud (export API key once; uses correct /api routing automatically) +export COMFY_CLOUD_API_KEY="comfyui-..." python3 scripts/run_workflow.py \ --workflow workflow_api.json \ - --args '{"prompt": "a beautiful sunset", "seed": 123}' \ + --args '{"prompt": "..."}' \ --host https://cloud.comfy.org \ - --api-key "$COMFY_CLOUD_API_KEY" \ --output-dir ./outputs + +# Real-time progress via WebSocket (requires `pip install websocket-client`) +python3 scripts/run_workflow.py \ + --workflow flux_dev.json \ + --args '{"prompt": "..."}' \ + --ws + +# img2img / inpaint: pass --input-image to upload + reference automatically +python3 scripts/run_workflow.py \ + --workflow sdxl_img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it watercolor", "denoise": 0.6}' + +# Batch / sweep: 8 random seeds, parallel up to cloud tier limit +python3 scripts/run_batch.py \ + --workflow sdxl.json \ + --args '{"prompt": "abstract"}' \ + --count 8 --randomize-seed --parallel 3 \ + --output-dir ./outputs/batch ``` -### Step 4: Present Results +`-1` for `seed` (or omitting it with `--randomize-seed`) generates a fresh +random seed per run. + +### Step 4: Present results + +The scripts emit JSON to stdout describing every output file: -The script outputs JSON with file paths: ```json { "status": "success", + "prompt_id": "abc-123", "outputs": [ - {"file": "./outputs/ComfyUI_00001_.png", "node_id": "9", "type": "image"} + {"file": "./outputs/sdxl_00001_.png", "node_id": "9", + "type": "image", "filename": "sdxl_00001_.png"} ] } ``` -Show images to the user via `vision_analyze` or return the file path directly. - ## Decision Tree | User says | Tool | Command | |-----------|------|---------| -| "install ComfyUI" | comfy-cli | `comfy install` | +| **Lifecycle (use comfy-cli)** | | | +| "install ComfyUI" | comfy-cli | `bash scripts/comfyui_setup.sh` | | "start ComfyUI" | comfy-cli | `comfy launch --background` | | "stop ComfyUI" | comfy-cli | `comfy stop` | | "install X node" | comfy-cli | `comfy node install ` | -| "download X model" | comfy-cli | `comfy model download --url ` | +| "download X model" | comfy-cli | `comfy model download --url --relative-path models/checkpoints` | | "list installed models" | comfy-cli | `comfy model list` | | "list installed nodes" | comfy-cli | `comfy node show installed` | -| "generate an image" | script | `run_workflow.py --args '{"prompt": "..."}'` | -| "use this image" (img2img) | REST | upload image, then run_workflow.py | -| "what can I change in this workflow?" | script | `extract_schema.py workflow.json` | -| "check if workflow deps are met" | script | `check_deps.py workflow.json` | -| "what's in the queue?" | REST | `curl http://HOST:8188/queue` | +| **Execution (use scripts)** | | | +| "is everything ready?" | script | `health_check.py` (optionally with `--workflow X --smoke-test`) | +| "what can I change in this workflow?" | script | `extract_schema.py W.json` | +| "check if W's deps are met" | script | `check_deps.py W.json` | +| "fix missing deps" | script | `auto_fix_deps.py W.json` | +| "generate an image" | script | `run_workflow.py --workflow W --args '{...}'` | +| "use this image" (img2img) | script | `run_workflow.py --input-image image=./x.png ...` | +| "8 variations with random seeds" | script | `run_batch.py --count 8 --randomize-seed ...` | +| "show me live progress" | script | `ws_monitor.py --prompt-id ` | +| "fetch the error from job X" | script | `fetch_logs.py ` | +| **Direct REST** | | | +| "what's in the queue?" | REST | `curl http://HOST:8188/queue` (local) or `--host https://cloud.comfy.org` | | "cancel that" | REST | `curl -X POST http://HOST:8188/interrupt` | | "free GPU memory" | REST | `curl -X POST http://HOST:8188/free` | ## Setup & Onboarding -When a user asks to set up ComfyUI, the FIRST thing to do is ask them whether -they want **Comfy Cloud** (hosted, zero install, API key) or **Local** (install -ComfyUI on their machine). Do NOT start running install commands or hardware +When a user asks to set up ComfyUI, **the FIRST thing to do is ask whether +they want Comfy Cloud (hosted, zero install, API key) or Local (install +ComfyUI on their machine)**. Don't start running install commands or hardware checks until they've answered. **Official docs:** https://docs.comfy.org/installation **CLI docs:** https://docs.comfy.org/comfy-cli/getting-started **Cloud docs:** https://docs.comfy.org/get_started/cloud +**Cloud API:** https://docs.comfy.org/development/cloud/overview ### Step 0: Ask Local vs Cloud (ALWAYS FIRST) -Present the tradeoff clearly and wait for the user to choose. Suggested script: +Suggested script: > "Do you want to run ComfyUI locally on your machine, or use Comfy Cloud? > -> - **Comfy Cloud** — hosted on RTX 6000 Pro GPUs, all models pre-installed, zero setup. Requires an API key (paid subscription). Best if you don't have a capable GPU or want to skip installation. +> - **Comfy Cloud** — hosted on RTX 6000 Pro GPUs, all common models pre-installed, +> zero setup. Requires an API key (paid subscription required to actually run +> workflows; free tier is read-only). Best if you don't have a capable GPU. > - **Local** — free, but your machine MUST meet the hardware requirements: -> - NVIDIA GPU with **≥6 GB VRAM** (≥8 GB recommended for SDXL, ≥12 GB for Flux/video), OR +> - NVIDIA GPU with **≥6 GB VRAM** (≥8 GB for SDXL, ≥12 GB for Flux/video), OR > - AMD GPU with ROCm support (Linux), OR -> - Apple Silicon Mac (M1 or newer) with **≥16 GB unified memory** (≥32 GB recommended). +> - Apple Silicon Mac (M1+) with **≥16 GB unified memory** (≥32 GB recommended). > - Intel Macs and machines with no GPU will NOT work — use Cloud instead. > > Which would you like?" -Route based on their answer: +Routing: -- **User picks Cloud** → skip to **Path A** (no hardware check needed). -- **User picks Local** → go to **Step 1: Hardware Check** to verify their machine actually meets the requirements, then pick an install path from Paths B-E based on the verdict. -- **User is unsure / asks for a recommendation** → run the hardware check anyway and let the verdict decide. +- **Cloud** → skip to **Path A**. +- **Local** → run hardware check first, then pick a path from Paths B–E based on the verdict. +- **Unsure** → run the hardware check and let the verdict decide. ### Step 1: Verify Hardware (ONLY if user chose local) ```bash python3 scripts/hardware_check.py --json +# Optional: also probe `torch` for actual CUDA/MPS: +python3 scripts/hardware_check.py --json --check-pytorch ``` -It detects OS, GPU (NVIDIA CUDA / AMD ROCm / Apple Silicon / Intel Arc), VRAM, -and unified/system RAM, then returns a verdict plus a suggested `comfy-cli` flag: +| Verdict | Meaning | Action | +|------------|---------------------------------------------------------------|--------| +| `ok` | ≥8 GB VRAM (discrete) OR ≥32 GB unified (Apple Silicon) | Local install — use `comfy_cli_flag` from report | +| `marginal` | SD1.5 works; SDXL tight; Flux/video unlikely | Local OK for light workflows, else **Path A (Cloud)** | +| `cloud` | No usable GPU, <6 GB VRAM, <16 GB Apple unified, Intel Mac, Rosetta Python | **Switch to Cloud** unless user explicitly forces local | -| Verdict | Meaning | Action | -|------------|-----------------------------------------------------------|-------------------------------------------------| -| `ok` | ≥8 GB VRAM (discrete) OR ≥32 GB unified (Apple Silicon) | Local install — use `comfy_cli_flag` from report | -| `marginal` | SD1.5 works; SDXL tight; Flux/video unlikely | Local OK for light workflows, else **Path A (Cloud)** | -| `cloud` | No usable GPU, <6 GB VRAM, <16 GB Apple unified, Intel Mac | **User chose local but their machine doesn't meet requirements** — surface the `notes` and ask if they want to switch to Cloud | +The script also surfaces `wsl: true` (WSL2 with NVIDIA passthrough) and +`rosetta: true` (x86_64 Python on Apple Silicon — must reinstall as ARM64). -Hardware thresholds the skill enforces: - -- **Discrete GPU minimum:** 6 GB VRAM. Below that, most modern models won't load. -- **Apple Silicon:** M1 or newer (ARM64). Intel Macs have no MPS backend — Cloud only. -- **Apple Silicon memory:** 16 GB unified minimum. 8 GB M1/M2 will swap/OOM on SDXL/Flux. -- **No accelerator at all:** CPU-only is listed as a comfy-cli option but a single SDXL - image takes 10+ minutes — treat it as unusable and route to Cloud. - -If verdict is `cloud` but the user explicitly wanted local, DO NOT proceed -silently. Show the `notes` array verbatim, explain which requirement they -don't meet, and ask whether they want to (a) switch to Cloud or (b) force -a local install anyway (marginal/cloud-verdict local installs will OOM or -be unusably slow on modern models). - -The report's `comfy_cli_flag` field gives you the exact flag for Step 2 below: -`--nvidia`, `--amd`, or `--m-series`. For Intel Arc, use Path E (manual install). - -Surface the `notes` array verbatim to the user so they understand why a -particular path was recommended. +If verdict is `cloud` but the user wants local, do not proceed silently. +Show the `notes` array verbatim and ask whether they want to (a) switch to +Cloud or (b) force a local install (will OOM or be unusably slow on modern models). ### Choosing an Installation Path -Use the hardware check result first. The table below is a fallback for when the user -has already told you their hardware or you need to narrow down between multiple -viable paths: +Use the hardware check first. The table below is the fallback for when the +user has already told you their hardware: | Situation | Recommended Path | -|-----------|-----------------| +|-----------|------------------| | `verdict: cloud` from hardware check | **Path A: Comfy Cloud** | -| No GPU / just want to try it | **Path A: Comfy Cloud** (zero setup) | -| Windows + NVIDIA GPU + non-technical | **Path B: ComfyUI Desktop** (one-click installer) | -| Windows + NVIDIA GPU + technical | **Path C: Portable** or **Path D: comfy-cli** | -| Linux + any GPU | **Path D: comfy-cli** (easiest) or Path E manual | -| macOS + Apple Silicon | **Path B: ComfyUI Desktop** or **Path D: comfy-cli** | -| Headless / server / CI | **Path D: comfy-cli** | +| No GPU / want to try without commitment | **Path A: Comfy Cloud** | +| Windows + NVIDIA + non-technical | **Path B: ComfyUI Desktop** | +| Windows + NVIDIA + technical | **Path C: Portable** or **Path D: comfy-cli** | +| Linux + any GPU | **Path D: comfy-cli** (easiest) | +| macOS + Apple Silicon | **Path B: Desktop** or **Path D: comfy-cli** | +| Headless / server / CI / agents | **Path D: comfy-cli** | -For the fully automated path (hardware check → install → launch), just run: +For the fully automated path (hardware check → install → launch → verify): ```bash bash scripts/comfyui_setup.sh +# Or with overrides: +bash scripts/comfyui_setup.sh --m-series --port=8190 --workspace=/data/comfy ``` -It runs `hardware_check.py` internally, refuses to install locally when the verdict -is `cloud`, picks the right `comfy-cli` flag otherwise, then installs and launches. +It runs `hardware_check.py` internally, refuses to install locally when the +verdict is `cloud` (unless `--force-cloud-override`), picks the right +`comfy-cli` flag, and prefers `pipx`/`uvx` over global `pip` to avoid polluting +system Python. --- ### Path A: Comfy Cloud (No Local Install) -For users without a capable GPU or who want zero setup. -Powered by RTX 6000 Pro GPUs, all models pre-installed. +For users without a capable GPU or who want zero setup. Hosted on RTX 6000 Pro. **Docs:** https://docs.comfy.org/get_started/cloud -1. Go to https://comfy.org/cloud and sign up -2. Get an API key at https://platform.comfy.org/login - - Click `+ New` in API Keys section → Generate - - Save immediately (only visible once) +1. Sign up at https://comfy.org/cloud +2. Generate an API key at https://platform.comfy.org/login 3. Set the key: ```bash export COMFY_CLOUD_API_KEY="comfyui-xxxxxxxxxxxx" ``` -4. Run workflows via the script or web UI: +4. Run workflows: ```bash python3 scripts/run_workflow.py \ - --workflow workflow_api.json \ - --args '{"prompt": "a cat"}' \ + --workflow workflows/flux_dev_txt2img.json \ + --args '{"prompt": "..."}' \ --host https://cloud.comfy.org \ - --api-key "$COMFY_CLOUD_API_KEY" \ --output-dir ./outputs ``` **Pricing:** https://www.comfy.org/cloud/pricing -Subscription required. Concurrent limits: Free/Standard: 1 job, Creator: 3, Pro: 5. +**Concurrent jobs:** Free/Standard 1, Creator 3, Pro 5. Free tier +**cannot run workflows via API** — only browse models. Paid subscription +required for `/api/prompt`, `/api/upload/*`, `/api/view`, etc. --- -### Path B: ComfyUI Desktop (Windows/macOS) +### Path B: ComfyUI Desktop (Windows / macOS) One-click installer for non-technical users. Currently Beta. **Docs:** https://docs.comfy.org/installation/desktop - - **Windows (NVIDIA):** https://download.comfy.org/windows/nsis/x64 -- **macOS (Apple Silicon):** Available from https://comfy.org (download page) +- **macOS (Apple Silicon):** https://comfy.org -Steps: -1. Download and run installer -2. Select GPU type (NVIDIA recommended, or CPU mode) -3. Choose install location (SSD recommended, ~15GB needed) -4. Optionally migrate from existing ComfyUI Portable install -5. Desktop launches automatically — web UI opens in browser - -Desktop manages its own Python environment. For CLI access to the bundled env: -```bash -cd /ComfyUI -.venv/Scripts/activate # Windows -# or use the built-in terminal in the Desktop UI -``` - -**Limitations:** Desktop uses stable releases (may lag behind latest). -Linux not supported for Desktop — use comfy-cli or manual install. +Linux is **not supported** for Desktop — use Path D. --- ### Path C: ComfyUI Portable (Windows Only) -Standalone package with embedded Python. Extract and run. No install. - **Docs:** https://docs.comfy.org/installation/comfyui_portable_windows -1. Download from https://github.com/comfyanonymous/ComfyUI/releases - - Standard: Python 3.13 + CUDA 13.0 (modern NVIDIA GPUs) - - Alt: PyTorch CUDA 12.6 + Python 3.12 (NVIDIA 10 series and older) - - AMD (experimental) -2. Extract with 7-Zip -3. Run `run_nvidia_gpu.bat` (or `run_cpu.bat`) -4. Wait for "To see the GUI go to: http://127.0.0.1:8188" - -Update: run `update/update_comfyui.bat` (latest commit) or -`update/update_comfyui_stable.bat` (latest stable release). +Download from https://github.com/comfyanonymous/ComfyUI/releases, extract, +run `run_nvidia_gpu.bat`. Update via `update/update_comfyui_stable.bat`. --- @@ -353,22 +363,19 @@ Update: run `update/update_comfyui.bat` (latest commit) or The official CLI is the best path for headless/automated setups. **Docs:** https://docs.comfy.org/comfy-cli/getting-started -**Repo:** https://github.com/Comfy-Org/comfy-cli - -#### Prerequisites -- Python 3.10+ (3.13 recommended) -- pip (or conda/uv) -- GPU drivers installed (CUDA for NVIDIA, ROCm for AMD) #### Install comfy-cli ```bash -pip install comfy-cli -# or +# Recommended: +pipx install comfy-cli +# Or use uvx without installing: uvx --from comfy-cli comfy --help +# Or (if pipx/uvx unavailable): +pip install --user comfy-cli ``` -Disable analytics (avoids interactive prompt): +Disable analytics non-interactively: ```bash comfy --skip-prompt tracking disable ``` @@ -376,270 +383,225 @@ comfy --skip-prompt tracking disable #### Install ComfyUI ```bash -# Interactive (prompts for GPU type) -comfy install - -# Non-interactive variants: comfy --skip-prompt install --nvidia # NVIDIA (CUDA) comfy --skip-prompt install --amd # AMD (ROCm, Linux) comfy --skip-prompt install --m-series # Apple Silicon (MPS) comfy --skip-prompt install --cpu # CPU only (slow) - -# With faster dependency resolution: -comfy --skip-prompt install --nvidia --fast-deps +comfy --skip-prompt install --nvidia --fast-deps # uv-based dep resolution ``` -Default location: `~/comfy/ComfyUI` (Linux), `~/Documents/comfy/ComfyUI` (macOS/Win). -Override with: `comfy --workspace /custom/path install` +Default location: `~/comfy/ComfyUI` (Linux), `~/Documents/comfy/ComfyUI` +(macOS/Win). Override with `comfy --workspace /custom/path install`. -#### Launch Server +#### Launch / verify ```bash -comfy launch --background # background daemon on :8188 -comfy launch # foreground (see logs) -comfy launch -- --listen 0.0.0.0 # accessible on LAN -comfy launch -- --port 8190 # custom port -comfy launch -- --lowvram # low VRAM mode (6GB cards) -``` - -Verify server is running: -```bash -curl -s http://127.0.0.1:8188/system_stats | python3 -m json.tool -``` - -Stop background server: -```bash -comfy stop +comfy launch --background # background daemon on :8188 +comfy launch -- --listen 0.0.0.0 --port 8190 # LAN-accessible custom port +curl -s http://127.0.0.1:8188/system_stats # health check ``` --- -### Path E: Manual Install (Advanced / All Hardware) +### Path E: Manual Install (Advanced / Unsupported Hardware) -For full control or unsupported hardware (Ascend NPU, Cambricon MLU, Intel Arc). +For Ascend NPU, Cambricon MLU, Intel Arc, or other unsupported hardware. **Docs:** https://docs.comfy.org/installation/manual_install -**GitHub:** https://github.com/comfyanonymous/ComfyUI ```bash -# 1. Create environment -conda create -n comfyenv python=3.13 -conda activate comfyenv - -# 2. Clone git clone https://github.com/comfyanonymous/ComfyUI.git cd ComfyUI - -# 3. Install PyTorch (pick your hardware) -# NVIDIA: pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu130 -# AMD (ROCm 6.4): -pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.4 -# Apple Silicon: -pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cpu -# Intel Arc: -pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/xpu -# CPU only: -pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cpu - -# 4. Install ComfyUI deps pip install -r requirements.txt - -# 5. Run python main.py -# With options: python main.py --listen 0.0.0.0 --port 8188 ``` --- ### Post-Install: Download Models -ComfyUI needs at least one checkpoint model to generate images. - -**Using comfy-cli:** ```bash -# SDXL (general purpose, ~6.5GB) +# SDXL (general purpose, ~6.5 GB) comfy model download \ --url "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors" \ --relative-path models/checkpoints -# SD 1.5 (lighter, ~4GB, good for low VRAM) +# SD 1.5 (lighter, ~4 GB, good for 6 GB cards) comfy model download \ --url "https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors" \ --relative-path models/checkpoints -# From CivitAI (may need API token): +# Flux Dev fp8 (smaller variant, ~12 GB) +comfy model download \ + --url "https://huggingface.co/Comfy-Org/flux1-dev/resolve/main/flux1-dev-fp8.safetensors" \ + --relative-path models/checkpoints + +# CivitAI (set token first): comfy model download \ --url "https://civitai.com/api/download/models/128713" \ --relative-path models/checkpoints \ --set-civitai-api-token "YOUR_TOKEN" - -# LoRA adapters: -comfy model download --url "" --relative-path models/loras ``` -**Manual download:** Place `.safetensors` / `.ckpt` files directly into the -`ComfyUI/models/checkpoints/` directory (or `loras/`, `vae/`, etc.). - -List installed models: -```bash -comfy model list -``` - ---- +List installed: `comfy model list`. ### Post-Install: Install Custom Nodes -Custom nodes extend ComfyUI's capabilities (upscaling, video, ControlNet, etc.). - ```bash -comfy node install comfyui-impact-pack # popular utility pack -comfy node install comfyui-animatediff-evolved # video generation -comfy node install comfyui-controlnet-aux # ControlNet preprocessors -comfy node install comfyui-essentials # common helpers -comfy node update all # update all nodes +comfy node install comfyui-impact-pack # popular utility pack +comfy node install comfyui-animatediff-evolved # video generation +comfy node install comfyui-controlnet-aux # ControlNet preprocessors +comfy node install comfyui-essentials # common helpers +comfy node update all +comfy node install-deps --workflow=workflow.json # install everything a workflow needs ``` -Check what's installed: -```bash -comfy node show installed -``` - -Install deps for a specific workflow: -```bash -comfy node install-deps --workflow=workflow_api.json -``` - ---- - -### Post-Install: Verify Setup +### Post-Install: Verify ```bash -# Check server is responsive -curl -s http://127.0.0.1:8188/system_stats | python3 -m json.tool +python3 scripts/health_check.py +# → comfy_cli on PATH? server reachable? checkpoints? smoke test? -# Check a workflow's dependencies -python3 scripts/check_deps.py workflow_api.json --host 127.0.0.1 --port 8188 +python3 scripts/check_deps.py my_workflow.json +# → are this workflow's nodes/models/embeddings installed? -# Test a generation python3 scripts/run_workflow.py \ - --workflow workflow_api.json \ - --args '{"prompt": "test image, high quality"}' \ + --workflow workflows/sd15_txt2img.json \ + --args '{"prompt": "test", "steps": 4}' \ --output-dir ./test-outputs ``` ## Image Upload (img2img / Inpainting) -Upload files directly via REST: +The simplest way is to use `--input-image` with `run_workflow.py`: ```bash -# Upload input image +python3 scripts/run_workflow.py \ + --workflow workflows/sdxl_img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it cyberpunk", "denoise": 0.6}' +``` + +The flag uploads `photo.png`, then injects its server-side filename into +whatever schema parameter is named `image`. For inpainting, pass both: + +```bash +python3 scripts/run_workflow.py \ + --workflow workflows/sdxl_inpaint.json \ + --input-image image=./photo.png \ + --input-image mask_image=./mask.png \ + --args '{"prompt": "fill with flowers"}' +``` + +Manual upload via REST: +```bash curl -X POST "http://127.0.0.1:8188/upload/image" \ -F "image=@photo.png" -F "type=input" -F "overwrite=true" # Returns: {"name": "photo.png", "subfolder": "", "type": "input"} -# Upload mask for inpainting -curl -X POST "http://127.0.0.1:8188/upload/mask" \ - -F "image=@mask.png" -F "type=input" \ - -F 'original_ref={"filename":"photo.png","subfolder":"","type":"input"}' -``` - -Then reference the uploaded filename in workflow args: -```bash -python3 scripts/run_workflow.py --workflow inpaint.json \ - --args '{"image": "photo.png", "mask": "mask.png", "prompt": "fill with flowers"}' -``` - -## Cloud Execution - -Base URL: `https://cloud.comfy.org` -Auth: `X-API-Key` header - -```bash -# Submit workflow -python3 scripts/run_workflow.py \ - --workflow workflow_api.json \ - --args '{"prompt": "cyberpunk city"}' \ - --host https://cloud.comfy.org \ - --api-key "$COMFY_CLOUD_API_KEY" \ - --output-dir ./outputs \ - --timeout 300 - -# Upload image for cloud workflows +# Cloud equivalent: curl -X POST "https://cloud.comfy.org/api/upload/image" \ -H "X-API-Key: $COMFY_CLOUD_API_KEY" \ - -F "image=@input.png" -F "type=input" -F "overwrite=true" + -F "image=@photo.png" -F "type=input" -F "overwrite=true" ``` -Concurrent job limits: -| Tier | Concurrent Jobs | -|------|----------------| -| Free/Standard | 1 | -| Creator | 3 | -| Pro | 5 | +## Cloud Specifics -Extra submissions queue automatically. +- **Base URL:** `https://cloud.comfy.org` +- **Auth:** `X-API-Key` header (or `?token=KEY` for WebSocket) +- **API key:** set `$COMFY_CLOUD_API_KEY` once and the scripts pick it up automatically +- **Output download:** `/api/view` returns a 302 to a signed URL; the scripts + follow it and strip `X-API-Key` before fetching from the storage backend + (don't leak the API key to S3/CloudFront). +- **Endpoint differences from local ComfyUI:** + - `/api/object_info`, `/api/queue`, `/api/userdata` — **403 on free tier**; + paid only. + - `/history` is renamed to `/history_v2` on cloud (the scripts route + automatically). + - `/models/` is renamed to `/experiment/models/` on cloud + (the scripts route automatically). + - `clientId` in WebSocket is currently ignored — all connections for a + user receive the same broadcast. Filter by `prompt_id` client-side. + - `subfolder` is accepted on uploads but ignored — cloud has a flat namespace. +- **Concurrent jobs:** Free/Standard: 1, Creator: 3, Pro: 5. Extras queue + automatically. Use `run_batch.py --parallel N` to saturate your tier. ## Queue & System Management ```bash -# Check queue +# Local curl -s http://127.0.0.1:8188/queue | python3 -m json.tool - -# Clear pending queue -curl -X POST http://127.0.0.1:8188/queue -d '{"clear": true}' - -# Cancel running job -curl -X POST http://127.0.0.1:8188/interrupt - -# Free GPU memory (unload all models) -curl -X POST http://127.0.0.1:8188/free -H "Content-Type: application/json" \ +curl -X POST http://127.0.0.1:8188/queue -d '{"clear": true}' # cancel pending +curl -X POST http://127.0.0.1:8188/interrupt # cancel running +curl -X POST http://127.0.0.1:8188/free \ + -H "Content-Type: application/json" \ -d '{"unload_models": true, "free_memory": true}' -# System stats (VRAM, RAM, GPU info) -curl -s http://127.0.0.1:8188/system_stats | python3 -m json.tool +# Cloud — same paths under /api/, plus: +python3 scripts/fetch_logs.py --tail-queue --host https://cloud.comfy.org ``` ## Pitfalls -1. **API format required** — `comfy run` and the scripts only accept API-format workflow JSON. - If the user has editor format (from "Save" not "Save (API Format)"), they need to - re-export. Check: API format has `class_type` in each node object, editor format has - top-level `nodes` and `links` arrays. +1. **API format required** — every script and the `/api/prompt` endpoint expect + API-format workflow JSON. The scripts detect editor format (top-level + `nodes` and `links` arrays) and tell you to re-export via + "Workflow → Export (API)" (newer UI) or "Save (API Format)" (older UI). -2. **Server must be running** — All execution requires a live server. `comfy launch --background` - starts one. Check with `curl http://127.0.0.1:8188/system_stats`. +2. **Server must be running** — all execution requires a live server. + `comfy launch --background` starts one. Verify with + `curl http://127.0.0.1:8188/system_stats`. -3. **Model names are exact** — Case-sensitive, includes file extension. Use +3. **Model names are exact** — case-sensitive, includes file extension. + `check_deps.py` does fuzzy matching (with/without extension and folder + prefix), but the workflow itself must use the canonical name. Use `comfy model list` to discover what's installed. -4. **Missing custom nodes** — "class_type not found" means a required node isn't installed. - Run `check_deps.py` to find what's missing, then `comfy node install `. +4. **Missing custom nodes** — "class_type not found" means a required node + isn't installed. `check_deps.py` reports which package to install; + `auto_fix_deps.py` runs the install for you. -5. **Working directory** — `comfy-cli` auto-detects the ComfyUI workspace. If commands - fail with "no workspace found", use `comfy --workspace /path/to/ComfyUI ` - or `comfy set-default /path/to/ComfyUI`. +5. **Working directory** — `comfy-cli` auto-detects the ComfyUI workspace. + If commands fail with "no workspace found", use + `comfy --workspace /path/to/ComfyUI ` or + `comfy set-default /path/to/ComfyUI`. -6. **Cloud vs local output download** — Cloud `/api/view` returns a 302 redirect to a - signed URL. Always follow redirects (`curl -L`). The `run_workflow.py` script handles - this automatically. +6. **Cloud free-tier API limits** — `/api/prompt`, `/api/view`, `/api/upload/*`, + `/api/object_info` all return 403 on free accounts. `health_check.py` and + `check_deps.py` handle this gracefully and surface a clear message. -7. **Timeout for video/audio** — Long generations (video, high step counts) can take - minutes. Pass `--timeout 600` to `run_workflow.py`. Default is 120 seconds. +7. **Timeout for video/audio workflows** — auto-detected when an output node + is `VHS_VideoCombine`, `SaveVideo`, etc.; the default jumps from 300 s to + 900 s. Override explicitly with `--timeout 1800`. -8. **tracking prompt** — First run of `comfy` may prompt for analytics tracking consent. - Use `comfy --skip-prompt tracking disable` to skip it non-interactively. +8. **Path traversal in output filenames** — server-supplied filenames are + passed through `safe_path_join` to refuse anything escaping `--output-dir`. + Keep this protection on — workflows with custom save nodes can produce + arbitrary paths. -9. **comfy-cli invocation via uvx** — If comfy-cli is not installed globally, invoke with - `uvx --from comfy-cli comfy `. All examples in this skill use bare `comfy` - but prepend `uvx --from comfy-cli` if needed. +9. **Workflow JSON is arbitrary code** — custom nodes run Python, so + submitting an unknown workflow has the same trust profile as `eval`. + Inspect workflows from untrusted sources before running. + +10. **Auto-randomized seed** — pass `seed: -1` in `--args` (or use + `--randomize-seed` and omit the seed) to get a fresh seed per run. + The actual seed is logged to stderr. + +11. **`tracking` prompt** — first run of `comfy` may prompt for analytics. + Use `comfy --skip-prompt tracking disable` to skip non-interactively. + `comfyui_setup.sh` does this for you. ## Verification Checklist +Use `python3 scripts/health_check.py` to run the whole list at once. Manual: + - [ ] `hardware_check.py` verdict is `ok` OR the user explicitly chose Comfy Cloud -- [ ] `comfy` available on PATH (or `uvx --from comfy-cli comfy --help` works) -- [ ] `curl http://127.0.0.1:8188/system_stats` returns JSON -- [ ] `comfy model list` shows at least one checkpoint -- [ ] Workflow JSON is in API format (has `class_type` keys) -- [ ] `check_deps.py` reports no missing nodes/models -- [ ] Test run completes and outputs are saved +- [ ] `comfy --version` works (or `uvx --from comfy-cli comfy --help`) +- [ ] `curl http://HOST:PORT/system_stats` returns JSON +- [ ] `comfy model list` shows at least one checkpoint (local) OR + `/api/experiment/models/checkpoints` returns models (cloud) +- [ ] Workflow JSON is in API format +- [ ] `check_deps.py` reports `is_ready: true` (or only `node_check_skipped` + on cloud free tier) +- [ ] Test run with a small workflow completes; outputs land in `--output-dir` diff --git a/skills/creative/comfyui/references/official-cli.md b/skills/creative/comfyui/references/official-cli.md index d9b9e5753fe..59a981b4a8b 100644 --- a/skills/creative/comfyui/references/official-cli.md +++ b/skills/creative/comfyui/references/official-cli.md @@ -5,12 +5,16 @@ Docs: https://docs.comfy.org/comfy-cli/getting-started ## Installation +Order of preference: + ```bash -pip install comfy-cli -# or -uvx --from comfy-cli comfy --help +pipx install comfy-cli # recommended (isolated env) +uvx --from comfy-cli comfy --help # zero-install via uv +pip install --user comfy-cli # fallback ``` +The skill's `comfyui_setup.sh` picks the best available method. + First run may prompt for analytics. Disable non-interactively: ```bash comfy --skip-prompt tracking disable @@ -32,9 +36,9 @@ Workspace resolution priority: 3. `--here` (cwd) 4. `comfy set-default` path 5. Most recently used -6. `~/comfy/ComfyUI` (Linux) or `~/Documents/comfy/ComfyUI` (macOS) +6. `~/comfy/ComfyUI` (Linux) or `~/Documents/comfy/ComfyUI` (macOS/Win) -## Commands +## Lifecycle Commands ### `comfy install` @@ -42,65 +46,53 @@ Download and install ComfyUI + ComfyUI-Manager. ```bash comfy install # interactive GPU selection -comfy install --nvidia # NVIDIA (CUDA) -comfy install --amd # AMD (ROCm) +comfy install --nvidia +comfy install --amd # ROCm (Linux) comfy install --m-series # Apple Silicon (MPS) -comfy install --cpu # CPU only -comfy install --fast-deps # use uv for faster deps +comfy install --cpu # CPU only (slow) +comfy install --fast-deps # use uv for deps comfy install --skip-manager # skip ComfyUI-Manager ``` | Option | Description | |--------|-------------| -| `--nvidia` | NVIDIA GPU | -| `--amd` | AMD GPU (ROCm) | -| `--m-series` | Apple Silicon | -| `--cpu` | CPU only | +| `--nvidia` / `--amd` / `--m-series` / `--cpu` | GPU type | | `--cuda-version` | 11.8, 12.1, 12.4, 12.6, 12.8, 12.9, 13.0 | | `--rocm-version` | 6.1, 6.2, 6.3, 7.0, 7.1 | -| `--fast-deps` | Use uv for dependency resolution | +| `--fast-deps` | uv-based dependency resolution | | `--skip-manager` | Don't install ComfyUI-Manager | | `--skip-torch-or-directml` | Skip PyTorch install | -| `--version ` | Specific ComfyUI version (e.g. `0.2.0`, `latest`, `nightly`) | +| `--version ` | `0.2.0`, `latest`, `nightly` | | `--commit ` | Install specific commit | | `--pr "#1234"` | Install from a PR | | `--restore` | Restore deps for existing install | -Default location: `~/comfy/ComfyUI` (Linux), `~/Documents/comfy/ComfyUI` (macOS/Win). - ### `comfy launch` -Start ComfyUI server. - ```bash -comfy launch # foreground on :8188 -comfy launch --background # background daemon -comfy launch -- --listen 0.0.0.0 # listen on all interfaces -comfy launch -- --port 8190 # custom port -comfy launch -- --cpu # force CPU mode +comfy launch # foreground :8188 +comfy launch --background # background daemon +comfy launch -- --listen 0.0.0.0 # LAN-accessible +comfy launch -- --port 8190 # custom port +comfy launch -- --cpu # force CPU mode +comfy launch -- --lowvram # 6 GB cards comfy launch --background -- --listen 0.0.0.0 --port 8190 ``` -| Option | Description | -|--------|-------------| -| `--background` | Run as background daemon | -| `--frontend-pr "#456"` | Test a frontend PR | -| Extra args after `--` | Passed directly to ComfyUI's `main.py` | - -Common extra args: `--listen`, `--port`, `--cpu`, `--lowvram`, `--novram`, -`--fp16-vae`, `--force-fp32`. +Common extra args after `--`: `--listen`, `--port`, `--cpu`, `--lowvram`, +`--novram`, `--fp16-vae`, `--force-fp32`, `--disable-cuda-malloc`. ### `comfy stop` -Stop background ComfyUI instance. - ```bash comfy stop ``` ### `comfy run` -Execute a raw workflow JSON file against a running server. +Submit a raw workflow JSON to a running server. **Limited** — no parameter +injection, no structured output download. For agents, use +`scripts/run_workflow.py` instead. ```bash comfy run --workflow workflow_api.json @@ -108,31 +100,15 @@ comfy run --workflow workflow_api.json --host 10.0.0.5 --port 8188 comfy run --workflow workflow_api.json --timeout 300 --wait ``` -| Option | Description | -|--------|-------------| -| `--workflow` | Path to API-format workflow JSON (required) | -| `--host` | Server hostname (default: 127.0.0.1) | -| `--port` | Server port (default: 8188) | -| `--timeout` | Seconds to wait (default: 30) | -| `--wait/--no-wait` | Wait for completion (default: wait) | -| `--verbose` | Show per-node execution details | - -**Limitations:** No parameter injection, no structured output, no image download. -For agent use, prefer `scripts/run_workflow.py` which adds those capabilities. - ### `comfy which` -Show which ComfyUI workspace is currently targeted. - ```bash -comfy which +comfy which # show targeted workspace comfy --recent which ``` ### `comfy set-default` -Set the default workspace path. - ```bash comfy set-default /path/to/ComfyUI comfy set-default /path/to/ComfyUI --launch-extras="--listen 0.0.0.0" @@ -140,8 +116,6 @@ comfy set-default /path/to/ComfyUI --launch-extras="--listen 0.0.0.0" ### `comfy update` -Update ComfyUI or custom nodes. - ```bash comfy update # update ComfyUI core comfy node update all # update all custom nodes @@ -151,33 +125,32 @@ comfy node update all # update all custom nodes ## `comfy node` — Custom Node Management -All node operations use ComfyUI-Manager (cm-cli) under the hood. +All node operations use ComfyUI-Manager (`cm-cli`) under the hood. ```bash -comfy node show installed # list installed nodes -comfy node show enabled # list enabled nodes -comfy node show all # all available nodes +comfy node show installed # list installed +comfy node show enabled # list enabled +comfy node show all # all available in registry comfy node simple-show installed # compact list -comfy node install comfyui-impact-pack # install by name -comfy node install --uv-compile # with unified dep resolution (Manager v4.1+) -comfy node uninstall # remove -comfy node update # update one -comfy node update all # update all -comfy node enable # enable disabled node -comfy node disable # disable without uninstalling -comfy node fix # fix broken dependencies +comfy node install comfyui-impact-pack +comfy node install --uv-compile # ComfyUI-Manager v4.1+ unified resolver +comfy node uninstall +comfy node update | all +comfy node enable +comfy node disable +comfy node fix # fix broken deps -comfy node install-deps --workflow=workflow.json # install all deps a workflow needs -comfy node deps-in-workflow --workflow=w.json --output=deps.json # extract dep list +comfy node install-deps --workflow=workflow.json +comfy node deps-in-workflow --workflow=w.json --output=deps.json -comfy node save-snapshot # save current state -comfy node restore-snapshot # restore from snapshot +comfy node save-snapshot +comfy node restore-snapshot -comfy node bisect start # find culprit node (binary search) -comfy node bisect good # current set is fine -comfy node bisect bad # problem is in current set -comfy node bisect reset # abort bisect +comfy node bisect start # binary-search a culprit node +comfy node bisect good +comfy node bisect bad +comfy node bisect reset ``` ### Dependency Resolution Options @@ -188,21 +161,21 @@ comfy node bisect reset # abort bisect | `--uv-compile` | ComfyUI-Manager v4.1+ unified resolver (recommended) | | `--no-deps` | Skip dep installation | -Set uv-compile as default: `comfy manager uv-compile-default true` +Make `uv-compile` default: `comfy manager uv-compile-default true` --- ## `comfy model` — Model Management ```bash -comfy model list # list all downloaded models -comfy model list --relative-path models/checkpoints # specific folder +comfy model list +comfy model list --relative-path models/checkpoints -comfy model download --url # download model +comfy model download --url comfy model download --url --relative-path models/loras comfy model download --url --filename custom_name.safetensors -comfy model remove # interactive removal +comfy model remove # interactive comfy model remove --relative-path models/checkpoints --model-names "model.safetensors" ``` @@ -210,24 +183,27 @@ comfy model remove --relative-path models/checkpoints --model-names "model.safet |--------|-------------| | `--url` | Download URL (CivitAI, HuggingFace, direct) | | `--relative-path` | Subdirectory under workspace (e.g. `models/checkpoints`) | -| `--filename` | Custom filename to save as | -| `--set-civitai-api-token` | Set CivitAI API token | -| `--set-hf-api-token` | Set HuggingFace API token | +| `--filename` | Custom save filename | +| `--set-civitai-api-token` | Persist CivitAI token | +| `--set-hf-api-token` | Persist HuggingFace token | | `--downloader` | `httpx` (default) or `aria2` | -Model directory structure: +Standard model directories: ``` ComfyUI/models/ -├── checkpoints/ # Full model files (.safetensors, .ckpt) -├── loras/ # LoRA adapters -├── vae/ # VAE models -├── controlnet/ # ControlNet models -├── clip/ # CLIP text encoders -├── clip_vision/ # CLIP vision encoders -├── upscale_models/ # Upscaler models (ESRGAN, etc.) -├── embeddings/ # Textual inversion embeddings -├── unet/ # UNet models -└── diffusion_models/ # Diffusion model files +├── checkpoints/ # Full model files +├── loras/ # LoRA adapters +├── vae/ # VAE models +├── controlnet/ # ControlNet models +├── clip/ # CLIP / T5 text encoders +├── clip_vision/ # CLIP vision encoders +├── upscale_models/ # ESRGAN / SwinIR / etc. +├── embeddings/ # Textual inversion embeddings +├── unet/ # Standalone UNet weights +├── diffusion_models/ # Flux / SD3 / Wan diffusion models +├── animatediff_models/ # AnimateDiff motion modules +├── ipadapter/ # IPAdapter weights +└── style_models/ # Style adapters ``` --- @@ -235,12 +211,12 @@ ComfyUI/models/ ## `comfy manager` — ComfyUI-Manager Settings ```bash -comfy manager disable # disable Manager completely -comfy manager enable-gui # enable new GUI -comfy manager disable-gui # disable GUI (API-only) -comfy manager enable-legacy-gui # legacy GUI +comfy manager disable # disable Manager completely +comfy manager enable-gui # enable new GUI +comfy manager disable-gui # API-only +comfy manager enable-legacy-gui # legacy GUI comfy manager uv-compile-default true # make --uv-compile the default -comfy manager clear # clear startup action +comfy manager clear # clear startup action ``` --- @@ -248,21 +224,32 @@ comfy manager clear # clear startup action ## `comfy pr-cache` — Frontend PR Cache ```bash -comfy pr-cache list # list cached PR builds -comfy pr-cache clean # clean all -comfy pr-cache clean 456 # clean specific PR +comfy pr-cache list +comfy pr-cache clean +comfy pr-cache clean 456 ``` -Cache expires after 7 days; max 10 builds kept. +Cache expires after 7 days; max 10 builds. --- ## Configuration -Config file location: -- Linux: `~/.config/comfy-cli/config.ini` -- macOS: `~/Library/Application Support/comfy-cli/config.ini` -- Windows: `~/AppData/Local/comfy-cli/config.ini` +| OS | Path | +|----|------| +| Linux | `~/.config/comfy-cli/config.ini` | +| macOS | `~/Library/Application Support/comfy-cli/config.ini` | +| Windows | `~/AppData/Local/comfy-cli/config.ini` | -Stores: default workspace, recent workspace, background server info, API tokens, -manager GUI mode, launch extras. +Stores: default workspace, recent workspace, background server PID, API +tokens, manager GUI mode, launch extras. + +## Discovery + +Custom-node registry: +- https://registry.comfy.org/ + +Model browsers: +- https://huggingface.co/models +- https://civitai.com (NSFW; requires API token for many) +- https://comfyworkflows.com (community workflows) diff --git a/skills/creative/comfyui/references/rest-api.md b/skills/creative/comfyui/references/rest-api.md index 47af548401c..64091c9d67e 100644 --- a/skills/creative/comfyui/references/rest-api.md +++ b/skills/creative/comfyui/references/rest-api.md @@ -1,16 +1,50 @@ -# ComfyUI REST API Reference +# ComfyUI REST + WebSocket API Reference -ComfyUI exposes a REST API + WebSocket for workflow execution and management. -Same API surface for local servers and Comfy Cloud (with auth differences). +ComfyUI exposes a REST + WebSocket interface for workflow execution and +management. **The same surface is used locally and on Comfy Cloud, with +auth/path differences.** ## Connection -| | Local | Cloud | +| | Local ComfyUI | Comfy Cloud | |---|---|---| | Base URL | `http://127.0.0.1:8188` | `https://cloud.comfy.org` | -| Auth | None (or bearer token) | `X-API-Key` header | +| API path prefix | none (`/prompt`, `/view`, …) | `/api/...` (`/api/prompt`, `/api/view`, …) | +| Auth | none (or bearer token if configured) | `X-API-Key` header | | WebSocket | `ws://host:port/ws?clientId={uuid}` | `wss://cloud.comfy.org/ws?clientId={uuid}&token={API_KEY}` | -| Output download | Direct bytes from `/view` | 302 redirect → signed URL (use `curl -L`) | +| `/api/view` response | direct bytes | 302 redirect → signed URL (use `curl -L`) | + +The skill scripts route URLs automatically via `_common.resolve_url()`. + +## Endpoint differences on Comfy Cloud + +The cloud surface diverges from local ComfyUI in several ways. The skill +scripts handle these transparently; document them here so anyone calling +`curl` directly knows. + +| Local path | Cloud path | Notes | +|------------|-----------|-------| +| `/system_stats` | `/api/system_stats` | Cloud version is **public** (no auth required) | +| `/object_info` | `/api/object_info` | **Paid tier only** — free returns 403 | +| `/queue` | `/api/queue` | Paid tier only | +| `/userdata` | `/api/userdata` | Paid tier only | +| `/prompt` (POST) | `/api/prompt` (POST) | Paid tier only | +| `/upload/image` | `/api/upload/image` | Paid tier only; `subfolder` accepted but ignored | +| `/upload/mask` | `/api/upload/mask` | Same as above | +| `/view` | `/api/view` | Paid tier only; **returns 302** to signed URL | +| `/history` | `/api/history_v2` | **Renamed**; old path returns 404 | +| `/history/{id}` | `/api/history_v2/{id}` or `/api/jobs/{id}` | Both work; `/jobs` returns full job | +| `/models` | `/api/experiment/models` | **Renamed** | +| `/models/{folder}` | `/api/experiment/models/{folder}` | **Renamed**; response shape differs (see below) | + +### Cloud model-list response shape + +- **Local:** `["a.safetensors", "b.safetensors", …]` — flat list of strings. +- **Cloud:** `[{"name": "a.safetensors", "pathIndex": 0}, …]` — list of objects. +- **Cloud 404 with `code: "folder_not_found"`** — folder is empty or unknown, + not an "endpoint missing" error. Distinguish by reading the body. + +The skill helper `_common.parse_model_list()` normalizes both. ## Workflow Execution @@ -34,7 +68,8 @@ curl -X POST "https://cloud.comfy.org/api/prompt" \ {"prompt_id": "abc-123-def", "number": 1, "node_errors": {}} ``` -If `node_errors` is non-empty, the workflow has validation errors (missing nodes, bad inputs). +If `node_errors` is non-empty, the workflow has validation errors (missing +nodes, bad inputs). ### Check Job Status (Cloud) @@ -43,133 +78,146 @@ curl -X GET "https://cloud.comfy.org/api/job/{prompt_id}/status" \ -H "X-API-Key: $COMFY_CLOUD_API_KEY" ``` -| Status | Description | -|--------|-------------| -| `pending` | Queued, waiting to start | -| `in_progress` | Currently executing | -| `completed` | Finished successfully | -| `failed` | Encountered an error | -| `cancelled` | Cancelled by user | +| Status | Description | +| ------------- | ---------------------------------- | +| `pending` | Job is queued and waiting to start | +| `in_progress` | Job is currently executing | +| `completed` | Job finished successfully | +| `failed` | Job encountered an error | +| `cancelled` | Job was cancelled by user | + +### Job detail with outputs (Cloud) + +```bash +curl -X GET "https://cloud.comfy.org/api/jobs/{prompt_id}" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" +``` + +Response includes `outputs` keyed by node ID. Cloud uses `video` (singular) +in the output structure; local uses `videos` (plural). The skill scripts +accept both. ### Get History (Local) ```bash -# All history -curl -s "http://127.0.0.1:8188/history" - -# Specific prompt -curl -s "http://127.0.0.1:8188/history/{prompt_id}" +curl -s "http://127.0.0.1:8188/history" # all +curl -s "http://127.0.0.1:8188/history/{id}" # one prompt_id ``` -Response contains `outputs` keyed by node ID with file references. +Local entry shape: +```json +{ + "": { + "prompt": [...], + "outputs": {"": {"images": [...]}}, + "status": { + "status_str": "success" | "error", + "completed": true | false, + "messages": [["execution_start", {...}], ["execution_error", {...}], …] + } + } +} +``` + +**Important:** when reading status, check `status_str == "error"` BEFORE +checking `completed`, because both can be true for failed runs. ### Download Output ```bash -# Local +# Local (direct bytes) curl -s "http://127.0.0.1:8188/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \ -o output.png -# Cloud (follow redirect) -curl -L "https://cloud.comfy.org/api/view?filename=ComfyUI_00001_.png&subfolder=&type=output" \ +# Cloud (302 → signed URL; -L follows; STRIP X-API-Key for the second hop) +curl -L "https://cloud.comfy.org/api/view?filename=...&type=output" \ -H "X-API-Key: $COMFY_CLOUD_API_KEY" \ -o output.png ``` ---- +The skill's `run_workflow.py` strips `X-API-Key` automatically on the +cross-host redirect, so the signed URL never sees your auth. ## WebSocket Monitoring -Connect to WebSocket for real-time execution progress. - -### Connection +Connect for real-time execution events. ```bash # Local wscat -c "ws://127.0.0.1:8188/ws?clientId=MY-UUID" # Cloud -wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=API_KEY" +wscat -c "wss://cloud.comfy.org/ws?clientId=MY-UUID&token=$COMFY_CLOUD_API_KEY" ``` -### Message Types (JSON) +**Note:** on Cloud the `clientId` is currently ignored — all messages for a +user are broadcast to every connection. Filter messages client-side by +`data.prompt_id`. + +### JSON Message Types | Type | When | Key Fields | |------|------|------------| -| `status` | Queue change | `queue_remaining` | +| `status` | Queue change | `status.exec_info.queue_remaining` | +| `notification` | User-friendly status string | `value` | | `execution_start` | Workflow begins | `prompt_id` | -| `executing` | Node running | `node` (ID), `prompt_id` | +| `executing` | Node running (or end-of-run if `node` is null on local) | `node`, `prompt_id` | | `progress` | Sampling steps | `node`, `value`, `max` | -| `executed` | Node output ready | `node`, `output` | -| `execution_cached` | Nodes skipped | `nodes` (list of IDs) | +| `progress_state` | Extended progress with per-node metadata | `nodes` (dict) | +| `executed` | Node output ready | `node`, `output` (with `images`/`video`/etc.) | +| `execution_cached` | Nodes skipped because of cache | `nodes` (list of IDs) | | `execution_success` | All done | `prompt_id` | -| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback` | +| `execution_error` | Failure | `exception_type`, `exception_message`, `traceback`, `node_id` | | `execution_interrupted` | Cancelled | `prompt_id` | -When `executing` has `node: null`, the workflow is complete. +### Binary Frames (Preview Images) -### Binary Messages (Preview Images) +| Type code | Meaning | +|-----------|---------| +| `0x00000001` | `PREVIEW_IMAGE` — `[type:4][image_type:4][data]` (image_type 1=JPEG, 2=PNG) | +| `0x00000003` | `TEXT` — `[type:4][nid_len:4][nid][text]` (UTF-8) | +| `0x00000004` | `PREVIEW_IMAGE_WITH_METADATA` — `[type:4][meta_len:4][json][image_data]` | -Format: `[4B type][4B image_type: 1=JPEG, 2=PNG][image_data...]` - ---- +`scripts/ws_monitor.py --previews ` saves preview frames to disk. ## File Upload -### Upload Image - ```bash +# Image curl -X POST "http://127.0.0.1:8188/upload/image" \ - -F "image=@photo.png" \ - -F "type=input" \ - -F "overwrite=true" -``` + -F "image=@photo.png" -F "type=input" -F "overwrite=true" +# Returns: {"name": "photo.png", "subfolder": "", "type": "input"} -Response: `{"name": "photo.png", "subfolder": "", "type": "input"}` - -### Upload Mask - -```bash +# Mask (linked to a previously uploaded image) curl -X POST "http://127.0.0.1:8188/upload/mask" \ - -F "image=@mask.png" \ - -F "type=input" \ + -F "image=@mask.png" -F "type=input" \ -F 'original_ref={"filename":"photo.png","subfolder":"","type":"input"}' ``` ---- +Cloud equivalent: prepend `https://cloud.comfy.org/api` and add `-H "X-API-Key: $COMFY_CLOUD_API_KEY"`. ## Node & Model Discovery -### Object Info (All Nodes) - ```bash +# All node types and their input specs curl -s "http://127.0.0.1:8188/object_info" | python3 -m json.tool -# Returns all node types with input/output definitions +# Specific node curl -s "http://127.0.0.1:8188/object_info/KSampler" -# Returns info for one specific node type -``` -### Models by Folder - -```bash +# Models per folder (local) curl -s "http://127.0.0.1:8188/models/checkpoints" curl -s "http://127.0.0.1:8188/models/loras" -curl -s "http://127.0.0.1:8188/models/vae" -curl -s "http://127.0.0.1:8188/models/controlnet" -curl -s "http://127.0.0.1:8188/models/clip" -curl -s "http://127.0.0.1:8188/models/upscale_models" -curl -s "http://127.0.0.1:8188/models/embeddings" + +# Models per folder (cloud — note the experimental prefix) +curl -s "https://cloud.comfy.org/api/experiment/models/checkpoints" \ + -H "X-API-Key: $COMFY_CLOUD_API_KEY" ``` -Returns arrays of filenames (relative to model folder). - ---- - ## Queue Management ```bash -# View queue (running + pending) +# View queue curl -s "http://127.0.0.1:8188/queue" # Clear all pending @@ -177,21 +225,19 @@ curl -X POST "http://127.0.0.1:8188/queue" \ -H "Content-Type: application/json" \ -d '{"clear": true}' -# Delete specific items from queue +# Delete specific items curl -X POST "http://127.0.0.1:8188/queue" \ -H "Content-Type: application/json" \ -d '{"delete": ["prompt_id_1", "prompt_id_2"]}' -# Cancel currently running job +# Cancel currently-running job curl -X POST "http://127.0.0.1:8188/interrupt" ``` ---- - ## System Management ```bash -# System stats (VRAM, RAM, GPU, versions) +# Stats (VRAM, RAM, GPU, ComfyUI version) curl -s "http://127.0.0.1:8188/system_stats" # Free GPU memory @@ -200,14 +246,13 @@ curl -X POST "http://127.0.0.1:8188/free" \ -d '{"unload_models": true, "free_memory": true}' ``` ---- +## ComfyUI-Manager Endpoints (Optional) -## ComfyUI Manager Endpoints (Optional) - -These require ComfyUI-Manager installed. +These require ComfyUI-Manager installed. Useful for installing nodes/models +via the API instead of `comfy-cli`. ```bash -# Install custom node from git repo +# Install a custom node from a git URL curl -X POST "http://127.0.0.1:8188/manager/queue/install" \ -H "Content-Type: application/json" \ -d '{"git_url": "https://github.com/user/comfyui-node.git"}' @@ -221,8 +266,6 @@ curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \ -d '{"url": "https://...", "path": "models/checkpoints", "filename": "model.safetensors"}' ``` ---- - ## POST /prompt Payload Format ```json @@ -246,11 +289,24 @@ curl -X POST "http://127.0.0.1:8188/manager/queue/install_model" \ }, "client_id": "unique-uuid-for-ws-filtering", "extra_data": { - "api_key_comfy_org": "optional-partner-node-key" + "api_key_comfy_org": "optional-PARTNER-NODE-key (NOT the cloud auth key)" } } ``` -- `prompt`: The workflow graph (API format) -- `client_id`: UUID for WebSocket event filtering -- `extra_data.api_key_comfy_org`: Required for paid partner nodes (Flux Pro, Ideogram, etc.) +- `prompt`: workflow graph in API format +- `client_id`: UUID — local server uses it to filter WebSocket events; cloud + ignores it. +- `extra_data.api_key_comfy_org`: ONLY required when the workflow uses + partner nodes (Flux Pro, Ideogram, etc.). Don't conflate with `X-API-Key`. + +## Error Categories (cloud `execution_error` `exception_type`) + +| Type | Meaning | +|------|---------| +| `ValidationError` | Bad workflow / inputs (often nicer to surface from `node_errors`) | +| `ModelDownloadError` | Required model not available | +| `ImageDownloadError` | Failed to fetch input image from URL | +| `OOMError` | Out of GPU memory | +| `InsufficientFundsError` | Account balance too low (partner nodes) | +| `InactiveSubscriptionError` | Subscription not active | diff --git a/skills/creative/comfyui/references/workflow-format.md b/skills/creative/comfyui/references/workflow-format.md index 64fb60dbf70..e8343de73ce 100644 --- a/skills/creative/comfyui/references/workflow-format.md +++ b/skills/creative/comfyui/references/workflow-format.md @@ -1,10 +1,12 @@ # ComfyUI Workflow JSON Format -## Two Formats +## Two Formats — Only API Format Is Executable -ComfyUI uses two workflow formats. **Only API format works for programmatic execution.** +**API format** is required for `/api/prompt` and every script in this skill. +The web UI also produces an "editor format" used for visual editing, which +**cannot** be submitted directly. -### API Format (what we use) +### API Format Top-level keys are string node IDs. Each node has `class_type` and `inputs`: @@ -28,191 +30,197 @@ Top-level keys are string node IDs. Each node has `class_type` and `inputs`: }, "4": { "class_type": "CheckpointLoaderSimple", - "inputs": { - "ckpt_name": "v1-5-pruned-emaonly.safetensors" - } - }, - "5": { - "class_type": "EmptyLatentImage", - "inputs": {"width": 512, "height": 512, "batch_size": 1} - }, - "6": { - "class_type": "CLIPTextEncode", - "inputs": { - "text": "a beautiful cat", - "clip": ["4", 1] - } - }, - "7": { - "class_type": "CLIPTextEncode", - "inputs": { - "text": "bad quality, ugly", - "clip": ["4", 1] - } - }, - "9": { - "class_type": "SaveImage", - "inputs": { - "filename_prefix": "ComfyUI", - "images": ["8", 0] - } + "inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"} } } ``` -**How to detect:** Top-level keys are numeric strings, each value has `class_type`. +**Detection:** every top-level value has `class_type`. The skill's +`_common.is_api_format()` does this check. ### Editor Format (not directly executable) -Has `nodes[]` and `links[]` arrays — the visual graph data from the ComfyUI web editor. -This is what "Save" produces. For API use, export with "Save (API Format)" instead. +Has `nodes[]` and `links[]` arrays — the visual graph. To convert: open in +ComfyUI's web UI and use **Workflow → Export (API)** (newer UI) or the +"Save (API Format)" button (older UI). -**How to detect:** Top-level has `"nodes"` and `"links"` keys. +**Detection:** top-level has `"nodes"` and `"links"` keys. ---- +## Inputs: Literals vs Links -## Input Connections +```json +"inputs": { + "text": "a cat", // literal — modifiable + "seed": 42, // literal — modifiable + "clip": ["4", 1] // link — wiring; do NOT overwrite +} +``` -Inputs can be: -- **Literal values**: `"text": "a cat"`, `"seed": 42`, `"width": 512` -- **Links to other nodes**: `["node_id", output_index]` — e.g., `["4", 0]` means - output slot 0 of node "4" - -Only literal values can be modified by parameter injection. Linked inputs are wiring. - ---- +Links are length-2 arrays of `[upstream_node_id, output_slot]`. The skill's +parameter injector refuses to overwrite a link with a literal (logs a +warning and skips). ## Common Node Types and Their Controllable Parameters +The full catalog lives in `scripts/_common.py` (`PARAM_PATTERNS` and +`MODEL_LOADERS`). Highlights: + ### Text Prompts | Node Class | Key Fields | -|------------|-----------| -| `CLIPTextEncode` | `text` (the prompt string) | +|------------|------------| +| `CLIPTextEncode` | `text` | | `CLIPTextEncodeSDXL` | `text_g`, `text_l`, `width`, `height` | +| `CLIPTextEncodeFlux` | `clip_l`, `t5xxl`, `guidance` | -Usually: positive prompt → one CLIPTextEncode, negative prompt → another. -Distinguish by checking the `_meta.title` field or by tracing which feeds into -positive vs negative inputs of the sampler. +To distinguish positive from negative the skill traces `KSampler.negative` +back through Reroute / Primitive nodes to the source CLIPTextEncode. Falls +back to `_meta.title` heuristics ("negative", "neg", "anti"). ### Sampling | Node Class | Key Fields | -|------------|-----------| +|------------|------------| | `KSampler` | `seed`, `steps`, `cfg`, `sampler_name`, `scheduler`, `denoise` | -| `KSamplerAdvanced` | `noise_seed`, `steps`, `cfg`, `sampler_name`, `scheduler`, `start_at_step`, `end_at_step` | -| `SamplerCustom` | `cfg`, `sampler`, `sigmas` | +| `KSamplerAdvanced` | `noise_seed`, `steps`, `cfg`, `start_at_step`, `end_at_step` | +| `SamplerCustom` | `noise_seed`, `cfg`, `sampler`, `sigmas` | +| `SamplerCustomAdvanced` | `noise_seed` (via RandomNoise input) | +| `RandomNoise` | `noise_seed` | +| `BasicScheduler` | `steps`, `scheduler`, `denoise` | +| `KSamplerSelect` | `sampler_name` | +| `BasicGuider` / `CFGGuider` | `cfg` | +| `ModelSamplingFlux` | `max_shift`, `base_shift`, `width`, `height` | +| `SDTurboScheduler` | `steps`, `denoise` | -### Image Dimensions +### Latent / Dimensions | Node Class | Key Fields | -|------------|-----------| +|------------|------------| | `EmptyLatentImage` | `width`, `height`, `batch_size` | -| `LatentUpscale` | `width`, `height`, `upscale_method` | +| `EmptySD3LatentImage` | `width`, `height`, `batch_size` | +| `EmptyHunyuanLatentVideo` | `width`, `height`, `length`, `batch_size` | +| `EmptyMochiLatentVideo` | `width`, `height`, `length`, `batch_size` | +| `EmptyLTXVLatentVideo` | `width`, `height`, `length`, `batch_size` | ### Model Loading -| Node Class | Key Fields | Model Folder | -|------------|-----------|-------------| +| Node Class | Key Fields | Folder | +|------------|------------|--------| | `CheckpointLoaderSimple` | `ckpt_name` | `checkpoints` | | `LoraLoader` | `lora_name`, `strength_model`, `strength_clip` | `loras` | +| `LoraLoaderModelOnly` | `lora_name`, `strength_model` | `loras` | | `VAELoader` | `vae_name` | `vae` | | `ControlNetLoader` | `control_net_name` | `controlnet` | | `CLIPLoader` | `clip_name` | `clip` | +| `DualCLIPLoader` | `clip_name1`, `clip_name2` | `clip` | +| `TripleCLIPLoader` | `clip_name1/2/3` | `clip` | | `UNETLoader` | `unet_name` | `unet` | | `DiffusionModelLoader` | `model_name` | `diffusion_models` | | `UpscaleModelLoader` | `model_name` | `upscale_models` | +| `IPAdapterModelLoader` | `ipadapter_file` | `ipadapter` | +| `ADE_AnimateDiffLoaderWithContext` | `model_name`, `motion_scale` | `animatediff_models` | ### Image Input/Output | Node Class | Key Fields | -|------------|-----------| -| `LoadImage` | `image` (filename on server, after upload) | -| `LoadImageMask` | `image`, `channel` | +|------------|------------| +| `LoadImage` | `image` (server-side filename, after upload) | +| `LoadImageMask` | `image`, `channel` (`red` / `green` / `blue` / `alpha`) | +| `VAEEncode` / `VAEDecode` | (no controllable fields) | +| `VAEEncodeForInpaint` | `grow_mask_by` | | `SaveImage` | `filename_prefix` | -| `PreviewImage` | (no controllable fields, just previews) | +| `VHS_VideoCombine` | `frame_rate`, `format`, `filename_prefix`, `loop_count`, `pingpong` | ### ControlNet | Node Class | Key Fields | -|------------|-----------| +|------------|------------| | `ControlNetApply` | `strength` | | `ControlNetApplyAdvanced` | `strength`, `start_percent`, `end_percent` | -### Video (AnimateDiff) +### IPAdapter (community pack `comfyui_ipadapter_plus`) | Node Class | Key Fields | -|------------|-----------| -| `ADE_AnimateDiffLoaderWithContext` | `model_name`, `motion_scale` | -| `VHS_VideoCombine` | `frame_rate`, `format`, `filename_prefix` | +|------------|------------| +| `IPAdapterAdvanced` | `weight`, `start_at`, `end_at` | +| `IPAdapter` | `weight` | ---- +### Embeddings (referenced inside prompt strings) + +ComfyUI scans prompt text for `embedding:NAME` syntax. The skill's +`_common.iter_embedding_refs()` extracts these as model dependencies. + +```text +"a beautiful cat, embedding:goodvibes:1.2, embedding:art-style" +``` + +`extract_schema.py` and `check_deps.py` surface these in +`embedding_dependencies` / `missing_embeddings`. ## Parameter Injection Pattern -To modify a workflow programmatically: - ```python import json, copy with open("workflow_api.json") as f: workflow = json.load(f) -# Deep copy to avoid mutating original wf = copy.deepcopy(workflow) - -# Inject parameters by node ID + field name -wf["6"]["inputs"]["text"] = "a beautiful sunset" # positive prompt -wf["7"]["inputs"]["text"] = "ugly, blurry" # negative prompt -wf["3"]["inputs"]["seed"] = 42 # seed -wf["3"]["inputs"]["steps"] = 30 # steps -wf["5"]["inputs"]["width"] = 1024 # width -wf["5"]["inputs"]["height"] = 1024 # height +wf["6"]["inputs"]["text"] = "a beautiful sunset" +wf["7"]["inputs"]["text"] = "ugly, blurry" +wf["3"]["inputs"]["seed"] = 42 +wf["3"]["inputs"]["steps"] = 30 +wf["5"]["inputs"]["width"] = 1024 +wf["5"]["inputs"]["height"] = 1024 ``` -The `scripts/extract_schema.py` in this skill automates discovering which -node IDs and fields correspond to which user-facing parameters. - ---- +`scripts/extract_schema.py` automates discovering which node IDs/fields +correspond to which user-facing parameters. It returns a `parameters` dict +that `run_workflow.py` reads to inject values from `--args`. ## Identifying Controllable Parameters (Heuristics) -When analyzing an unknown workflow, these patterns identify user-facing params: +For unknown workflows: -1. **Prompt text**: Any `CLIPTextEncode` → `text` field. Title/meta usually - indicates positive vs negative. - -2. **Seed**: Any `KSampler` / `KSamplerAdvanced` → `seed` / `noise_seed`. - Randomizable — set to different values for variations. - -3. **Dimensions**: `EmptyLatentImage` → `width`, `height`. Common: 512, 768, - 1024 (must be multiples of 8). - -4. **Steps**: `KSampler` → `steps`. More = higher quality + slower. 20-50 typical. - -5. **CFG scale**: `KSampler` → `cfg`. How closely to follow prompt. 5-15 typical. - -6. **Model/checkpoint**: `CheckpointLoaderSimple` → `ckpt_name`. Must match an - installed model filename exactly. - -7. **LoRA**: `LoraLoader` → `lora_name`, `strength_model`. Adapter name + weight. - -8. **Images for img2img**: `LoadImage` → `image`. Filename on server after upload. - -9. **Denoise strength**: `KSampler` → `denoise`. 0.0-1.0. Lower = closer to input - image. Only relevant for img2img. - ---- +1. **Prompt text** — any `CLIPTextEncode.text`. Use connection tracing back + from `KSampler.positive` / `.negative` to disambiguate (don't trust + meta-title alone). +2. **Seed** — `KSampler.seed` / `KSamplerAdvanced.noise_seed` / `RandomNoise.noise_seed`. +3. **Dimensions** — `Empty*LatentImage.width/height` (must be multiples of 8). +4. **Steps / CFG** — `KSampler.steps`, `KSampler.cfg`. Steps 20–50 typical. + CFG 5–15 typical (Flux uses guidance, not CFG). +5. **Model / checkpoint** — `CheckpointLoaderSimple.ckpt_name`. Filename must + match an installed file *exactly*. +6. **LoRA** — `LoraLoader.lora_name`, `.strength_model`. +7. **Images for img2img / inpaint** — `LoadImage.image`. Server-side filename + after upload. +8. **Denoise** — `KSampler.denoise`. 0.0–1.0; 1.0 = ignore input image, + 0.0 = pass through. Sweet spot for img2img: 0.4–0.7. ## Output Nodes -Output is produced by these node types: +Output is produced by these node types. The skill's `OUTPUT_NODES` set +extends to common community packs. | Node | Output Key | Content | |------|-----------|---------| | `SaveImage` | `images` | List of `{filename, subfolder, type}` | -| `VHS_VideoCombine` | `gifs` or `videos` | Video file references | -| `SaveAudio` | `audio` | Audio file references | | `PreviewImage` | `images` | Temporary preview (not saved) | +| `VHS_VideoCombine` | `gifs` (older) or `videos`/`video` (newer cloud) | Video file refs | +| `SaveAudio` | `audio` | Audio file refs | +| `SaveAnimatedWEBP` / `SaveAnimatedPNG` | `images` | Animated images | +| `Save3D` | `3d` | 3D asset refs | -After execution, fetch outputs from `/history/{prompt_id}` → `outputs` → `{node_id}`. +After execution, fetch outputs from `/history/{prompt_id}` (local) or +`/api/jobs/{prompt_id}` (cloud) → `outputs` → `{node_id}` → `{key}`. + +## Wrapper Variants + +Some saved JSON files wrap the workflow under a `"prompt"` key (matching +the `/api/prompt` payload shape). The skill's `_common.unwrap_workflow()` +handles this — pass any of: + +- raw API format: `{"3": {...}, "4": {...}}` +- wrapped: `{"prompt": {"3": {...}}, "client_id": "..."}` + +It rejects editor format with a clear error and a re-export instruction. diff --git a/skills/creative/comfyui/scripts/_common.py b/skills/creative/comfyui/scripts/_common.py new file mode 100644 index 00000000000..ead2baf8372 --- /dev/null +++ b/skills/creative/comfyui/scripts/_common.py @@ -0,0 +1,835 @@ +""" +_common.py — Shared logic for ComfyUI skill scripts. + +Single source of truth for: +- HTTP transport (with retry/backoff, streaming, timeout handling) +- Cloud detection and endpoint mapping (local ComfyUI vs Comfy Cloud) +- Workflow node-type catalogs (param patterns, model loaders, output nodes) +- API-format validation +- Path-traversal-safe file writes +- API-key loading from env / CLI + +Stdlib-only by design (with optional `requests` upgrade if installed). Python 3.10+. +""" + +from __future__ import annotations + +import json +import os +import random +import re +import sys +import time +import uuid +from dataclasses import dataclass +from pathlib import Path +from typing import Any, Iterator +from urllib.parse import urlparse + +# Optional: prefer `requests` if installed (better redirects, streaming, header handling) +try: + import requests # type: ignore[import-not-found] + HAS_REQUESTS = True +except ImportError: # pragma: no cover - exercised via stdlib fallback + HAS_REQUESTS = False + import urllib.error + import urllib.request + + +# ============================================================================= +# Constants & catalogs +# ============================================================================= + +DEFAULT_LOCAL_HOST = "http://127.0.0.1:8188" +DEFAULT_CLOUD_HOST = "https://cloud.comfy.org" +ENV_API_KEY = "COMFY_CLOUD_API_KEY" + +# Connection / retry defaults +DEFAULT_HTTP_TIMEOUT = 60 # seconds — single-attempt request timeout +DEFAULT_RETRIES = 3 # total attempts including the first +RETRY_BASE_DELAY = 1.0 # seconds — exponential backoff base +RETRY_MAX_DELAY = 30.0 # seconds — cap on backoff +RETRY_STATUS_CODES = {408, 429, 500, 502, 503, 504, 522, 524} + +# Streaming download chunk size (bytes) +DOWNLOAD_CHUNK_SIZE = 1 << 16 # 64 KiB + +# Heuristic: workflows with these node types tend to be slow → larger default timeout +SLOW_OUTPUT_NODES = { + "VHS_VideoCombine", "SaveAnimatedWEBP", "SaveAnimatedPNG", + "SaveVideo", "SaveAudio", "SaveAnimateDiffVideo", + "SVD_img2vid_Conditioning", + "WanVideoSampler", "HunyuanVideoSampler", + "CogVideoSampler", "LTXVideoSampler", +} + +# --------------------------------------------------------------------------- +# Output node catalog (extensible — community packs add their own) +# --------------------------------------------------------------------------- +OUTPUT_NODES: set[str] = { + # Built-in + "SaveImage", "PreviewImage", + "SaveAudio", "SaveVideo", "PreviewAudio", "PreviewVideo", + "SaveAnimatedWEBP", "SaveAnimatedPNG", + # Common community packs + "VHS_VideoCombine", # Video Helper Suite + "ImageSave", # Was Node Suite + "Image Save", # Was Node Suite (alt name) + "easy imageSave", # easy-use + "Image Save With Metadata", + "PreviewImage|pysssss", # pysssss preview + "ShowText|pysssss", + "SaveLatent", + "SaveGLB", # 3D + "Save3D", +} + +# --------------------------------------------------------------------------- +# Folder aliases — handle ComfyUI's gradual folder renames +# --------------------------------------------------------------------------- +# When `check_deps.py` queries `/models/` and gets 404 / empty, +# it tries each alias in turn. Critical for Comfy Cloud which has fully +# migrated to the new naming (unet → diffusion_models, clip → text_encoders). +FOLDER_ALIASES: dict[str, list[str]] = { + "unet": ["unet", "diffusion_models"], + "diffusion_models": ["diffusion_models", "unet"], + "clip": ["clip", "text_encoders"], + "text_encoders": ["text_encoders", "clip"], + "controlnet": ["controlnet", "control_net"], +} + + +def folder_aliases_for(folder: str) -> list[str]: + """Return the search order of folder names (primary first).""" + return FOLDER_ALIASES.get(folder, [folder]) + + +# --------------------------------------------------------------------------- +# Model-loader catalog: class_type -> (input field, model folder) +# --------------------------------------------------------------------------- +# A loader can have multiple fields (e.g., DualCLIPLoader has clip_name1 and +# clip_name2). We list them with explicit entries. The folder name is the +# *canonical* one; FOLDER_ALIASES is consulted when querying. +MODEL_LOADERS: dict[str, list[tuple[str, str]]] = { + # Checkpoints + "CheckpointLoaderSimple": [("ckpt_name", "checkpoints")], + "CheckpointLoader": [("ckpt_name", "checkpoints")], + "CheckpointLoader (Simple)": [("ckpt_name", "checkpoints")], + "ImageOnlyCheckpointLoader": [("ckpt_name", "checkpoints")], + "unCLIPCheckpointLoader": [("ckpt_name", "checkpoints")], + # LoRA + "LoraLoader": [("lora_name", "loras")], + "LoraLoaderModelOnly": [("lora_name", "loras")], + "LoraLoaderTagsQuery": [("lora_name", "loras")], + # VAE + "VAELoader": [("vae_name", "vae")], + # ControlNet + "ControlNetLoader": [("control_net_name", "controlnet")], + "DiffControlNetLoader": [("control_net_name", "controlnet")], + "ControlNetLoaderAdvanced": [("control_net_name", "controlnet")], + # CLIP / text encoders (primary "clip" folder; check_deps tries text_encoders too) + "CLIPLoader": [("clip_name", "clip")], + "DualCLIPLoader": [("clip_name1", "clip"), ("clip_name2", "clip")], + "TripleCLIPLoader": [("clip_name1", "clip"), ("clip_name2", "clip"), ("clip_name3", "clip")], + "CLIPVisionLoader": [("clip_name", "clip_vision")], + # UNET / Diffusion model (primary "unet"; check_deps tries diffusion_models too) + "UNETLoader": [("unet_name", "unet")], + "DiffusionModelLoader": [("model_name", "diffusion_models")], + "UNETLoaderGGUF": [("unet_name", "unet")], + # Upscaler + "UpscaleModelLoader": [("model_name", "upscale_models")], + # Style / GLIGEN / Hypernetwork + "StyleModelLoader": [("style_model_name", "style_models")], + "GLIGENLoader": [("gligen_name", "gligen")], + "HypernetworkLoader": [("hypernetwork_name", "hypernetworks")], + # IPAdapter family (community) + "IPAdapterModelLoader": [("ipadapter_file", "ipadapter")], + "IPAdapterUnifiedLoader": [("preset", "ipadapter")], + "IPAdapterInsightFaceLoader": [("provider", "insightface")], + "InsightFaceLoader": [("provider", "insightface")], + "InstantIDModelLoader": [("instantid_file", "instantid")], + # AnimateDiff / video + "ADE_LoadAnimateDiffModel": [("model_name", "animatediff_models")], + "ADE_AnimateDiffLoaderWithContext": [("model_name", "animatediff_models")], + "ADE_AnimateDiffLoaderGen1": [("model_name", "animatediff_models")], + # Photomaker + "PhotoMakerLoader": [("photomaker_model_name", "photomaker")], + # Sampler / scheduler models + "ModelSamplingFlux": [], # parametric only +} + +# --------------------------------------------------------------------------- +# Param patterns: (class_type, field_name) -> friendly_name +# Order matters — first match wins for naming. Use _meta.title for disambiguation. +# --------------------------------------------------------------------------- +PARAM_PATTERNS: list[tuple[str, str, str]] = [ + # ---- Prompts ---- + ("CLIPTextEncode", "text", "prompt"), + ("CLIPTextEncodeSDXL", "text_g", "prompt"), + ("CLIPTextEncodeSDXL", "text_l", "prompt_l"), + ("CLIPTextEncodeSDXLRefiner", "text", "refiner_prompt"), + ("CLIPTextEncodeFlux", "clip_l", "prompt_l"), + ("CLIPTextEncodeFlux", "t5xxl", "prompt"), + ("CLIPTextEncodeFlux", "guidance", "guidance"), + ("smZ CLIPTextEncode", "text", "prompt"), + ("BNK_CLIPTextEncodeAdvanced", "text", "prompt"), + + # ---- Standard sampling ---- + ("KSampler", "seed", "seed"), + ("KSampler", "steps", "steps"), + ("KSampler", "cfg", "cfg"), + ("KSampler", "sampler_name", "sampler_name"), + ("KSampler", "scheduler", "scheduler"), + ("KSampler", "denoise", "denoise"), + ("KSamplerAdvanced", "noise_seed", "seed"), + ("KSamplerAdvanced", "steps", "steps"), + ("KSamplerAdvanced", "cfg", "cfg"), + ("KSamplerAdvanced", "sampler_name", "sampler_name"), + ("KSamplerAdvanced", "scheduler", "scheduler"), + ("KSamplerAdvanced", "start_at_step", "start_at_step"), + ("KSamplerAdvanced", "end_at_step", "end_at_step"), + + # ---- Modern sampler chain (Flux / SD3 / SDXL refiner via SamplerCustom) ---- + ("RandomNoise", "noise_seed", "seed"), + ("BasicScheduler", "steps", "steps"), + ("BasicScheduler", "scheduler", "scheduler"), + ("BasicScheduler", "denoise", "denoise"), + ("KSamplerSelect", "sampler_name", "sampler_name"), + ("BasicGuider", "cfg", "cfg"), + ("CFGGuider", "cfg", "cfg"), + ("DualCFGGuider", "cfg_conds", "cfg"), + ("DualCFGGuider", "cfg_cond2_negative", "cfg_negative"), + ("ModelSamplingFlux", "max_shift", "max_shift"), + ("ModelSamplingFlux", "base_shift", "base_shift"), + ("ModelSamplingFlux", "width", "model_width"), + ("ModelSamplingFlux", "height", "model_height"), + ("ModelSamplingSD3", "shift", "shift"), + ("ModelSamplingDiscrete", "sampling", "sampling"), + ("SDTurboScheduler", "steps", "steps"), + ("SDTurboScheduler", "denoise", "denoise"), + ("SamplerCustom", "noise_seed", "seed"), + ("SamplerCustom", "cfg", "cfg"), + ("SamplerCustomAdvanced", "noise_seed", "seed"), + + # ---- Dimensions / latent ---- + ("EmptyLatentImage", "width", "width"), + ("EmptyLatentImage", "height", "height"), + ("EmptyLatentImage", "batch_size", "batch_size"), + ("EmptySD3LatentImage", "width", "width"), + ("EmptySD3LatentImage", "height", "height"), + ("EmptySD3LatentImage", "batch_size", "batch_size"), + ("EmptyHunyuanLatentVideo", "width", "width"), + ("EmptyHunyuanLatentVideo", "height", "height"), + ("EmptyHunyuanLatentVideo", "length", "length"), + ("EmptyHunyuanLatentVideo", "batch_size", "batch_size"), + ("EmptyMochiLatentVideo", "width", "width"), + ("EmptyMochiLatentVideo", "height", "height"), + ("EmptyMochiLatentVideo", "length", "length"), + ("EmptyLTXVLatentVideo", "width", "width"), + ("EmptyLTXVLatentVideo", "height", "height"), + ("EmptyLTXVLatentVideo", "length", "length"), + ("LatentUpscale", "width", "upscale_width"), + ("LatentUpscale", "height", "upscale_height"), + ("LatentUpscaleBy", "scale_by", "scale_by"), + ("ImageScale", "width", "width"), + ("ImageScale", "height", "height"), + + # ---- Image input ---- + ("LoadImage", "image", "image"), + ("LoadImageMask", "image", "mask_image"), + ("LoadImageOutput", "image", "image"), + ("VHS_LoadVideo", "video", "video"), + ("VHS_LoadAudio", "audio", "audio"), + + # ---- Model selection (sometimes useful to swap per run) ---- + ("CheckpointLoaderSimple", "ckpt_name", "ckpt_name"), + ("CheckpointLoader", "ckpt_name", "ckpt_name"), + ("ImageOnlyCheckpointLoader", "ckpt_name", "ckpt_name"), + ("VAELoader", "vae_name", "vae_name"), + ("UNETLoader", "unet_name", "unet_name"), + ("DiffusionModelLoader", "model_name", "diffusion_model_name"), + ("UpscaleModelLoader", "model_name", "upscale_model_name"), + ("CLIPLoader", "clip_name", "clip_name"), + ("DualCLIPLoader", "clip_name1", "clip_name1"), + ("DualCLIPLoader", "clip_name2", "clip_name2"), + ("ControlNetLoader", "control_net_name", "controlnet_name"), + + # ---- LoRA ---- + ("LoraLoader", "lora_name", "lora_name"), + ("LoraLoader", "strength_model", "lora_strength"), + ("LoraLoader", "strength_clip", "lora_strength_clip"), + ("LoraLoaderModelOnly", "lora_name", "lora_name"), + ("LoraLoaderModelOnly", "strength_model", "lora_strength"), + + # ---- ControlNet ---- + ("ControlNetApply", "strength", "controlnet_strength"), + ("ControlNetApplyAdvanced", "strength", "controlnet_strength"), + ("ControlNetApplyAdvanced", "start_percent", "controlnet_start"), + ("ControlNetApplyAdvanced", "end_percent", "controlnet_end"), + + # ---- IPAdapter ---- + ("IPAdapterAdvanced", "weight", "ipadapter_weight"), + ("IPAdapterAdvanced", "start_at", "ipadapter_start"), + ("IPAdapterAdvanced", "end_at", "ipadapter_end"), + ("IPAdapter", "weight", "ipadapter_weight"), + + # ---- Upscale ---- + ("ImageUpscaleWithModel", "upscale_method", "upscale_method"), + + # ---- AnimateDiff ---- + ("ADE_AnimateDiffLoaderWithContext", "motion_scale", "motion_scale"), + ("ADE_AnimateDiffLoaderGen1", "motion_scale", "motion_scale"), + + # ---- Video / Save ---- + ("VHS_VideoCombine", "frame_rate", "frame_rate"), + ("VHS_VideoCombine", "format", "video_format"), + ("VHS_VideoCombine", "filename_prefix", "filename_prefix"), + ("SaveImage", "filename_prefix", "filename_prefix"), + + # ---- Hunyuan / Wan / LTX video ---- + ("HunyuanVideoSampler", "seed", "seed"), + ("HunyuanVideoSampler", "steps", "steps"), + ("HunyuanVideoSampler", "cfg", "cfg"), + ("WanVideoSampler", "seed", "seed"), + ("WanVideoSampler", "steps", "steps"), + ("WanVideoSampler", "cfg", "cfg"), + ("LTXVScheduler", "max_shift", "max_shift"), + ("LTXVScheduler", "base_shift", "base_shift"), + + # ---- rgthree primitives (often used as user-facing inputs) ---- + ("Seed (rgthree)", "seed", "seed"), + ("Image Comparer (rgthree)", "image_a", "image"), + ("Power Lora Loader (rgthree)", "PowerLoraLoaderHeaderWidget", "_lora_header"), + + # ---- Easy-use / utility primitives ---- + ("PrimitiveNode", "value", "primitive_value"), + ("easy seed", "seed", "seed"), + ("easy positive", "positive", "prompt"), + ("easy negative", "negative", "negative_prompt"), + ("easy fullLoader", "ckpt_name", "ckpt_name"), + ("easy fullLoader", "vae_name", "vae_name"), + ("easy fullLoader", "lora_name", "lora_name"), + ("easy fullLoader", "positive", "prompt"), + ("easy fullLoader", "negative", "negative_prompt"), +] + +# Prompt-like fields whose value should be scanned for embedding references +PROMPT_FIELDS = {"text", "text_g", "text_l", "t5xxl", "clip_l", "positive", "negative"} + +# Pattern matches: embedding:name, embedding:name.pt, embedding:name:1.2, (embedding:name:1.2) +# Word-boundary at start avoids matching things like "no_embedding:foo". +EMBEDDING_REGEX = re.compile( + r"(?:^|[\s,(\[])embedding\s*:\s*([A-Za-z0-9_\-\./\\]+?)(?:\.(?:pt|safetensors|bin))?(?=[\s:,)\(\]]|$)", + re.IGNORECASE, +) + + +# ============================================================================= +# Cloud detection & endpoint routing +# ============================================================================= + +CLOUD_DOMAIN_SUFFIXES = (".comfy.org",) +CLOUD_DOMAIN_EXACT = {"cloud.comfy.org"} + + +def is_cloud_host(host: str) -> bool: + """True if the host points at Comfy Cloud (or staging/preview subdomain).""" + parsed = urlparse(host if "://" in host else f"http://{host}") + hostname = (parsed.hostname or "").lower() + if hostname in CLOUD_DOMAIN_EXACT: + return True + return any(hostname.endswith(s) for s in CLOUD_DOMAIN_SUFFIXES) + + +def build_cloud_aware_url(base: str, path: str, *, force_cloud: bool | None = None) -> str: + """Build a URL that adds /api prefix when targeting Comfy Cloud. + + Local ComfyUI accepts both `/foo` and `/api/foo` for many endpoints. + Cloud requires `/api/foo`. + + `path` should be a path component (e.g. "/prompt") or full path with query + (e.g. "/view?filename=x"). + """ + base = base.rstrip("/") + cloud = is_cloud_host(base) if force_cloud is None else force_cloud + if not path.startswith("/"): + path = "/" + path + if cloud and not path.startswith("/api/"): + path = "/api" + path + return base + path + + +def cloud_endpoint(path: str) -> str: + """Map a cloud endpoint path to its current canonical form. + + Handles known renames documented in the Comfy Cloud API: + /history -> /history_v2 + /models/ -> /experiment/models/ + /models -> /experiment/models + """ + if path.startswith("/history") and not path.startswith("/history_v2"): + return "/history_v2" + path[len("/history"):] + if path.startswith("/models/"): + return "/experiment/models/" + path[len("/models/"):] + if path == "/models": + return "/experiment/models" + return path + + +def resolve_url(base: str, path: str, *, is_cloud: bool | None = None) -> str: + """Top-level URL resolver. Applies cloud rename + /api prefix as needed.""" + cloud = is_cloud_host(base) if is_cloud is None else is_cloud + if cloud: + path = cloud_endpoint(path) + return build_cloud_aware_url(base, path, force_cloud=cloud) + + +# ============================================================================= +# API key resolution +# ============================================================================= + +def resolve_api_key(explicit: str | None) -> str | None: + """Look up API key from CLI flag → env var. Strips whitespace and quotes.""" + val = explicit if explicit else os.environ.get(ENV_API_KEY) + if val is None: + return None + val = val.strip().strip("'\"") + return val or None + + +# ============================================================================= +# HTTP transport +# ============================================================================= + +@dataclass +class HTTPResponse: + status: int + headers: dict[str, str] + body: bytes + url: str # final URL after redirects + + def text(self, encoding: str = "utf-8") -> str: + return self.body.decode(encoding, errors="replace") + + def json(self) -> Any: + return json.loads(self.body.decode("utf-8", errors="replace")) + + +def _sleep_backoff(attempt: int, base: float = RETRY_BASE_DELAY, cap: float = RETRY_MAX_DELAY) -> None: + """Sleep with full-jitter exponential backoff.""" + delay = min(cap, base * (2 ** attempt)) + delay = random.uniform(0, delay) + time.sleep(delay) + + +def http_request( + method: str, + url: str, + *, + headers: dict[str, str] | None = None, + json_body: Any = None, + data: bytes | None = None, + files: dict | None = None, + form: dict | None = None, + timeout: float = DEFAULT_HTTP_TIMEOUT, + follow_redirects: bool = True, + retries: int = DEFAULT_RETRIES, + stream: bool = False, + sink: Path | None = None, +) -> HTTPResponse: + """Single entry point for all HTTP traffic. + + Behavior: + - Retries on connection errors and on HTTP statuses in RETRY_STATUS_CODES, + with exponential backoff + jitter. + - For cross-host redirects, drops Authorization-style headers (so signed + URLs don't leak the API key to S3/CloudFront). + - When `stream=True` and `sink` is a Path, streams the response body to + disk in 64 KiB chunks instead of buffering. + + Either `json_body`, `data`, or `files`+`form` may be supplied (mutually exclusive). + """ + if headers is None: + headers = {} + headers = dict(headers) # copy + headers.setdefault("User-Agent", "hermes-comfyui-skill/5.0") + + if files or form is not None: + # Multipart upload — needs `requests`. The stdlib fallback lacks + # multipart encoding helpers; raise a clear error. + if not HAS_REQUESTS: + raise RuntimeError( + "Multipart upload requires the `requests` package. " + "Install with: pip install requests" + ) + + last_exc: Exception | None = None + for attempt in range(retries): + try: + resp = _http_once( + method=method, url=url, headers=headers, + json_body=json_body, data=data, files=files, form=form, + timeout=timeout, follow_redirects=follow_redirects, + stream=stream, sink=sink, + ) + if resp.status in RETRY_STATUS_CODES and attempt + 1 < retries: + _sleep_backoff(attempt) + continue + return resp + except (TimeoutError, ConnectionError, OSError) as e: + last_exc = e + if attempt + 1 < retries: + _sleep_backoff(attempt) + continue + raise + + # Should not reach here unless retries was 0 + if last_exc: + raise last_exc + raise RuntimeError("http_request: retries exhausted with no response") + + +_SENSITIVE_HEADERS = ("x-api-key", "authorization", "cookie") + + +if HAS_REQUESTS: + class _StripSensitiveOnRedirectSession(requests.Session): + """Session that drops sensitive headers on cross-host redirects. + + `requests` already strips `Authorization` cross-host (rebuild_auth), + but it does NOT strip custom headers like `X-API-Key`. We override + `rebuild_auth` to additionally strip every header in + `_SENSITIVE_HEADERS` when the destination is a different host — + critical when ComfyUI Cloud's `/api/view` redirects to a signed S3 URL. + """ + + def rebuild_auth(self, prepared_request, response): # type: ignore[override] + super().rebuild_auth(prepared_request, response) + try: + old_url = response.request.url + new_url = prepared_request.url + old_host = (urlparse(old_url).hostname or "").lower() + new_host = (urlparse(new_url).hostname or "").lower() + if old_host and new_host and old_host != new_host: + headers = prepared_request.headers + for key in list(headers.keys()): + if key.lower() in _SENSITIVE_HEADERS: + del headers[key] + except Exception: + # Defensive: never let header stripping break a redirect. + pass + + +def _http_once( + *, method: str, url: str, headers: dict[str, str], + json_body: Any, data: bytes | None, files: dict | None, form: dict | None, + timeout: float, follow_redirects: bool, + stream: bool, sink: Path | None, +) -> HTTPResponse: + """One HTTP attempt. No retry.""" + if HAS_REQUESTS: + kwargs: dict[str, Any] = { + "method": method, "url": url, "headers": headers, + "timeout": timeout, "allow_redirects": follow_redirects, + } + if json_body is not None: + kwargs["json"] = json_body + elif data is not None: + kwargs["data"] = data + elif files is not None or form is not None: + kwargs["files"] = files + kwargs["data"] = form + if stream: + kwargs["stream"] = True + + # Use the subclass that strips sensitive headers cross-host + with _StripSensitiveOnRedirectSession() as s: + try: + r = s.request(**kwargs) + if stream and sink is not None: + sink.parent.mkdir(parents=True, exist_ok=True) + with sink.open("wb") as f: + for chunk in r.iter_content(DOWNLOAD_CHUNK_SIZE): + if chunk: + f.write(chunk) + body = b"" # already drained + else: + body = r.content + return HTTPResponse( + status=r.status_code, + headers={k: v for k, v in r.headers.items()}, + body=body, + url=r.url, + ) + except requests.exceptions.RequestException as e: + # Convert to TimeoutError / ConnectionError so the retry loop + # picks them up uniformly with the stdlib path. + if isinstance(e, requests.exceptions.Timeout): + raise TimeoutError(str(e)) from e + raise ConnectionError(str(e)) from e + + # ---------- stdlib fallback ---------- + if json_body is not None: + body_bytes = json.dumps(json_body).encode("utf-8") + headers.setdefault("Content-Type", "application/json") + else: + body_bytes = data + req = urllib.request.Request(url, data=body_bytes, headers=headers, method=method) + + # urllib follows redirects by default. We need to: + # 1) intercept cross-host redirects and drop X-API-Key + # 2) optionally NOT follow redirects when follow_redirects=False + class _RedirectHandler(urllib.request.HTTPRedirectHandler): + def __init__(self, original_host: str, follow: bool): + self.original_host = original_host + self.follow = follow + + def redirect_request(self, req2, fp, code, msg, hdrs, newurl): + if not self.follow: + return None + new_host = (urlparse(newurl).hostname or "").lower() + if new_host != self.original_host: + # Build a new request with cleaned headers + clean_headers = { + k: v for k, v in req2.header_items() + if k.lower() not in ("x-api-key", "authorization", "cookie") + } + new_req = urllib.request.Request(newurl, headers=clean_headers, method="GET") + return new_req + return super().redirect_request(req2, fp, code, msg, hdrs, newurl) + + original_host = (urlparse(url).hostname or "").lower() + opener = urllib.request.build_opener(_RedirectHandler(original_host, follow_redirects)) + + try: + resp = opener.open(req, timeout=timeout) + except urllib.error.HTTPError as e: + return HTTPResponse( + status=e.code, + headers=dict(e.headers) if e.headers else {}, + body=e.read() or b"", + url=getattr(e, "url", url), + ) + + final_url = resp.geturl() + final_status = resp.status + final_headers = dict(resp.headers) + + if stream and sink is not None: + sink.parent.mkdir(parents=True, exist_ok=True) + with sink.open("wb") as f: + while True: + chunk = resp.read(DOWNLOAD_CHUNK_SIZE) + if not chunk: + break + f.write(chunk) + return HTTPResponse(status=final_status, headers=final_headers, body=b"", url=final_url) + + return HTTPResponse(status=final_status, headers=final_headers, body=resp.read(), url=final_url) + + +def http_get(url: str, **kwargs: Any) -> HTTPResponse: + return http_request("GET", url, **kwargs) + + +def http_post(url: str, **kwargs: Any) -> HTTPResponse: + return http_request("POST", url, **kwargs) + + +# ============================================================================= +# Workflow validation & helpers +# ============================================================================= + +def is_api_format(workflow: Any) -> bool: + """API format = top-level dict where each value has `class_type`.""" + if not isinstance(workflow, dict): + return False + if "nodes" in workflow and "links" in workflow: + return False + for v in workflow.values(): + if isinstance(v, dict) and "class_type" in v: + return True + return False + + +def unwrap_workflow(payload: Any) -> dict: + """Unwrap common wrapper variants. Returns API-format workflow or raises ValueError.""" + if isinstance(payload, dict) and is_api_format(payload): + return payload + # Some files wrap workflow under "prompt" key (e.g. saved /prompt payloads) + if isinstance(payload, dict) and "prompt" in payload and is_api_format(payload["prompt"]): + return payload["prompt"] + # Editor format + if isinstance(payload, dict) and "nodes" in payload and "links" in payload: + raise ValueError( + "Workflow is in editor format (has top-level 'nodes' and 'links' arrays). " + "Re-export from ComfyUI using 'Workflow → Export (API)' (newer UI) " + "or 'Save (API Format)' (older UI)." + ) + raise ValueError( + "Workflow is not in API format. Each top-level entry must have a 'class_type' field." + ) + + +def is_link(value: Any) -> bool: + """True if `value` is a [node_id, output_index] connection (length-2 list).""" + return ( + isinstance(value, list) + and len(value) == 2 + and isinstance(value[0], str) + and isinstance(value[1], int) + ) + + +def iter_nodes(workflow: dict) -> Iterator[tuple[str, dict]]: + """Yield (node_id, node) for each valid API-format node.""" + for node_id, node in workflow.items(): + if isinstance(node, dict) and "class_type" in node: + yield node_id, node + + +def iter_model_deps(workflow: dict) -> Iterator[dict]: + """Yield {node_id, class_type, field, value, folder} for each model dependency.""" + for node_id, node in iter_nodes(workflow): + cls = node["class_type"] + if cls not in MODEL_LOADERS: + continue + inputs = node.get("inputs", {}) or {} + for field_name, folder in MODEL_LOADERS[cls]: + val = inputs.get(field_name) + if val and isinstance(val, str) and not is_link(val): + yield { + "node_id": node_id, + "class_type": cls, + "field": field_name, + "value": val, + "folder": folder, + } + + +def iter_embedding_refs(workflow: dict) -> Iterator[tuple[str, str]]: + """Yield (node_id, embedding_name) for every embedding mention in prompts.""" + for node_id, node in iter_nodes(workflow): + inputs = node.get("inputs", {}) or {} + for field_name, val in inputs.items(): + if field_name not in PROMPT_FIELDS: + continue + if not isinstance(val, str): + continue + for m in EMBEDDING_REGEX.finditer(val): + yield node_id, m.group(1) + + +# ============================================================================= +# Path safety +# ============================================================================= + +def safe_path_join(base: Path, *parts: str) -> Path: + """Join paths, raising if the result escapes `base`. + + Server-supplied filenames may contain `../` etc. This guards against + path-traversal attacks when downloading outputs. + """ + base_resolved = base.resolve() + candidate = base.joinpath(*parts).resolve() + try: + candidate.relative_to(base_resolved) + except ValueError as e: + raise ValueError( + f"Refusing path traversal: {candidate} is outside {base_resolved}" + ) from e + return candidate + + +def media_type_from_filename(filename: str) -> str: + ext = Path(filename).suffix.lower() + if ext in (".mp4", ".webm", ".avi", ".mov", ".mkv", ".gif", ".webp"): + return "video" + if ext in (".wav", ".mp3", ".flac", ".ogg", ".m4a"): + return "audio" + if ext in (".glb", ".obj", ".ply", ".gltf"): + return "3d" + if ext in (".json", ".txt", ".md"): + return "text" + return "image" + + +def looks_like_video_workflow(workflow: dict) -> bool: + """Used to bump default timeout for video workflows.""" + for _, node in iter_nodes(workflow): + if node["class_type"] in SLOW_OUTPUT_NODES: + return True + if node["class_type"].lower().startswith(("animatediff", "ade_", "wanvideo", "hunyuanvideo", "ltxvideo", "cogvideo")): + return True + return False + + +# ============================================================================= +# Seed handling +# ============================================================================= + +# ComfyUI's max seed range. Many UIs treat `-1` as "randomize on submit". +SEED_MAX = 2**63 - 1 +SEED_MIN = 0 + + +def coerce_seed(value: Any) -> int: + """Convert -1 or None to a fresh random seed; otherwise return int(value). + + Accepts numeric -1 OR string "-1" (both treated as "randomize"). Other + parse failures raise TypeError/ValueError for the caller to surface. + """ + if value is None: + return random.randint(SEED_MIN, SEED_MAX) + # Stringly-typed -1 from CLI / JSON should also randomize + if isinstance(value, str) and value.strip() == "-1": + return random.randint(SEED_MIN, SEED_MAX) + if value == -1: + return random.randint(SEED_MIN, SEED_MAX) + return int(value) + + +# ============================================================================= +# Cloud model-list normalization +# ============================================================================= + +def parse_model_list(payload: Any) -> set[str]: + """Normalize model-list responses from local ComfyUI vs Comfy Cloud. + + Local: `["a.safetensors", "b.safetensors"]` + Cloud: `[{"name": "a.safetensors", "pathIndex": 0}, ...]` + """ + if not isinstance(payload, list): + return set() + out: set[str] = set() + for item in payload: + if isinstance(item, str): + out.add(item) + elif isinstance(item, dict): + name = item.get("name") or item.get("filename") or item.get("path") + if isinstance(name, str): + out.add(name) + return out + + +# ============================================================================= +# Misc utilities +# ============================================================================= + +def new_client_id() -> str: + return str(uuid.uuid4()) + + +def fmt_kv(d: dict) -> str: + """Pretty key=value for log lines.""" + return " ".join(f"{k}={v!r}" for k, v in d.items()) + + +def emit_json(obj: Any, *, indent: int = 2) -> None: + """Print JSON to stdout. Centralised so behavior can be tweaked (e.g., --raw).""" + print(json.dumps(obj, indent=indent, default=str)) + + +def log(msg: str) -> None: + """stderr log with consistent prefix (so JSON stdout stays clean).""" + print(f"[comfyui-skill] {msg}", file=sys.stderr) diff --git a/skills/creative/comfyui/scripts/auto_fix_deps.py b/skills/creative/comfyui/scripts/auto_fix_deps.py new file mode 100755 index 00000000000..788bf8e9e3b --- /dev/null +++ b/skills/creative/comfyui/scripts/auto_fix_deps.py @@ -0,0 +1,225 @@ +#!/usr/bin/env python3 +""" +auto_fix_deps.py — Run check_deps.py, then attempt to install whatever is missing. + +For local servers: + - Missing custom nodes → `comfy node install ` + - Missing models → `comfy model download` (only if a URL is supplied via + --model-source-file or detected via well-known names) + +For cloud: prints what would be needed but cannot install (cloud preinstalls +custom nodes and most models server-side; if something genuinely isn't there, +ask Comfy support). + +This is conservative: it never installs without an explicit URL for models +(downloading the wrong model is hard to undo). Custom nodes from the registry +are auto-installed by name. + +Usage: + python3 auto_fix_deps.py workflow_api.json + python3 auto_fix_deps.py workflow_api.json --models-from-file urls.json + python3 auto_fix_deps.py workflow_api.json --dry-run +""" + +from __future__ import annotations + +import argparse +import json +import shutil +import subprocess +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, log, resolve_api_key, +) +from check_deps import check_deps # noqa: E402 +from _common import unwrap_workflow # noqa: E402 + + +def comfy_cli_available() -> str | None: + """Return command prefix for comfy-cli, or None.""" + if shutil.which("comfy"): + return "comfy" + if shutil.which("uvx"): + return "uvx --from comfy-cli comfy" + return None + + +def run_cmd(cmd: list[str], *, dry_run: bool = False) -> tuple[int, str]: + if dry_run: + return 0, "[dry-run]" + log(f"$ {' '.join(cmd)}") + proc = subprocess.run(cmd, capture_output=True, text=True, check=False) + out = (proc.stdout or "") + (proc.stderr or "") + return proc.returncode, out + + +def install_node(package: str, *, dry_run: bool = False, comfy_cmd: str = "comfy") -> bool: + cmd = comfy_cmd.split() + ["--skip-prompt", "node", "install", package] + code, _ = run_cmd(cmd, dry_run=dry_run) + return code == 0 + + +def install_model(url: str, folder: str, filename: str | None = None, + *, dry_run: bool = False, comfy_cmd: str = "comfy", + hf_token: str | None = None, civitai_token: str | None = None) -> bool: + cmd = comfy_cmd.split() + [ + "--skip-prompt", "model", "download", + "--url", url, + "--relative-path", f"models/{folder}", + ] + if filename: + cmd.extend(["--filename", filename]) + if hf_token: + cmd.extend(["--set-hf-api-token", hf_token]) + if civitai_token: + cmd.extend(["--set-civitai-api-token", civitai_token]) + code, _ = run_cmd(cmd, dry_run=dry_run) + return code == 0 + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Run check_deps and install whatever is missing") + p.add_argument("workflow") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--models-from-file", + help="JSON file mapping {model_filename: download_url} for models that need install") + p.add_argument("--hf-token", help="HuggingFace token for downloads") + p.add_argument("--civitai-token", help="CivitAI token for downloads") + p.add_argument("--dry-run", action="store_true", + help="Show what would be installed without doing it") + p.add_argument("--no-restart", action="store_true", + help="Don't suggest restarting the server after node install") + args = p.parse_args(argv) + + api_key = resolve_api_key(args.api_key) + + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + workflow = unwrap_workflow(json.load(f)) + except (ValueError, json.JSONDecodeError) as e: + emit_json({"error": str(e)}) + return 1 + + report = check_deps(workflow, host=args.host, api_key=api_key) + + if report["is_ready"]: + emit_json({"status": "ready", "report": report}) + return 0 + + if report["is_cloud"]: + emit_json({ + "status": "cannot_fix_cloud", + "reason": "Comfy Cloud preinstalls nodes; if something is genuinely missing, contact support.", + "report": report, + }) + return 1 + + comfy_cmd = comfy_cli_available() + if not comfy_cmd: + emit_json({ + "status": "cannot_fix", + "reason": "comfy-cli not on PATH; install with `pip install comfy-cli` or `pipx install comfy-cli`", + "report": report, + }) + return 1 + + actions: list[dict] = [] + failures: list[dict] = [] + + # ---- Install missing custom nodes ---- + seen_packages: set[str] = set() + for entry in report["missing_nodes"]: + cmd = entry.get("fix_command", "") + if cmd.startswith("comfy node install "): + package = cmd.split(" ")[-1] + if package in seen_packages: + continue + seen_packages.add(package) + ok = install_node(package, dry_run=args.dry_run, comfy_cmd=comfy_cmd) + (actions if ok else failures).append({ + "kind": "node", "package": package, "node_class": entry["class_type"], + "ok": ok, + }) + else: + failures.append({ + "kind": "node", "node_class": entry["class_type"], + "ok": False, "reason": "No registry mapping known. " + entry.get("fix_hint", ""), + }) + + # ---- Install missing models (only when URL provided) ---- + sources: dict[str, str] = {} + if args.models_from_file: + try: + sources = json.loads(Path(args.models_from_file).read_text()) + except (OSError, json.JSONDecodeError) as e: + log(f"Could not read --models-from-file: {e}") + + for entry in report["missing_models"]: + filename = entry["value"] + url = sources.get(filename) + if not url: + failures.append({ + "kind": "model", "filename": filename, "folder": entry["folder"], + "ok": False, "reason": "No URL provided in --models-from-file. " + "Refusing to guess.", + }) + continue + ok = install_model( + url, entry["folder"], filename, + dry_run=args.dry_run, comfy_cmd=comfy_cmd, + hf_token=args.hf_token, civitai_token=args.civitai_token, + ) + (actions if ok else failures).append({ + "kind": "model", "filename": filename, "folder": entry["folder"], + "url": url, "ok": ok, + }) + + # ---- Embeddings ---- + for entry in report["missing_embeddings"]: + emb_name = entry["embedding_name"] + # Try common extensions in user-supplied source map + url = (sources.get(f"{emb_name}.pt") + or sources.get(f"{emb_name}.safetensors") + or sources.get(emb_name)) + if not url: + failures.append({ + "kind": "embedding", "name": emb_name, + "ok": False, "reason": "No URL provided in --models-from-file.", + }) + continue + target_filename = ( + f"{emb_name}.safetensors" if url.endswith(".safetensors") + else f"{emb_name}.pt" + ) + ok = install_model( + url, "embeddings", target_filename, + dry_run=args.dry_run, comfy_cmd=comfy_cmd, + hf_token=args.hf_token, civitai_token=args.civitai_token, + ) + (actions if ok else failures).append({ + "kind": "embedding", "name": emb_name, "url": url, "ok": ok, + }) + + needs_restart = any(a["kind"] == "node" and a.get("ok") for a in actions) + + emit_json({ + "status": "fixed" if not failures else "partial", + "actions_taken": actions, + "failures": failures, + "needs_server_restart": needs_restart and not args.no_restart, + "restart_hint": "comfy stop && comfy launch --background", + "dry_run": args.dry_run, + }) + return 0 if not failures else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/creative/comfyui/scripts/check_deps.py b/skills/creative/comfyui/scripts/check_deps.py old mode 100644 new mode 100755 index cc67de19104..ec415346c33 --- a/skills/creative/comfyui/scripts/check_deps.py +++ b/skills/creative/comfyui/scripts/check_deps.py @@ -1,182 +1,417 @@ #!/usr/bin/env python3 """ -check_deps.py — Check if a ComfyUI workflow's dependencies (custom nodes and models) are installed. +check_deps.py — Verify a ComfyUI workflow's dependencies (custom nodes, models, +embeddings) against a running server. -Queries the running ComfyUI server for installed nodes (via /object_info) and models -(via /models/{folder}), then diffs against what the workflow requires. +Improvements over v1: + - Cloud-aware endpoint mapping (handles `/api/experiment/models/{folder}` and + `/api/object_info` variants verified against live cloud API) + - Distinguishes 200-empty (genuinely no models in folder) vs 404 + (folder doesn't exist) vs 403 (auth/tier issue) — no silent passes + - Outputs concrete remediation commands (e.g. `comfy node install `) + when nodes are missing + - Detects embedding references inside prompt strings as model deps + - Skips check on cloud free tier `/api/object_info` (403) without false alarm + - Accepts API key from CLI flag OR $COMFY_CLOUD_API_KEY env var Usage: python3 check_deps.py workflow_api.json python3 check_deps.py workflow_api.json --host 127.0.0.1 --port 8188 - python3 check_deps.py workflow_api.json --host https://cloud.comfy.org --api-key KEY + python3 check_deps.py workflow_api.json --host https://cloud.comfy.org -Output format: - { - "is_ready": true/false, - "missing_nodes": ["NodeClassName", ...], - "missing_models": [{"class_type": "...", "field": "...", "value": "...", "folder": "..."}], - "installed_nodes_count": 123, - "required_nodes": ["KSampler", "CLIPTextEncode", ...] - } - -Requires: Python 3.10+, requests (or urllib as fallback) +Stdlib-only. Python 3.10+. """ +from __future__ import annotations + +import argparse import json import sys -import argparse from pathlib import Path -from urllib.parse import urljoin, urlparse -try: - import requests - HAS_REQUESTS = True -except ImportError: - HAS_REQUESTS = False - import urllib.request - import urllib.error +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, + emit_json, folder_aliases_for, http_get, is_cloud_host, + iter_embedding_refs, iter_model_deps, iter_nodes, parse_model_list, + resolve_api_key, resolve_url, unwrap_workflow, +) -# Known model loader node types and which folder they reference -MODEL_LOADERS = { - "CheckpointLoaderSimple": ("ckpt_name", "checkpoints"), - "CheckpointLoader": ("ckpt_name", "checkpoints"), - "unCLIPCheckpointLoader": ("ckpt_name", "checkpoints"), - "LoraLoader": ("lora_name", "loras"), - "LoraLoaderModelOnly": ("lora_name", "loras"), - "VAELoader": ("vae_name", "vae"), - "ControlNetLoader": ("control_net_name", "controlnet"), - "DiffControlNetLoader": ("control_net_name", "controlnet"), - "CLIPLoader": ("clip_name", "clip"), - "DualCLIPLoader": ("clip_name1", "clip"), - "UNETLoader": ("unet_name", "unet"), - "DiffusionModelLoader": ("model_name", "diffusion_models"), - "UpscaleModelLoader": ("model_name", "upscale_models"), - "CLIPVisionLoader": ("clip_name", "clip_vision"), - "StyleModelLoader": ("style_model_name", "style_models"), - "GLIGENLoader": ("gligen_name", "gligen"), - "HypernetworkLoader": ("hypernetwork_name", "hypernetworks"), + +# Known node → custom-node-package map. When a workflow needs a node we don't +# recognize, suggesting the right `comfy node install ...` makes the difference +# between a working agent and a stuck one. +NODE_TO_PACKAGE: dict[str, str] = { + # rgthree + "Power Lora Loader (rgthree)": "rgthree-comfy", + "Image Comparer (rgthree)": "rgthree-comfy", + "Seed (rgthree)": "rgthree-comfy", + "Reroute (rgthree)": "rgthree-comfy", + "Display Any (rgthree)": "rgthree-comfy", + # Impact pack + "FaceDetailer": "comfyui-impact-pack", + "DetailerForEach": "comfyui-impact-pack", + "UltralyticsDetectorProvider": "comfyui-impact-pack", + "BboxDetectorSEGS": "comfyui-impact-pack", + "SAMLoader": "comfyui-impact-pack", + "ImpactWildcardProcessor": "comfyui-impact-pack", + # Was Node Suite + "Image Save": "was-node-suite-comfyui", + "Number Counter": "was-node-suite-comfyui", + "Text String": "was-node-suite-comfyui", + # easy-use + "easy fullLoader": "comfyui-easy-use", + "easy positive": "comfyui-easy-use", + "easy negative": "comfyui-easy-use", + "easy seed": "comfyui-easy-use", + "easy imageSave": "comfyui-easy-use", + # Video Helper Suite + "VHS_VideoCombine": "comfyui-videohelpersuite", + "VHS_LoadVideo": "comfyui-videohelpersuite", + "VHS_LoadAudio": "comfyui-videohelpersuite", + # AnimateDiff + "ADE_AnimateDiffLoaderWithContext": "comfyui-animatediff-evolved", + "ADE_AnimateDiffLoaderGen1": "comfyui-animatediff-evolved", + "ADE_LoadAnimateDiffModel": "comfyui-animatediff-evolved", + # ControlNet aux + "Canny": "comfyui_controlnet_aux", + "DWPreprocessor": "comfyui_controlnet_aux", + "OpenposePreprocessor": "comfyui_controlnet_aux", + "DepthAnythingPreprocessor": "comfyui_controlnet_aux", + # IPAdapter Plus + "IPAdapterAdvanced": "comfyui_ipadapter_plus", + "IPAdapterUnifiedLoader": "comfyui_ipadapter_plus", + "IPAdapterModelLoader": "comfyui_ipadapter_plus", + "IPAdapterInsightFaceLoader": "comfyui_ipadapter_plus", + # InstantID + "InstantIDModelLoader": "comfyui_instantid", + "ApplyInstantID": "comfyui_instantid", + # Comfy essentials + "GetImageSize+": "comfyui-essentials", + "ImageBatchMultiple+": "comfyui-essentials", + # pysssss + "ShowText|pysssss": "comfyui-custom-scripts", + "PreviewImage|pysssss": "comfyui-custom-scripts", + # SUPIR + "SUPIR_Upscale": "comfyui-supir", + "SUPIR_first_stage": "comfyui-supir", + # GGUF + "UNETLoaderGGUF": "comfyui-gguf", + "DualCLIPLoaderGGUF": "comfyui-gguf", + # Florence2 + "Florence2Run": "comfyui-florence2", + # WAS + "Image Filter Adjustments": "was-node-suite-comfyui", + # Photomaker + "PhotoMakerLoader": "comfyui-photomaker-plus", + # Wan / Hunyuan video + "WanVideoSampler": "comfyui-wanvideowrapper", + "WanVideoModelLoader": "comfyui-wanvideowrapper", + "HunyuanVideoSampler": "comfyui-hunyuanvideowrapper", + "HunyuanVideoModelLoader": "comfyui-hunyuanvideowrapper", } -def http_get(url: str, headers: dict = None) -> tuple: - """GET request, returns (status_code, body_text).""" - if HAS_REQUESTS: - r = requests.get(url, headers=headers or {}, timeout=30) - return r.status_code, r.text - else: - req = urllib.request.Request(url, headers=headers or {}) +def fetch_object_info(url: str, headers: dict) -> tuple[set[str] | None, dict | None]: + """Returns (installed_node_set, error_info). Error info is a dict if we + couldn't query (e.g. cloud free tier), else None. + """ + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status == 200: try: - resp = urllib.request.urlopen(req, timeout=30) - return resp.status, resp.read().decode() - except urllib.error.HTTPError as e: - return e.code, e.read().decode() + data = r.json() + if isinstance(data, dict): + return set(data.keys()), None + except Exception: + pass + return None, {"http_status": 200, "reason": "non-dict response"} + if r.status == 403: + try: + body = r.json() + except Exception: + body = {"raw": r.text()[:200]} + return None, {"http_status": 403, "reason": "forbidden", "body": body} + if r.status == 404: + return None, {"http_status": 404, "reason": "endpoint not found"} + return None, {"http_status": r.status, "reason": "unexpected", "body": r.text()[:200]} -def check_deps(workflow_path: str, host: str = "http://127.0.0.1:8188", api_key: str = None): - """Check workflow dependencies against a running server.""" - # Load workflow - with open(workflow_path) as f: - workflow = json.load(f) +def _fetch_one_folder( + base: str, folder: str, headers: dict, *, is_cloud: bool, +) -> tuple[set[str] | None, dict | None]: + """Single-folder fetch, no aliasing. Returns (installed_set, error_info).""" + url = resolve_url(base, f"/models/{folder}", is_cloud=is_cloud) + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status == 200: + try: + return parse_model_list(r.json()), None + except Exception: + return set(), {"http_status": 200, "reason": "non-list response"} + if r.status == 404: + body_text = r.text() + try: + body = r.json() + except Exception: + body = {"raw": body_text[:200]} + code = body.get("code") if isinstance(body, dict) else None + if code == "folder_not_found": + # Folder is genuinely empty/missing on server — not the same as + # "endpoint missing". Return empty set with informational error. + return set(), {"http_status": 404, "reason": "folder_empty_or_unknown", "body": body} + return None, {"http_status": 404, "reason": "endpoint not found", "body": body} + if r.status == 403: + try: + body = r.json() + except Exception: + body = {} + return None, {"http_status": 403, "reason": "forbidden", "body": body} + return None, {"http_status": r.status, "reason": "unexpected"} - # Validate format - if "nodes" in workflow and "links" in workflow: - return {"error": "Workflow is in editor format, not API format."} - headers = {} +def fetch_models_for_folder( + base: str, folder: str, headers: dict, *, is_cloud: bool, +) -> tuple[set[str] | None, dict | None]: + """Fetch installed models for a folder, trying aliases. + + Folder renames over time (e.g. unet → diffusion_models, clip → text_encoders) + mean a workflow asking for a model in `unet` may need to look in + `diffusion_models`. We union models from every reachable alias. + + Returns (combined_set | None, last_error | None). + """ + aliases = folder_aliases_for(folder) + combined: set[str] = set() + any_success = False + last_err: dict | None = None + for alias in aliases: + models, err = _fetch_one_folder(base, alias, headers, is_cloud=is_cloud) + if models is not None: + combined.update(models) + any_success = True + last_err = None + else: + last_err = err + if not any_success: + return None, last_err + return combined, None + + +def fetch_embeddings(base: str, headers: dict, *, is_cloud: bool) -> tuple[set[str] | None, dict | None]: + """Local ComfyUI exposes /embeddings; cloud uses /experiment/models/embeddings.""" + if is_cloud: + return fetch_models_for_folder(base, "embeddings", headers, is_cloud=True) + # Local: dedicated /embeddings returns a flat list of names + r = http_get(resolve_url(base, "/embeddings", is_cloud=False), headers=headers, retries=2) + if r.status == 200: + try: + data = r.json() + if isinstance(data, list): + # Strip extensions from the registered names since prompt syntax + # usually omits them ("embedding:goodvibes" vs "goodvibes.pt") + names = set() + for n in data: + if isinstance(n, str): + names.add(n) + # Also store stem for fuzzy matching + names.add(Path(n).stem) + return names, None + except Exception: + pass + return None, {"http_status": r.status, "reason": "unexpected"} + + +def normalize_for_match(name: str) -> set[str]: + """Generate matching variants of a model name (with/without extension, slashes, etc.)""" + s = {name} + s.add(Path(name).stem) + s.add(Path(name).name) + # ComfyUI sometimes strips/keeps the leading folder + if "/" in name or "\\" in name: + flat = name.replace("\\", "/").split("/")[-1] + s.add(flat) + s.add(Path(flat).stem) + return {x for x in s if x} + + +def model_present(needed: str, installed: set[str]) -> bool: + if not installed: + return False + needed_variants = normalize_for_match(needed) + installed_norm: set[str] = set() + for inst in installed: + installed_norm.update(normalize_for_match(inst)) + return bool(needed_variants & installed_norm) + + +def suggest_install_command(node_class: str) -> str | None: + pkg = NODE_TO_PACKAGE.get(node_class) + if pkg: + return f"comfy node install {pkg}" + return None + + +def check_deps( + workflow: dict, host: str, *, api_key: str | None = None, +) -> dict: + headers: dict[str, str] = {} if api_key: headers["X-API-Key"] = api_key - parsed_host = urlparse(host) - hostname = (parsed_host.hostname or "").lower() - is_cloud_host = hostname == "cloud.comfy.org" or hostname.endswith(".cloud.comfy.org") - is_cloud = is_cloud_host or api_key is not None + is_cloud = is_cloud_host(host) base = host.rstrip("/") - # Get installed node types - object_info_url = f"{base}/api/object_info" if is_cloud else f"{base}/object_info" - status, body = http_get(object_info_url, headers) - if status != 200: - return {"error": f"Cannot reach server at {host}. Is ComfyUI running? HTTP {status}"} + # ---- 1. Required nodes ---- + required_nodes: set[str] = set() + for _, node in iter_nodes(workflow): + required_nodes.add(node["class_type"]) - installed_nodes = set(json.loads(body).keys()) + object_info_url = resolve_url(base, "/object_info", is_cloud=is_cloud) + installed_nodes, obj_err = fetch_object_info(object_info_url, headers) - # Find required node types from workflow - required_nodes = set() - for node_id, node in workflow.items(): - if isinstance(node, dict) and "class_type" in node: - required_nodes.add(node["class_type"]) + missing_nodes: list[dict] = [] + node_check_skipped = False + if installed_nodes is None: + # Couldn't query (e.g. cloud free tier). Don't false-alarm; mark skipped. + node_check_skipped = True + else: + for cls in sorted(required_nodes): + if cls not in installed_nodes: + entry = {"class_type": cls} + cmd = suggest_install_command(cls) + if cmd: + entry["fix_command"] = cmd + else: + entry["fix_hint"] = ( + "Search https://registry.comfy.org or " + "use ComfyUI-Manager UI to find the package providing this node." + ) + missing_nodes.append(entry) - missing_nodes = sorted(required_nodes - installed_nodes) + # ---- 2. Required models ---- + model_cache: dict[str, tuple[set[str] | None, dict | None]] = {} + missing_models: list[dict] = [] + folder_errors: dict[str, dict] = {} - # Check model dependencies - missing_models = [] - model_cache = {} # folder → set of installed model filenames - - for node_id, node in workflow.items(): - if not isinstance(node, dict) or "class_type" not in node: - continue - class_type = node["class_type"] - if class_type not in MODEL_LOADERS: - continue - - field, folder = MODEL_LOADERS[class_type] - inputs = node.get("inputs", {}) - model_name = inputs.get(field) - - if not model_name or not isinstance(model_name, str): - continue - - # Fetch installed models for this folder (cached) + for dep in iter_model_deps(workflow): + folder = dep["folder"] if folder not in model_cache: - models_url = f"{base}/api/models/{folder}" if is_cloud else f"{base}/models/{folder}" - s, b = http_get(models_url, headers) - if s == 200: - model_cache[folder] = set(json.loads(b)) - else: - model_cache[folder] = set() + model_cache[folder] = fetch_models_for_folder( + base, folder, headers, is_cloud=is_cloud, + ) + installed, err = model_cache[folder] + if installed is None: + # Couldn't enumerate this folder — record once + folder_errors.setdefault(folder, err or {}) + # Don't flag as missing (we don't know); the folder_errors block surfaces this + continue + if not model_present(dep["value"], installed): + entry = dict(dep) + entry["fix_hint"] = ( + f"comfy model download --url --relative-path models/{folder} " + f"--filename {dep['value']!r}" + ) + missing_models.append(entry) - if model_name not in model_cache[folder]: - missing_models.append({ - "node_id": node_id, - "class_type": class_type, - "field": field, - "value": model_name, - "folder": folder, + # ---- 3. Embedding refs in prompts ---- + emb_installed, emb_err = fetch_embeddings(base, headers, is_cloud=is_cloud) + missing_embeddings: list[dict] = [] + seen_emb: set[tuple[str, str]] = set() + for nid, emb_name in iter_embedding_refs(workflow): + if (nid, emb_name) in seen_emb: + continue + seen_emb.add((nid, emb_name)) + if emb_installed is None: + # Couldn't enumerate — skip silently here, surface the error in the + # folder_errors block + continue + if not model_present(emb_name, emb_installed): + missing_embeddings.append({ + "node_id": nid, + "embedding_name": emb_name, + "folder": "embeddings", + "fix_hint": ( + f"Download {emb_name}.pt or .safetensors and place in " + f"models/embeddings/, or `comfy model download --url " + f"--relative-path models/embeddings`" + ), }) - is_ready = len(missing_nodes) == 0 and len(missing_models) == 0 + if emb_err and emb_installed is None: + folder_errors.setdefault("embeddings", emb_err) + + is_ready = ( + not node_check_skipped + and not missing_nodes + and not missing_models + and not missing_embeddings + ) return { "is_ready": is_ready, + "node_check_skipped": node_check_skipped, + "node_check_skip_reason": obj_err if node_check_skipped else None, "missing_nodes": missing_nodes, "missing_models": missing_models, - "installed_nodes_count": len(installed_nodes), + "missing_embeddings": missing_embeddings, + "folder_errors": folder_errors, + # 0 is a legitimate count (e.g. empty server). Use None only when not queried. + "installed_node_count": len(installed_nodes) if installed_nodes is not None else None, + "required_node_count": len(required_nodes), "required_nodes": sorted(required_nodes), + "host": base, + "is_cloud": is_cloud, } -def main(): - parser = argparse.ArgumentParser(description="Check ComfyUI workflow dependencies") - parser.add_argument("workflow", help="Path to workflow API JSON file") - parser.add_argument("--host", default="http://127.0.0.1:8188", help="ComfyUI server URL") - parser.add_argument("--port", type=int, help="Server port (overrides --host port)") - parser.add_argument("--api-key", help="API key for cloud") - args = parser.parse_args() +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Check ComfyUI workflow dependencies against a running server") + p.add_argument("workflow", help="Path to workflow API JSON file") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL") + p.add_argument("--port", type=int, help="Server port (overrides --host port)") + p.add_argument("--api-key", help=f"API key for cloud (or set ${ENV_API_KEY} env var)") + p.add_argument("--strict", action="store_true", + help="Exit non-zero if node check is skipped (e.g. on cloud free tier)") + args = p.parse_args(argv) - # Handle --port override host = args.host - if args.port and ":" not in host.split("//")[-1]: - host = f"{host}:{args.port}" + if args.port is not None: + # Strip any port from host and append --port + from urllib.parse import urlparse, urlunparse + parsed = urlparse(host if "://" in host else f"http://{host}") + new_netloc = f"{parsed.hostname}:{args.port}" + host = urlunparse(parsed._replace(netloc=new_netloc)) - result = check_deps(args.workflow, host=host, api_key=args.api_key) - print(json.dumps(result, indent=2)) + api_key = resolve_api_key(args.api_key) - if result.get("error"): - sys.exit(1) - if not result.get("is_ready", False): - sys.exit(1) - sys.exit(0) + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow file not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + payload = json.load(f) + workflow = unwrap_workflow(payload) + except ValueError as e: + emit_json({"error": str(e)}) + return 1 + except json.JSONDecodeError as e: + emit_json({"error": f"Invalid JSON: {e}"}) + return 1 + + try: + result = check_deps(workflow, host=host, api_key=api_key) + except Exception as e: + emit_json({"error": f"Dep check failed: {e}", "host": host}) + return 1 + + emit_json(result) + + if not result["is_ready"]: + return 1 + if args.strict and result["node_check_skipped"]: + return 1 + return 0 if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/skills/creative/comfyui/scripts/comfyui_setup.sh b/skills/creative/comfyui/scripts/comfyui_setup.sh index cb9d17a10ab..dd0369833dc 100755 --- a/skills/creative/comfyui/scripts/comfyui_setup.sh +++ b/skills/creative/comfyui/scripts/comfyui_setup.sh @@ -1,113 +1,263 @@ #!/usr/bin/env bash # ComfyUI Setup — Install, launch, and verify using the official comfy-cli. -# Usage: bash scripts/comfyui_setup.sh [--nvidia|--amd|--m-series|--cpu] # -# If no flag is passed, runs hardware_check.py to detect the right one -# automatically, and refuses to install locally when the verdict is "cloud" -# (no usable GPU, too little VRAM, Intel Mac, etc.) — pointing the user -# at Comfy Cloud instead. +# Improvements over v1: +# - Prefers `pipx` / `uvx` over global `pip install` (avoids polluting system Python) +# - Idempotent: detects already-running server and skips re-launch +# - Configurable port via --port=N (default 8188) +# - Configurable workspace via --workspace=PATH +# - Persistent log file in /tmp/comfyui_setup..log for debugging +# - SIGINT trap cleans up partial state +# - Refuses local install when hardware_check.py verdict is "cloud" +# - Forwards extra flags to comfy-cli (e.g. --cuda-version=12.4) # -# Prerequisites: Python 3.10+, pip -# What it does: -# 0. Hardware check (skipped if a flag was passed explicitly) -# 1. Installs comfy-cli (if not present) -# 2. Disables analytics tracking -# 3. Installs ComfyUI + ComfyUI-Manager -# 4. Launches server in background -# 5. Verifies server is reachable +# Usage: +# bash scripts/comfyui_setup.sh +# (auto-detects GPU; uses recommendation from hardware_check.py) +# bash scripts/comfyui_setup.sh --nvidia +# bash scripts/comfyui_setup.sh --m-series --port=8190 +# bash scripts/comfyui_setup.sh --amd --workspace=/data/comfy +# +# Flags: +# --nvidia | --amd | --m-series | --cpu GPU selection (skips hw check) +# --port=N HTTP port (default 8188) +# --workspace=PATH ComfyUI install location +# --skip-launch Install only, don't start server +# --force-cloud-override Install locally even if hw says cloud +# -- Pass remaining args to `comfy install` set -euo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" HARDWARE_CHECK="$SCRIPT_DIR/hardware_check.py" +LOG_FILE="/tmp/comfyui_setup.$$.log" +PORT=8188 +WORKSPACE="" +GPU_FLAG="" +SKIP_LAUNCH=0 +FORCE_CLOUD_OVERRIDE=0 +EXTRA_INSTALL_ARGS=() -# Step 0: Hardware check (auto-detect GPU flag when none was provided) -if [ $# -ge 1 ]; then - GPU_FLAG="$1" - echo "==> GPU flag: $GPU_FLAG (user-supplied, skipping hardware check)" -else +cleanup() { + local exit_code=$? + if [ $exit_code -ne 0 ]; then + echo "==> Setup exited with status $exit_code. Log: $LOG_FILE" >&2 + fi + exit $exit_code +} +trap cleanup EXIT INT TERM + +log() { echo "==> $*" | tee -a "$LOG_FILE" >&2; } +err() { echo "ERROR: $*" | tee -a "$LOG_FILE" >&2; } + +# --- Argument parsing --- +PASSTHROUGH=0 +for arg in "$@"; do + if [ "$PASSTHROUGH" -eq 1 ]; then + EXTRA_INSTALL_ARGS+=("$arg") + continue + fi + case "$arg" in + --nvidia|--amd|--m-series|--cpu) + GPU_FLAG="$arg" + ;; + --port=*) + PORT="${arg#*=}" + ;; + --workspace=*) + WORKSPACE="${arg#*=}" + ;; + --skip-launch) + SKIP_LAUNCH=1 + ;; + --force-cloud-override) + FORCE_CLOUD_OVERRIDE=1 + ;; + --) + PASSTHROUGH=1 + ;; + --help|-h) + # Print the leading comment block, stripping the `# ` prefix. + # Stops at the first blank line which separates docs from code. + awk ' + NR == 1 { next } # skip shebang + /^[^#]/ { exit } # stop at first non-comment line + /^$/ { exit } # ...or first blank line + { sub(/^# ?/, ""); print } + ' "$0" + exit 0 + ;; + *) + err "Unknown argument: $arg" + exit 64 + ;; + esac +done + +log "Logging to $LOG_FILE" + +# --- Step 0: Hardware check (skipped if user gave an explicit GPU flag) --- +if [ -z "$GPU_FLAG" ]; then if [ ! -f "$HARDWARE_CHECK" ]; then - echo "==> hardware_check.py not found, defaulting to --nvidia" + log "hardware_check.py not found — defaulting to --nvidia" GPU_FLAG="--nvidia" else - echo "==> Running hardware check..." + log "Running hardware check…" set +e - HW_JSON="$(python3 "$HARDWARE_CHECK" --json)" + HW_JSON="$(python3 "$HARDWARE_CHECK" --json 2>>"$LOG_FILE")" HW_EXIT=$? set -e - echo "$HW_JSON" - echo "" + + if [ -z "$HW_JSON" ]; then + err "hardware_check.py produced no output (exit $HW_EXIT). Pass an explicit flag." + exit 1 + fi + echo "$HW_JSON" | tee -a "$LOG_FILE" >&2 VERDICT="$(echo "$HW_JSON" | python3 -c 'import sys,json; print(json.load(sys.stdin).get("verdict",""))')" FLAG="$(echo "$HW_JSON" | python3 -c 'import sys,json; print(json.load(sys.stdin).get("comfy_cli_flag") or "")')" - if [ "$VERDICT" = "cloud" ]; then - echo "" - echo "==> Hardware check: this machine is not suitable for local ComfyUI." - echo " Recommended: Comfy Cloud — https://platform.comfy.org" - echo "" - echo " If you want to override and install anyway, re-run with an" - echo " explicit flag: bash $0 --nvidia|--amd|--m-series|--cpu" + if [ "$VERDICT" = "cloud" ] && [ "$FORCE_CLOUD_OVERRIDE" -ne 1 ]; then + log "" + log "Hardware check: this machine is not suitable for local ComfyUI." + log "Recommended: Comfy Cloud — https://platform.comfy.org" + log "" + log "To override and force a local install, re-run with --force-cloud-override" + log "or pass an explicit GPU flag (--nvidia|--amd|--m-series|--cpu)." exit 2 fi + if [ "$VERDICT" = "marginal" ]; then + log "Hardware check: verdict is MARGINAL." + log " SD1.5 should work; SDXL/Flux may be slow or OOM." + log " Consider Comfy Cloud for heavier workflows: https://platform.comfy.org" + fi + if [ -z "$FLAG" ]; then - echo "==> Hardware check couldn't pick a comfy-cli flag. Defaulting to --nvidia." - echo " (For Intel Arc or unsupported hardware, use the manual install path.)" + log "hardware_check could not pick a comfy-cli flag. Defaulting to --nvidia." + log "(For Intel Arc or unsupported hardware, use the manual install path.)" GPU_FLAG="--nvidia" else GPU_FLAG="$FLAG" fi + fi +fi - if [ "$VERDICT" = "marginal" ]; then - echo "==> Hardware check: verdict is MARGINAL." - echo " SD1.5 should work; SDXL/Flux may be slow or OOM." - echo " Consider Comfy Cloud for heavier workflows: https://platform.comfy.org" - echo "" +log "GPU flag: $GPU_FLAG" +log "Port: $PORT" +[ -n "$WORKSPACE" ] && log "Workspace: $WORKSPACE" +[ "${#EXTRA_INSTALL_ARGS[@]}" -gt 0 ] && log "Extra install args: ${EXTRA_INSTALL_ARGS[*]}" + +# --- Step 1: Install comfy-cli (prefer pipx / uvx over global pip) --- +COMFY_BIN="" +if command -v comfy >/dev/null 2>&1; then + COMFY_BIN="comfy" + log "comfy-cli already on PATH: $(comfy -v 2>/dev/null || echo 'unknown version')" +elif command -v uvx >/dev/null 2>&1; then + log "Using uvx (no install needed)" + COMFY_BIN="uvx --from comfy-cli comfy" +elif command -v pipx >/dev/null 2>&1; then + log "Installing comfy-cli via pipx…" + pipx install comfy-cli >>"$LOG_FILE" 2>&1 + COMFY_BIN="comfy" + # pipx adds shims to ~/.local/bin which may need to be on PATH + if ! command -v comfy >/dev/null 2>&1; then + if [ -x "$HOME/.local/bin/comfy" ]; then + export PATH="$HOME/.local/bin:$PATH" + COMFY_BIN="$HOME/.local/bin/comfy" + fi + fi +else + log "Neither pipx nor uvx found. Falling back to pip install --user…" + log " (Recommend installing pipx: https://pipx.pypa.io)" + if ! pip install --user comfy-cli >>"$LOG_FILE" 2>&1; then + # macOS: PEP 668 externally-managed-environment may block --user + log "pip install --user failed. Retrying with --break-system-packages…" + pip install --user --break-system-packages comfy-cli >>"$LOG_FILE" 2>&1 || { + err "Could not install comfy-cli. Install pipx or uv first." + exit 1 + } + fi + # Resolve the actual `comfy` script — pip --user puts it in: + # Linux: ~/.local/bin/comfy + # macOS: ~/Library/Python//bin/comfy OR ~/.local/bin/comfy + COMFY_BIN="" + for candidate in "$HOME/.local/bin/comfy" \ + "$HOME/Library/Python/3.13/bin/comfy" \ + "$HOME/Library/Python/3.12/bin/comfy" \ + "$HOME/Library/Python/3.11/bin/comfy" \ + "$HOME/Library/Python/3.10/bin/comfy"; do + if [ -x "$candidate" ]; then + COMFY_BIN="$candidate" + export PATH="$(dirname "$candidate"):$PATH" + break + fi + done + if [ -z "$COMFY_BIN" ]; then + if command -v comfy >/dev/null 2>&1; then + COMFY_BIN="comfy" + else + err "Installed comfy-cli but couldn't find the 'comfy' script." + err "Add the right Python user-bin directory to PATH and retry." + exit 1 fi fi fi -echo "==> ComfyUI Setup" -echo " GPU flag: $GPU_FLAG" -echo "" +# --- Step 2: Disable analytics tracking (avoid interactive prompt) --- +log "Disabling analytics tracking…" +$COMFY_BIN --skip-prompt tracking disable >>"$LOG_FILE" 2>&1 || true -# Step 1: Install comfy-cli -if command -v comfy >/dev/null 2>&1; then - echo "==> comfy-cli already installed: $(comfy -v 2>/dev/null || echo 'unknown version')" -else - echo "==> Installing comfy-cli..." - pip install comfy-cli +# --- Step 3: Install ComfyUI --- +WORKSPACE_ARG=() +if [ -n "$WORKSPACE" ]; then + WORKSPACE_ARG=(--workspace "$WORKSPACE") fi -# Step 2: Disable tracking (avoid interactive prompt) -echo "==> Disabling analytics tracking..." -comfy --skip-prompt tracking disable 2>/dev/null || true - -# Step 3: Install ComfyUI -if comfy which 2>/dev/null | grep -q "ComfyUI"; then - echo "==> ComfyUI already installed at: $(comfy which 2>/dev/null)" +if $COMFY_BIN "${WORKSPACE_ARG[@]}" which 2>/dev/null | grep -q "ComfyUI"; then + EXISTING_WS="$($COMFY_BIN "${WORKSPACE_ARG[@]}" which 2>/dev/null || true)" + log "ComfyUI already installed at: $EXISTING_WS" else - echo "==> Installing ComfyUI ($GPU_FLAG)..." - comfy --skip-prompt install $GPU_FLAG + log "Installing ComfyUI ($GPU_FLAG)…" + if ! $COMFY_BIN "${WORKSPACE_ARG[@]}" --skip-prompt install "$GPU_FLAG" "${EXTRA_INSTALL_ARGS[@]}" >>"$LOG_FILE" 2>&1; then + err "Install failed. Tail of log:" + tail -20 "$LOG_FILE" >&2 + exit 1 + fi fi -# Step 4: Launch in background -echo "==> Launching ComfyUI in background..." -comfy launch --background 2>/dev/null || { - echo "==> Background launch failed. Trying foreground check..." - echo " You may need to run: comfy launch" +if [ "$SKIP_LAUNCH" -eq 1 ]; then + log "Setup complete (--skip-launch). Run \`$COMFY_BIN launch --background -- --port $PORT\` when ready." + exit 0 +fi + +# --- Step 4: Detect already-running server --- +if curl -fsS "http://127.0.0.1:$PORT/system_stats" >/dev/null 2>&1; then + log "Server already running on port $PORT — skipping launch." + log "Stop with \`$COMFY_BIN stop\` if you want a fresh start." + curl -fsS "http://127.0.0.1:$PORT/system_stats" | python3 -m json.tool 2>/dev/null || true + log "Done." + exit 0 +fi + +# --- Step 5: Launch --- +log "Launching ComfyUI in background on port $PORT…" +LAUNCH_EXTRAS=("--" "--port" "$PORT") +if ! $COMFY_BIN "${WORKSPACE_ARG[@]}" launch --background "${LAUNCH_EXTRAS[@]}" >>"$LOG_FILE" 2>&1; then + err "Background launch failed. Tail of log:" + tail -20 "$LOG_FILE" >&2 + err "Try foreground launch to see real-time errors: $COMFY_BIN launch -- --port $PORT" exit 1 -} +fi -# Step 5: Wait for server to be ready -echo "==> Waiting for server..." -MAX_WAIT=30 +# --- Step 6: Wait for server --- +log "Waiting for server…" +MAX_WAIT=60 ELAPSED=0 while [ $ELAPSED -lt $MAX_WAIT ]; do - if curl -s http://127.0.0.1:8188/system_stats >/dev/null 2>&1; then - echo "==> Server is running!" - curl -s http://127.0.0.1:8188/system_stats | python3 -m json.tool 2>/dev/null || true + if curl -fsS "http://127.0.0.1:$PORT/system_stats" >/dev/null 2>&1; then + log "Server is running!" + curl -fsS "http://127.0.0.1:$PORT/system_stats" | python3 -m json.tool 2>/dev/null || true break fi sleep 2 @@ -115,17 +265,22 @@ while [ $ELAPSED -lt $MAX_WAIT ]; do done if [ $ELAPSED -ge $MAX_WAIT ]; then - echo "==> Server did not start within ${MAX_WAIT}s." - echo " Check logs with: comfy launch (foreground) to see errors." + err "Server did not start within ${MAX_WAIT}s." + err "Inspect log: $LOG_FILE" + err "Or run foreground: $COMFY_BIN launch -- --port $PORT" exit 1 fi -echo "" -echo "==> Setup complete!" -echo " Server: http://127.0.0.1:8188" -echo " Web UI: http://127.0.0.1:8188 (open in browser)" -echo " Stop: comfy stop" -echo "" -echo " Next steps:" -echo " - Download a model: comfy model download --url --relative-path models/checkpoints" -echo " - Run a workflow: python3 scripts/run_workflow.py --workflow --args '{...}'" +log "" +log "Setup complete!" +log " Server: http://127.0.0.1:$PORT" +log " Web UI: http://127.0.0.1:$PORT (open in browser)" +log " Stop: $COMFY_BIN stop" +log " Log: $LOG_FILE (kept until shell closes)" +log "" +log "Next steps:" +log " - Download a model: $COMFY_BIN model download --url --relative-path models/checkpoints" +log " - Run a workflow: python3 $SCRIPT_DIR/run_workflow.py --workflow --args '{...}'" + +# Disable trap on success path +trap - EXIT diff --git a/skills/creative/comfyui/scripts/extract_schema.py b/skills/creative/comfyui/scripts/extract_schema.py old mode 100644 new mode 100755 index a8dc524a140..ba44cfdf6a2 --- a/skills/creative/comfyui/scripts/extract_schema.py +++ b/skills/creative/comfyui/scripts/extract_schema.py @@ -1,100 +1,51 @@ #!/usr/bin/env python3 """ -extract_schema.py — Analyze a ComfyUI API-format workflow and extract controllable parameters. +extract_schema.py — Analyze a ComfyUI API-format workflow and extract +controllable parameters. -Reads a workflow JSON, identifies user-facing parameters (prompts, seed, dimensions, etc.) -by scanning node types and field names, and outputs a schema mapping. +Improvements over v1: + - Catalogs live in `_common.py`, shared with `check_deps.py` + - Coverage expanded for Flux / SD3 / Wan / Hunyuan / LTX / IPAdapter / rgthree + - Symmetric duplicate-name resolution: ALL duplicates get a node-id suffix + (instead of "first wins, second renamed"), so callers see consistent names + - Negative prompt detected by tracing `KSampler.negative` connections back to + the source CLIPTextEncode (more reliable than meta-title heuristic) + - Embedding references in prompt text are extracted as model dependencies + - Detects Primitive nodes that drive other nodes' inputs (and surfaces them + as the user-facing parameter) + - Reroutes are followed when tracing connections Usage: python3 extract_schema.py workflow_api.json python3 extract_schema.py workflow_api.json --output schema.json -Output format: - { - "parameters": { - "prompt": {"node_id": "6", "field": "text", "type": "string", "value": "..."}, - "seed": {"node_id": "3", "field": "seed", "type": "int", "value": 42}, - ... - }, - "output_nodes": ["9"], - "model_dependencies": [ - {"node_id": "4", "class_type": "CheckpointLoaderSimple", "field": "ckpt_name", "value": "..."} - ] - } - -Requires: Python 3.10+ (stdlib only) +Stdlib-only. Python 3.10+. """ +from __future__ import annotations + +import argparse import json import sys -import argparse from pathlib import Path +from typing import Any -# Known parameter patterns: (class_type, field_name) → friendly_name -PARAM_PATTERNS = [ - # Prompts - ("CLIPTextEncode", "text", "prompt"), - ("CLIPTextEncodeSDXL", "text_g", "prompt"), - ("CLIPTextEncodeSDXL", "text_l", "prompt_l"), - # Sampling - ("KSampler", "seed", "seed"), - ("KSampler", "steps", "steps"), - ("KSampler", "cfg", "cfg"), - ("KSampler", "sampler_name", "sampler_name"), - ("KSampler", "scheduler", "scheduler"), - ("KSampler", "denoise", "denoise"), - ("KSamplerAdvanced", "noise_seed", "seed"), - ("KSamplerAdvanced", "steps", "steps"), - ("KSamplerAdvanced", "cfg", "cfg"), - ("KSamplerAdvanced", "sampler_name", "sampler_name"), - ("KSamplerAdvanced", "scheduler", "scheduler"), - # Dimensions - ("EmptyLatentImage", "width", "width"), - ("EmptyLatentImage", "height", "height"), - ("EmptyLatentImage", "batch_size", "batch_size"), - # Image input - ("LoadImage", "image", "image"), - ("LoadImageMask", "image", "mask_image"), - # LoRA - ("LoraLoader", "lora_name", "lora_name"), - ("LoraLoader", "strength_model", "lora_strength"), - # Output - ("SaveImage", "filename_prefix", "filename_prefix"), -] +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + OUTPUT_NODES, PARAM_PATTERNS, PROMPT_FIELDS, + is_link, iter_embedding_refs, iter_model_deps, iter_nodes, unwrap_workflow, +) -# Node types that produce output files -OUTPUT_NODES = {"SaveImage", "PreviewImage", "VHS_VideoCombine", "SaveAudio", "SaveAnimatedWEBP", "SaveAnimatedPNG"} -# Node types that load models (for dependency checking) -MODEL_LOADERS = { - "CheckpointLoaderSimple": ("ckpt_name", "checkpoints"), - "CheckpointLoader": ("ckpt_name", "checkpoints"), - "LoraLoader": ("lora_name", "loras"), - "LoraLoaderModelOnly": ("lora_name", "loras"), - "VAELoader": ("vae_name", "vae"), - "ControlNetLoader": ("control_net_name", "controlnet"), - "CLIPLoader": ("clip_name", "clip"), - "DualCLIPLoader": ("clip_name1", "clip"), - "UNETLoader": ("unet_name", "unet"), - "DiffusionModelLoader": ("model_name", "diffusion_models"), - "UpscaleModelLoader": ("model_name", "upscale_models"), - "CLIPVisionLoader": ("clip_name", "clip_vision"), +# Sampler nodes whose `positive` / `negative` connections we trace +SAMPLER_NODE_FAMILY = { + "KSampler", "KSamplerAdvanced", + "SamplerCustom", "SamplerCustomAdvanced", + "BasicGuider", "CFGGuider", "DualCFGGuider", } -def validate_api_format(workflow: dict) -> bool: - """Check if workflow is in API format (not editor format).""" - if "nodes" in workflow and "links" in workflow: - return False - # API format: top-level keys are node IDs, each has class_type - for node_id, node in workflow.items(): - if isinstance(node, dict) and "class_type" in node: - return True - return False - - -def infer_type(value) -> str: - """Infer JSON schema type from a Python value.""" +def infer_type(value: Any) -> str: if isinstance(value, bool): return "bool" if isinstance(value, int): @@ -104,109 +55,261 @@ def infer_type(value) -> str: if isinstance(value, str): return "string" if isinstance(value, list): - return "link" # connections to other nodes + return "link" + if isinstance(value, dict): + return "object" return "unknown" -def extract_schema(workflow: dict) -> dict: - """Extract controllable parameters from a workflow.""" - parameters = {} - output_nodes = [] - model_deps = [] - name_counts = {} # track duplicate friendly names +def trace_to_node(workflow: dict, link: list, *, max_hops: int = 8) -> str | None: + """Follow a [node_id, slot] link, hopping through Reroute / Primitive nodes + if needed, to find the *upstream* node id that holds the actual value/input. - for node_id, node in workflow.items(): - if not isinstance(node, dict) or "class_type" not in node: + Bounded by both `max_hops` AND a visited-set to prevent infinite loops on + pathological graphs. + """ + if not is_link(link): + return None + nid: str | None = link[0] + visited: set[str] = set() + for _ in range(max_hops): + if nid is None or nid in visited: + return nid + visited.add(nid) + node = workflow.get(nid) + if not isinstance(node, dict): + return None + cls = node.get("class_type", "") + # Reroute / Primitive / passthrough wrappers + if cls in ("Reroute", "PrimitiveNode", "Note", "easy showAnything"): + inputs = node.get("inputs", {}) or {} + # Find first link-shaped input and follow it + next_link = next((v for v in inputs.values() if is_link(v)), None) + if next_link is None: + return nid + nid = next_link[0] continue + return nid + return nid - class_type = node["class_type"] - inputs = node.get("inputs", {}) - meta_title = node.get("_meta", {}).get("title", "") - # Check if this is an output node - if class_type in OUTPUT_NODES: +def find_negative_prompt_node(workflow: dict) -> str | None: + """Trace `negative` input of a sampler back to the source text encoder.""" + for nid, node in iter_nodes(workflow): + if node["class_type"] not in SAMPLER_NODE_FAMILY: + continue + inputs = node.get("inputs", {}) or {} + neg = inputs.get("negative") + if not is_link(neg): + continue + src = trace_to_node(workflow, neg) + if src and isinstance(workflow.get(src), dict): + cls = workflow[src].get("class_type", "") + if cls.startswith("CLIPTextEncode") or cls in ("smZ CLIPTextEncode", "BNK_CLIPTextEncodeAdvanced"): + return src + return None + + +def find_positive_prompt_node(workflow: dict) -> str | None: + for nid, node in iter_nodes(workflow): + if node["class_type"] not in SAMPLER_NODE_FAMILY: + continue + inputs = node.get("inputs", {}) or {} + pos = inputs.get("positive") + if not is_link(pos): + continue + src = trace_to_node(workflow, pos) + if src and isinstance(workflow.get(src), dict): + cls = workflow[src].get("class_type", "") + if cls.startswith("CLIPTextEncode") or cls in ("smZ CLIPTextEncode", "BNK_CLIPTextEncodeAdvanced"): + return src + return None + + +def extract_schema(workflow: dict) -> dict: + """Extract controllable parameters from a workflow. + + Returns: + { + "parameters": { friendly_name: {node_id, field, type, value, ...} }, + "output_nodes": [node_id, ...], + "model_dependencies": [{node_id, class_type, field, value, folder}], + "embedding_dependencies": [{node_id, embedding_name, found_in_field, value_excerpt}], + "summary": {...} + } + """ + output_nodes: list[str] = [] + + # First pass: identify positive / negative prompt nodes via connection tracing + pos_node = find_positive_prompt_node(workflow) + neg_node = find_negative_prompt_node(workflow) + + # ----- collect raw parameter candidates ----- + # Each candidate = (friendly_name, node_id, field, value) + # We resolve duplicate friendly_names AFTER the loop so dedup is symmetric. + raw_params: list[dict] = [] + + for node_id, node in iter_nodes(workflow): + cls = node["class_type"] + inputs = node.get("inputs", {}) or {} + + if cls in OUTPUT_NODES: output_nodes.append(node_id) - # Check if this is a model loader - if class_type in MODEL_LOADERS: - field, folder = MODEL_LOADERS[class_type] - if field in inputs and isinstance(inputs[field], str): - model_deps.append({ - "node_id": node_id, - "class_type": class_type, - "field": field, - "value": inputs[field], - "folder": folder, - }) - - # Extract controllable parameters - for pattern_class, pattern_field, friendly_name in PARAM_PATTERNS: - if class_type != pattern_class: + # Match this node against PARAM_PATTERNS + for p_class, p_field, friendly in PARAM_PATTERNS: + if cls != p_class: continue - if pattern_field not in inputs: + if p_field not in inputs: continue - value = inputs[pattern_field] - val_type = infer_type(value) - if val_type == "link": - continue # skip linked inputs — not directly controllable + value = inputs[p_field] + t = infer_type(value) + if t == "link": + continue # connections aren't directly controllable - # Disambiguate duplicate friendly names - # Use title hint for prompt fields - actual_name = friendly_name - if friendly_name == "prompt" and meta_title: - title_lower = meta_title.lower() - if "negative" in title_lower or "neg" in title_lower: + actual_name = friendly + + # Disambiguate prompt vs negative_prompt by connection tracing + if friendly == "prompt": + if node_id == neg_node and pos_node != neg_node: actual_name = "negative_prompt" + elif node_id == pos_node: + actual_name = "prompt" + else: + # Fallback: use _meta.title hints if present + meta_title = (node.get("_meta") or {}).get("title", "").lower() + if any(t_ in meta_title for t_ in ("negative", "neg", "-prompt", "anti")): + actual_name = "negative_prompt" - # Handle remaining duplicates by appending node_id - if actual_name in name_counts: - name_counts[actual_name] += 1 - actual_name = f"{actual_name}_{node_id}" - else: - name_counts[actual_name] = 1 - - parameters[actual_name] = { + raw_params.append({ + "name_hint": actual_name, "node_id": node_id, - "field": pattern_field, - "type": val_type, + "field": p_field, + "type": t, "value": value, + "class_type": cls, + }) + + # ----- symmetric duplicate-name resolution ----- + # Group by name_hint. If a hint appears once, keep it. If multiple, suffix + # ALL with their node_id. Always-stable, always-uniquely-addressable. + by_name: dict[str, list[dict]] = {} + for r in raw_params: + by_name.setdefault(r["name_hint"], []).append(r) + + parameters: dict[str, dict] = {} + for name, entries in by_name.items(): + if len(entries) == 1: + r = entries[0] + parameters[name] = { + "node_id": r["node_id"], "field": r["field"], + "type": r["type"], "value": r["value"], + "class_type": r["class_type"], } + else: + # Sort by node_id (string-natural) for stability + entries.sort(key=lambda x: (str(x["node_id"]).zfill(8), x["field"])) + for r in entries: + full_name = f"{name}_{r['node_id']}" + parameters[full_name] = { + "node_id": r["node_id"], "field": r["field"], + "type": r["type"], "value": r["value"], + "class_type": r["class_type"], + "alias_of": name, + } + + # ----- model dependencies ----- + model_deps = list(iter_model_deps(workflow)) + + # ----- embedding dependencies (in prompt text) ----- + embedding_deps: list[dict] = [] + seen_emb: set[tuple[str, str]] = set() + for nid, emb_name in iter_embedding_refs(workflow): + key = (nid, emb_name) + if key in seen_emb: + continue + seen_emb.add(key) + # Find which field had the reference, for context + node = workflow.get(nid, {}) + inputs = node.get("inputs", {}) or {} + found_field = None + excerpt = None + for fname, fval in inputs.items(): + if isinstance(fval, str) and fname in PROMPT_FIELDS and emb_name in fval: + found_field = fname + excerpt = fval[:120] + break + embedding_deps.append({ + "node_id": nid, + "embedding_name": emb_name, + "field": found_field, + "value_excerpt": excerpt, + "folder": "embeddings", + }) + + # ----- summary ----- + summary = { + "parameter_count": len(parameters), + "output_node_count": len(output_nodes), + "model_dep_count": len(model_deps), + "embedding_dep_count": len(embedding_deps), + "has_negative_prompt": "negative_prompt" in parameters, + "has_seed": "seed" in parameters or any(p.startswith("seed_") for p in parameters), + "is_video_workflow": any( + workflow.get(n, {}).get("class_type", "") in { + "VHS_VideoCombine", "SaveVideo", "SaveAnimatedWEBP", "SaveAnimatedPNG", + } for n in output_nodes + ), + } return { "parameters": parameters, "output_nodes": output_nodes, "model_dependencies": model_deps, + "embedding_dependencies": embedding_deps, + "summary": summary, } -def main(): - parser = argparse.ArgumentParser(description="Extract controllable parameters from a ComfyUI workflow") - parser.add_argument("workflow", help="Path to workflow API JSON file") - parser.add_argument("--output", "-o", help="Output file (default: stdout)") - args = parser.parse_args() +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Extract controllable parameters from a ComfyUI workflow") + p.add_argument("workflow", help="Path to workflow API JSON file") + p.add_argument("--output", "-o", help="Output file (default: stdout)") + p.add_argument("--summary-only", action="store_true", + help="Only print the summary block") + args = p.parse_args(argv) - workflow_path = Path(args.workflow) - if not workflow_path.exists(): - print(f"Error: {workflow_path} not found", file=sys.stderr) - sys.exit(1) + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + print(f"Error: {wf_path} not found", file=sys.stderr) + return 1 - with open(workflow_path) as f: - workflow = json.load(f) - - if not validate_api_format(workflow): - print("Error: Workflow is in editor format, not API format.", file=sys.stderr) - print("Re-export from ComfyUI using 'Save (API Format)' button.", file=sys.stderr) - sys.exit(1) + try: + with wf_path.open() as f: + payload = json.load(f) + workflow = unwrap_workflow(payload) + except ValueError as e: + print(f"Error: {e}", file=sys.stderr) + return 1 + except json.JSONDecodeError as e: + print(f"Error: invalid JSON — {e}", file=sys.stderr) + return 1 schema = extract_schema(workflow) - output_json = json.dumps(schema, indent=2) + if args.summary_only: + out = json.dumps(schema["summary"], indent=2) + else: + out = json.dumps(schema, indent=2, default=str) + if args.output: - Path(args.output).write_text(output_json) + Path(args.output).write_text(out) print(f"Schema written to {args.output}", file=sys.stderr) else: - print(output_json) + print(out) + + return 0 if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/skills/creative/comfyui/scripts/fetch_logs.py b/skills/creative/comfyui/scripts/fetch_logs.py new file mode 100755 index 00000000000..c7b3b084807 --- /dev/null +++ b/skills/creative/comfyui/scripts/fetch_logs.py @@ -0,0 +1,158 @@ +#!/usr/bin/env python3 +""" +fetch_logs.py — Retrieve workflow execution diagnostics from a ComfyUI server. + +When a workflow errors, the server's /history (local) or /jobs (cloud) entry +contains the full Python traceback. This script makes it easy to fetch by +prompt_id, with sensible formatting. + +Usage: + python3 fetch_logs.py + python3 fetch_logs.py --host https://cloud.comfy.org + python3 fetch_logs.py --tail-queue # show currently queued/running jobs +""" + +from __future__ import annotations + +import argparse +import json +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, http_get, is_cloud_host, + resolve_api_key, resolve_url, +) + + +def fetch_history_entry(host: str, headers: dict, prompt_id: str, *, is_cloud: bool) -> dict: + if is_cloud: + # Try /jobs/{id} first + url = resolve_url(host, f"/jobs/{prompt_id}", is_cloud=True) + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status == 200: + try: + return {"ok": True, "entry": r.json(), "source": "/api/jobs"} + except Exception: + pass + # Fallback to history_v2 + url = resolve_url(host, f"/history/{prompt_id}", is_cloud=True) + r = http_get(url, headers=headers, retries=2, timeout=30) + try: + data = r.json() + except Exception: + data = None + if r.status == 200 and data: + return {"ok": True, "entry": data, "source": "/api/history_v2"} + return {"ok": False, "http_status": r.status, "body": r.text()[:500]} + + url = resolve_url(host, f"/history/{prompt_id}", is_cloud=False) + r = http_get(url, headers=headers, retries=2, timeout=30) + if r.status != 200: + return {"ok": False, "http_status": r.status, "body": r.text()[:500]} + try: + data = r.json() + except Exception: + return {"ok": False, "reason": "non-JSON response"} + if not isinstance(data, dict) or prompt_id not in data: + return {"ok": False, "reason": "prompt_id not found in history", + "history_keys": list(data.keys())[:5] if isinstance(data, dict) else []} + return {"ok": True, "entry": data[prompt_id], "source": "/history"} + + +def fetch_queue(host: str, headers: dict) -> dict: + url = resolve_url(host, "/queue") + r = http_get(url, headers=headers, retries=2, timeout=15) + try: + data = r.json() + except Exception: + data = {"raw": r.text()[:500]} + return {"http_status": r.status, "data": data} + + +def extract_diagnostics(entry: dict) -> dict: + """Pull out the parts a human cares about: status, errors, traceback, timing.""" + diag: dict = {} + status = entry.get("status") or {} + diag["status_str"] = status.get("status_str") + diag["completed"] = status.get("completed") + + messages = status.get("messages") or [] + diag["execution_log"] = [] + for msg in messages: + if isinstance(msg, list) and len(msg) >= 2: + mtype, mdata = msg[0], msg[1] + diag["execution_log"].append({"type": mtype, "data": mdata}) + else: + diag["execution_log"].append(msg) + + # Look for execution_error inside messages + errors = [] + for msg in messages: + if isinstance(msg, list) and len(msg) >= 2 and msg[0] == "execution_error": + errors.append(msg[1]) + if errors: + diag["errors"] = errors + + # Cloud's /jobs response shape: top-level outputs / status / etc. + if "outputs" in entry: + out = entry["outputs"] or {} + if isinstance(out, dict): + diag["output_node_ids"] = list(out.keys()) + # Count file refs across all output buckets (images / video / etc.) + total = 0 + for node_output in out.values(): + if not isinstance(node_output, dict): + continue + for v in node_output.values(): + if isinstance(v, list): + total += len(v) + diag["output_count"] = total + else: + diag["output_node_ids"] = [] + diag["output_count"] = 0 + return diag + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Fetch workflow execution diagnostics") + p.add_argument("prompt_id", nargs="?", help="prompt_id to look up") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--raw", action="store_true", + help="Print the full history entry instead of the digest") + p.add_argument("--tail-queue", action="store_true", + help="Show currently running/pending jobs instead") + args = p.parse_args(argv) + + api_key = resolve_api_key(args.api_key) + headers = {"X-API-Key": api_key} if api_key else {} + is_cloud = is_cloud_host(args.host) + + if args.tail_queue: + emit_json(fetch_queue(args.host, headers)) + return 0 + + if not args.prompt_id: + print("Error: prompt_id is required (or use --tail-queue)", file=sys.stderr) + return 1 + + res = fetch_history_entry(args.host, headers, args.prompt_id, is_cloud=is_cloud) + if not res.get("ok"): + emit_json(res) + return 1 + + if args.raw: + emit_json(res) + return 0 + + diag = extract_diagnostics(res["entry"]) + diag["source"] = res.get("source") + diag["prompt_id"] = args.prompt_id + emit_json(diag) + return 0 if diag.get("status_str") not in ("error",) else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/creative/comfyui/scripts/hardware_check.py b/skills/creative/comfyui/scripts/hardware_check.py index 270ef37e91b..6a4d6c6d406 100755 --- a/skills/creative/comfyui/scripts/hardware_check.py +++ b/skills/creative/comfyui/scripts/hardware_check.py @@ -1,27 +1,24 @@ #!/usr/bin/env python3 -"""Detect whether this machine can realistically run ComfyUI locally. +"""hardware_check.py — Detect whether this machine can realistically run ComfyUI locally. -Emits a structured JSON report the agent can read to decide whether to: - - help the user install ComfyUI locally, or - - steer them to Comfy Cloud instead. +Improvements over v1: + - Multi-GPU detection: scans all NVIDIA / AMD GPUs, picks the best one (most VRAM) + - Apple Silicon: detects Rosetta-via-x86_64 false negative; warns instead of misclassifying + - Apple generation: defaults to None (unknown) instead of mis-tagging as M1 + - WSL2 detection: identifies WSL2 + nvidia-smi situation explicitly + - ROCm: prefers `rocm-smi --json` for new ROCm 6.x output + - Disk space check: warns if /home or workspace volume has < 25 GB free + - PyTorch verification (optional): tries to import torch and check device availability + - Windows: prefers PowerShell `Get-CimInstance` over deprecated `wmic` + - More accurate VRAM thresholds and verdict reasons + +Emits a structured JSON report. Exit codes match `verdict`: + 0 → ok + 1 → marginal + 2 → cloud Usage: - python3 hardware_check.py [--json] - -Exit code: - 0 → "ok" — can run local ComfyUI at reasonable speed - 1 → "marginal" — technically works but slow / memory-tight - 2 → "cloud" — local is not viable, recommend Comfy Cloud - -The JSON report always prints to stdout regardless of exit code. - -Output fields the agent should read: - verdict: "ok" | "marginal" | "cloud" - recommended_install_path: "nvidia" | "amd" | "apple-silicon" | "intel" | "comfy-cloud" - comfy_cli_flag: "--nvidia" | "--amd" | "--m-series" | None - (pass directly to `comfy install` when verdict != cloud) - gpu: detected GPU info or null - notes: list of human-readable strings to surface to the user + python3 hardware_check.py [--json] [--check-pytorch] """ from __future__ import annotations @@ -33,18 +30,28 @@ import re import shutil import subprocess import sys +from typing import Any -# Rough thresholds. SDXL/Flux need real VRAM; SD1.5 will scrape by on 6GB. -# Apple Silicon shares RAM with GPU — unified memory budget is total RAM. -MIN_VRAM_GB_USABLE = 6 # below this, most modern models won't load -OK_VRAM_GB = 8 # SDXL fits comfortably here -GREAT_VRAM_GB = 12 # Flux / video models start being realistic -MIN_MAC_RAM_GB = 16 # Apple Silicon unified memory; below = pain -OK_MAC_RAM_GB = 32 # smooth for SDXL / most workflows +# Thresholds (GiB). +MIN_VRAM_GB_USABLE = 6 +OK_VRAM_GB = 8 +GREAT_VRAM_GB = 12 +MIN_MAC_RAM_GB = 16 +OK_MAC_RAM_GB = 32 +MIN_FREE_DISK_GB = 25 # ComfyUI core ~5 GB + one model ~5–24 GB + +_COMFY_CLI_FLAG = { + "nvidia": "--nvidia", + "amd": "--amd", + "apple-silicon": "--m-series", + "intel": None, + "comfy-cloud": None, + "cpu": "--cpu", +} -def _run(cmd: list[str], timeout: int = 5) -> str: +def _run(cmd: list[str], timeout: int = 8) -> str: try: out = subprocess.run( cmd, capture_output=True, text=True, timeout=timeout, check=False @@ -54,45 +61,108 @@ def _run(cmd: list[str], timeout: int = 5) -> str: return "" +def is_wsl() -> bool: + """Return True when running under Windows Subsystem for Linux.""" + if platform.system() != "Linux": + return False + if "microsoft" in platform.release().lower() or "wsl" in platform.release().lower(): + return True + try: + with open("/proc/version", "r") as fh: + return "microsoft" in fh.read().lower() + except OSError: + return False + + +def is_rosetta() -> bool: + """Return True when Python is running translated under Rosetta on Apple Silicon.""" + if platform.system() != "Darwin": + return False + if platform.machine() == "arm64": + return False + # x86_64 on Darwin — could be Intel Mac or Rosetta. Probe sysctl. + out = _run(["sysctl", "-in", "sysctl.proc_translated"]).strip() + return out == "1" + + def detect_nvidia() -> dict | None: + """Detect NVIDIA GPUs. Returns the GPU with the most VRAM, plus list of all.""" if not shutil.which("nvidia-smi"): return None out = _run([ "nvidia-smi", - "--query-gpu=name,memory.total,driver_version", + "--query-gpu=index,name,memory.total,driver_version", "--format=csv,noheader,nounits", ]) if not out.strip(): return None - first = out.strip().splitlines()[0] - parts = [p.strip() for p in first.split(",")] - if len(parts) < 2: + gpus = [] + for line in out.strip().splitlines(): + parts = [p.strip() for p in line.split(",")] + if len(parts) < 3: + continue + try: + idx = int(parts[0]) + name = parts[1] + vram_mb = int(parts[2]) + except ValueError: + continue + driver = parts[3] if len(parts) > 3 else "" + gpus.append({ + "vendor": "nvidia", + "index": idx, + "name": name, + "vram_gb": round(vram_mb / 1024, 1), + "driver": driver, + }) + if not gpus: return None - name = parts[0] - try: - vram_mb = int(parts[1]) - except ValueError: - vram_mb = 0 - driver = parts[2] if len(parts) > 2 else "" - return { - "vendor": "nvidia", - "name": name, - "vram_gb": round(vram_mb / 1024, 1), - "driver": driver, - } + # Pick GPU with most VRAM + best = max(gpus, key=lambda g: g["vram_gb"]) + if len(gpus) > 1: + best["all_gpus"] = gpus + return best def detect_rocm() -> dict | None: if not shutil.which("rocm-smi"): return None + # Prefer JSON output (new ROCm 6.x) + out = _run(["rocm-smi", "--showproductname", "--showmeminfo", "vram", "--json"]) + if out.strip().startswith("{"): + try: + data = json.loads(out) + cards = [] + for card_id, info in data.items(): + if not card_id.startswith("card"): + continue + name = (info.get("Card series") or info.get("Card model") + or info.get("Marketing Name") or "AMD GPU") + vram_b = info.get("VRAM Total Memory (B)") or info.get("vram_total_memory_b") or 0 + try: + vram_b = int(vram_b) + except (ValueError, TypeError): + vram_b = 0 + cards.append({ + "vendor": "amd", + "name": str(name).strip(), + "vram_gb": round(vram_b / (1024**3), 1), + "driver": "rocm", + }) + if cards: + best = max(cards, key=lambda c: c["vram_gb"]) + if len(cards) > 1: + best["all_gpus"] = cards + return best + except json.JSONDecodeError: + pass + # Fall back to text parsing out = _run(["rocm-smi", "--showproductname", "--showmeminfo", "vram"]) if not out.strip(): return None - name_m = re.search(r"Card series:\s*(.+)", out) + name_m = re.search(r"Card (?:series|model|Marketing Name):\s*(.+)", out) vram_m = re.search(r"VRAM Total Memory \(B\):\s*(\d+)", out) - vram_gb = 0.0 - if vram_m: - vram_gb = round(int(vram_m.group(1)) / (1024**3), 1) + vram_gb = round(int(vram_m.group(1)) / (1024**3), 1) if vram_m else 0.0 return { "vendor": "amd", "name": name_m.group(1).strip() if name_m else "AMD GPU", @@ -105,33 +175,46 @@ def detect_apple_silicon() -> dict | None: if platform.system() != "Darwin": return None if platform.machine() != "arm64": - return None # Intel Mac — no usable MPS + return None chip = _run(["sysctl", "-n", "machdep.cpu.brand_string"]).strip() - # Examples: "Apple M1", "Apple M1 Pro", "Apple M2 Max", "Apple M3 Ultra" m = re.search(r"Apple M(\d+)", chip) - generation = int(m.group(1)) if m else 1 + generation = int(m.group(1)) if m else None mem_bytes = 0 try: mem_bytes = int(_run(["sysctl", "-n", "hw.memsize"]).strip() or 0) except ValueError: pass ram_gb = round(mem_bytes / (1024**3), 1) if mem_bytes else 0.0 + + # Detect chip variant ("Pro", "Max", "Ultra") — affects performance even at same gen + variant = None + for v in ("Ultra", "Max", "Pro"): + if v in chip: + variant = v + break + return { "vendor": "apple", "name": chip or "Apple Silicon", "generation": generation, + "variant": variant, "unified_memory_gb": ram_gb, } def detect_intel_arc() -> dict | None: - if platform.system() != "Linux": + if platform.system() not in ("Linux", "Windows"): return None - if not shutil.which("clinfo"): - return None - out = _run(["clinfo", "--list"]) - if "Intel" in out and ("Arc" in out or "Xe" in out): - return {"vendor": "intel", "name": "Intel Arc/Xe", "vram_gb": 0.0} + if shutil.which("clinfo"): + out = _run(["clinfo", "--list"]) + if "Intel" in out and ("Arc" in out or "Xe" in out): + return {"vendor": "intel", "name": "Intel Arc/Xe", "vram_gb": 0.0} + # Windows: try Get-CimInstance + if platform.system() == "Windows" and shutil.which("powershell"): + out = _run(["powershell", "-NoProfile", + "Get-CimInstance Win32_VideoController | Select-Object Name | Format-List"]) + if "Intel" in out and ("Arc" in out or "Iris Xe" in out): + return {"vendor": "intel", "name": "Intel Arc/Iris Xe", "vram_gb": 0.0} return None @@ -152,6 +235,15 @@ def total_system_ram_gb() -> float: except OSError: return 0.0 if sysname == "Windows": + if shutil.which("powershell"): + out = _run([ + "powershell", "-NoProfile", + "(Get-CimInstance Win32_ComputerSystem).TotalPhysicalMemory", + ]) + m = re.search(r"(\d{8,})", out) + if m: + return round(int(m.group(1)) / (1024**3), 1) + # Fall back to wmic for older Windows out = _run(["wmic", "ComputerSystem", "get", "TotalPhysicalMemory"]) m = re.search(r"(\d{6,})", out) if m: @@ -159,21 +251,67 @@ def total_system_ram_gb() -> float: return 0.0 -# Map recommended_install_path → flag the agent can pass to `comfy install` -# Set to None when no local install is advised (verdict=cloud). -_COMFY_CLI_FLAG = { - "nvidia": "--nvidia", - "amd": "--amd", - "apple-silicon": "--m-series", - "intel": None, # comfy-cli has no Intel Arc flag — manual install - "comfy-cloud": None, -} +def total_free_disk_gb(path: str = ".") -> float: + try: + usage = shutil.disk_usage(path) + return round(usage.free / (1024**3), 1) + except OSError: + return 0.0 -def classify(gpu: dict | None, ram_gb: float) -> tuple[str, str, list[str]]: - """Return (verdict, recommended_install_path, notes).""" +def check_pytorch_cuda() -> dict | None: + """Optional PyTorch availability check. Only run when --check-pytorch is set.""" + try: + import torch # type: ignore[import-not-found] + except Exception as e: + return {"available": False, "reason": f"torch not importable: {e}"} + info: dict[str, Any] = { + "available": True, + "torch_version": torch.__version__, + } + try: + info["cuda_available"] = bool(torch.cuda.is_available()) + if info["cuda_available"]: + info["cuda_device_count"] = torch.cuda.device_count() + info["cuda_device_0"] = torch.cuda.get_device_name(0) + except Exception: + info["cuda_available"] = False + try: + info["mps_available"] = bool(torch.backends.mps.is_available()) + except Exception: + info["mps_available"] = False + return info + + +def classify(gpu: dict | None, ram_gb: float, free_disk_gb: float, *, wsl: bool, rosetta: bool) -> tuple[str, str, list[str]]: notes: list[str] = [] + if rosetta: + notes.append( + "Detected Python running under Rosetta on Apple Silicon. " + "ComfyUI MPS support requires native ARM64 Python — install via " + "`brew install python` or arm64 Miniforge, then re-run." + ) + return "cloud", "comfy-cloud", notes + + if wsl and gpu and gpu["vendor"] == "nvidia": + notes.append("Detected WSL2 + NVIDIA — confirm `nvidia-smi` works in your WSL distro before installing.") + + if free_disk_gb and free_disk_gb < MIN_FREE_DISK_GB: + notes.append( + f"Free disk space ({free_disk_gb} GB) is below the {MIN_FREE_DISK_GB} GB recommended minimum. " + "ComfyUI core (~5 GB) plus one SDXL model (~6.5 GB) needs space; Flux Dev needs ~24 GB." + ) + + # Host RAM matters even for discrete-GPU systems: ComfyUI swaps model + # weights through CPU RAM when shuffling between text encoders / VAE / UNet. + # Apple's unified-memory check is handled below so don't double-warn. + if ram_gb and ram_gb < 8 and gpu and gpu.get("vendor") != "apple": + notes.append( + f"System RAM ({ram_gb} GB) is low. ComfyUI swaps model weights through " + "host RAM; <8 GB causes severe slowdowns. 16+ GB recommended." + ) + if gpu is None: notes.append( "No supported accelerator found (NVIDIA CUDA / AMD ROCm / Apple Silicon / Intel Arc)." @@ -184,49 +322,59 @@ def classify(gpu: dict | None, ram_gb: float) -> tuple[str, str, list[str]]: return "cloud", "comfy-cloud", notes if gpu["vendor"] == "apple": - gen = gpu.get("generation", 1) + gen = gpu.get("generation") + variant = gpu.get("variant") mem = gpu.get("unified_memory_gb", 0.0) + gen_str = f"M{gen}" if gen else "Apple Silicon" + if variant: + gen_str += f" {variant}" if mem < MIN_MAC_RAM_GB: notes.append( - f"Apple Silicon with {mem} GB unified memory — below the {MIN_MAC_RAM_GB} GB practical minimum." + f"{gen_str} with {mem} GB unified memory — below the {MIN_MAC_RAM_GB} GB practical minimum." ) notes.append("SD1.5 may work; SDXL/Flux will swap or OOM. Recommend Comfy Cloud.") return "cloud", "comfy-cloud", notes if mem < OK_MAC_RAM_GB: notes.append( - f"Apple Silicon M{gen} with {mem} GB — SDXL works but slow. Flux/video likely too tight." + f"{gen_str} with {mem} GB — SDXL works but slow. Flux/video likely too tight." ) return "marginal", "apple-silicon", notes - notes.append(f"Apple Silicon M{gen} with {mem} GB unified memory — good for SDXL/Flux.") + notes.append(f"{gen_str} with {mem} GB unified memory — good for SDXL/Flux.") return "ok", "apple-silicon", notes - # Discrete GPU path (nvidia/amd/intel) - vram = gpu.get("vram_gb", 0.0) if gpu["vendor"] == "intel": notes.append("Intel Arc detected — ComfyUI IPEX support is experimental; Comfy Cloud is more reliable.") return "marginal", "intel", notes + + # Discrete NVIDIA / AMD + vram = gpu.get("vram_gb", 0.0) + name = gpu["name"] if vram < MIN_VRAM_GB_USABLE: notes.append( - f"{gpu['name']} has only {vram} GB VRAM — below the {MIN_VRAM_GB_USABLE} GB practical minimum." + f"{name} has only {vram} GB VRAM — below the {MIN_VRAM_GB_USABLE} GB practical minimum." ) notes.append("Most modern models won't load. Recommend Comfy Cloud.") return "cloud", "comfy-cloud", notes if vram < OK_VRAM_GB: notes.append( - f"{gpu['name']} ({vram} GB VRAM) — SD1.5 works, SDXL tight, Flux/video unlikely." + f"{name} ({vram} GB VRAM) — SD1.5 works, SDXL tight, Flux/video unlikely." ) return "marginal", gpu["vendor"], notes if vram < GREAT_VRAM_GB: - notes.append(f"{gpu['name']} ({vram} GB VRAM) — SDXL comfortable, Flux possible with optimizations.") + notes.append(f"{name} ({vram} GB VRAM) — SDXL comfortable, Flux possible with optimizations.") return "ok", gpu["vendor"], notes - notes.append(f"{gpu['name']} ({vram} GB VRAM) — can run everything including Flux/video.") + notes.append(f"{name} ({vram} GB VRAM) — can run everything including Flux/video.") return "ok", gpu["vendor"], notes -def build_report() -> dict: +def build_report(*, check_pytorch: bool = False) -> dict: sysname = platform.system() arch = platform.machine() ram_gb = total_system_ram_gb() + free_disk_gb = total_free_disk_gb(os.path.expanduser("~")) + + rosetta = is_rosetta() + wsl = is_wsl() gpu = ( detect_nvidia() @@ -235,16 +383,19 @@ def build_report() -> dict: or detect_intel_arc() ) - # Intel Mac special case — fall out of apple-silicon detection with no GPU - if gpu is None and sysname == "Darwin" and platform.machine() != "arm64": + # Intel Mac: arm64 detect failed AND no other GPU paths + if gpu is None and sysname == "Darwin" and arch != "arm64" and not rosetta: notes = [ "Intel Mac detected — no MPS backend available.", "ComfyUI will fall back to CPU which is unusably slow. Use Comfy Cloud.", ] - return { + report = { "os": sysname, "arch": arch, "system_ram_gb": ram_gb, + "free_disk_gb": free_disk_gb, + "wsl": False, + "rosetta": False, "gpu": None, "verdict": "cloud", "recommended_install_path": "comfy-cloud", @@ -252,13 +403,21 @@ def build_report() -> dict: "notes": notes, "install_urls": _install_urls(), } + if check_pytorch: + report["pytorch"] = check_pytorch_cuda() + return report - verdict, install_path, notes = classify(gpu, ram_gb) + verdict, install_path, notes = classify( + gpu, ram_gb, free_disk_gb, wsl=wsl, rosetta=rosetta, + ) - return { + report = { "os": sysname, "arch": arch, "system_ram_gb": ram_gb, + "free_disk_gb": free_disk_gb, + "wsl": wsl, + "rosetta": rosetta, "gpu": gpu, "verdict": verdict, "recommended_install_path": install_path, @@ -266,6 +425,9 @@ def build_report() -> dict: "notes": notes, "install_urls": _install_urls(), } + if check_pytorch: + report["pytorch"] = check_pytorch_cuda() + return report def _install_urls() -> dict: @@ -277,26 +439,50 @@ def _install_urls() -> dict: } -def main() -> int: - report = build_report() - json_mode = "--json" in sys.argv +def main(argv: list[str] | None = None) -> int: + import argparse + p = argparse.ArgumentParser(description="Check whether this machine can run ComfyUI locally.") + p.add_argument("--json", action="store_true", help="Emit machine-readable JSON only") + p.add_argument("--check-pytorch", action="store_true", + help="Also probe `torch` for CUDA/MPS availability (slower)") + args = p.parse_args(argv) - if json_mode: + report = build_report(check_pytorch=args.check_pytorch) + + if args.json: print(json.dumps(report, indent=2)) else: - print(f"OS: {report['os']} ({report['arch']})") - print(f"RAM: {report['system_ram_gb']} GB") + print(f"OS: {report['os']} ({report['arch']})") + if report.get("wsl"): + print("Env: WSL2") + if report.get("rosetta"): + print("Env: Rosetta (x86_64 Python on Apple Silicon)") + print(f"RAM: {report['system_ram_gb']} GB") + print(f"Free disk: {report['free_disk_gb']} GB (~/)") if report["gpu"]: g = report["gpu"] if g["vendor"] == "apple": - print(f"GPU: {g['name']} — {g.get('unified_memory_gb', 0)} GB unified memory") + print(f"GPU: {g['name']} — {g.get('unified_memory_gb', 0)} GB unified memory") else: - print(f"GPU: {g['name']} — {g.get('vram_gb', 0)} GB VRAM") + print(f"GPU: {g['name']} — {g.get('vram_gb', 0)} GB VRAM") + if g.get("all_gpus") and len(g["all_gpus"]) > 1: + print(f" ({len(g['all_gpus'])} GPUs total; using best by VRAM)") else: - print("GPU: (none detected)") - print(f"Verdict: {report['verdict']} → {report['recommended_install_path']}") + print("GPU: (none detected)") + print(f"Verdict: {report['verdict']} → {report['recommended_install_path']}") if report["comfy_cli_flag"]: - print(f" → run: comfy --skip-prompt install {report['comfy_cli_flag']}") + print(f" run: comfy --skip-prompt install {report['comfy_cli_flag']}") + if report.get("pytorch"): + pt = report["pytorch"] + if pt.get("available"): + line = f"PyTorch: {pt.get('torch_version')}" + if pt.get("cuda_available"): + line += f" + CUDA ({pt.get('cuda_device_0', '?')})" + if pt.get("mps_available"): + line += " + MPS" + print(line) + else: + print(f"PyTorch: not available — {pt.get('reason')}") for n in report["notes"]: print(f" • {n}") diff --git a/skills/creative/comfyui/scripts/health_check.py b/skills/creative/comfyui/scripts/health_check.py new file mode 100755 index 00000000000..63c5025ca99 --- /dev/null +++ b/skills/creative/comfyui/scripts/health_check.py @@ -0,0 +1,223 @@ +#!/usr/bin/env python3 +""" +health_check.py — One-stop verification that the ComfyUI environment is ready. + +Runs through the verification checklist: + 1. comfy-cli on PATH + 2. server reachable (/system_stats) + 3. at least one checkpoint installed + 4. (optional) a specific workflow's deps are met + 5. (optional) actually submit a tiny test workflow and verify round-trip + +Usage: + python3 health_check.py + python3 health_check.py --host https://cloud.comfy.org + python3 health_check.py --workflow my.json + python3 health_check.py --smoke-test # actually submit a tiny workflow +""" + +from __future__ import annotations + +import argparse +import json +import shutil +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, emit_json, http_get, parse_model_list, + resolve_api_key, resolve_url, unwrap_workflow, +) + + +def comfy_cli_status() -> dict: + if shutil.which("comfy"): + return {"available": True, "method": "comfy", "path": shutil.which("comfy")} + if shutil.which("uvx"): + return {"available": True, "method": "uvx", + "hint": "Invoke as `uvx --from comfy-cli comfy ...`"} + return { + "available": False, + "hint": "Install with: pipx install comfy-cli (or `pip install comfy-cli`)", + } + + +def server_status(host: str, headers: dict) -> dict: + url = resolve_url(host, "/system_stats") + try: + r = http_get(url, headers=headers, retries=2, timeout=10) + if r.status == 200: + try: + stats = r.json() or {} + except Exception: + stats = {} + return {"reachable": True, "url": url, "stats": stats} + return {"reachable": False, "url": url, "http_status": r.status, "body": r.text()[:200]} + except Exception as e: + return {"reachable": False, "url": url, "error": str(e)} + + +def checkpoint_status(host: str, headers: dict) -> dict: + url = resolve_url(host, "/models/checkpoints") + try: + r = http_get(url, headers=headers, retries=2, timeout=15) + except Exception as e: + return {"queryable": False, "error": str(e)} + if r.status != 200: + return {"queryable": False, "http_status": r.status, "url": url, "body": r.text()[:200]} + try: + models = parse_model_list(r.json()) + except Exception: + models = set() + return {"queryable": True, "count": len(models), + "first_few": sorted(models)[:5]} + + +SMOKE_WORKFLOW = { + # Minimal SD1.5 workflow that doesn't depend on rare nodes. + # 256x256 + 1 step is the smallest config that doesn't trigger SDXL/Flux + # validation errors while still executing fast. + "3": { + "class_type": "KSampler", + "inputs": { + "seed": 1, "steps": 1, "cfg": 7.0, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + "model": ["4", 0], "positive": ["6", 0], "negative": ["7", 0], + "latent_image": ["5", 0], + }, + }, + "4": {"class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "REPLACE_ME"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 256, "height": 256, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", + "inputs": {"text": "test", "clip": ["4", 1]}}, + "7": {"class_type": "CLIPTextEncode", + "inputs": {"text": "", "clip": ["4", 1]}}, + "9": {"class_type": "SaveImage", + "inputs": {"filename_prefix": "smoke", "images": ["3", 0]}}, +} + + +def smoke_test(host: str, headers: dict, ckpt_name: str | None) -> dict: + """Submit a tiny workflow and verify the server accepts it. + + Cancels the job immediately after acceptance so we don't burn GPU + time / cloud minutes on a smoke test. + """ + if not ckpt_name: + return {"ran": False, "reason": "no checkpoint available"} + wf = json.loads(json.dumps(SMOKE_WORKFLOW)) + wf["4"]["inputs"]["ckpt_name"] = ckpt_name + + # Lazy import to avoid circular issues + from run_workflow import ComfyRunner + api_key = headers.get("X-API-Key") + runner = ComfyRunner(host=host, api_key=api_key) + sub = runner.submit(wf) + if "_http_error" in sub: + return {"ran": True, "submitted": False, + "http_status": sub["_http_error"], "body": sub.get("body")} + pid = sub.get("prompt_id") + if not pid: + return {"ran": True, "submitted": False, "response": sub} + + # Cancel so we don't actually waste compute on the smoke test. + cancelled = False + try: + cancelled = runner.cancel(pid) + except Exception: + pass + + return { + "ran": True, "submitted": True, "prompt_id": pid, + "cancelled_after_submit": cancelled, + "note": "Submission accepted; cancelled to avoid running the full pipeline.", + } + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="One-stop ComfyUI health check") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--workflow", help="Optional: also run check_deps on this workflow") + p.add_argument("--smoke-test", action="store_true", + help="Submit a tiny test workflow and verify round-trip") + p.add_argument("--strict", action="store_true", + help="Exit non-zero on any non-pass condition (including warnings)") + args = p.parse_args(argv) + + api_key = resolve_api_key(args.api_key) + headers = {"X-API-Key": api_key} if api_key else {} + + cli = comfy_cli_status() + server = server_status(args.host, headers) + ckpts = checkpoint_status(args.host, headers) if server.get("reachable") else None + + # ---- workflow check ---- + workflow_check: dict | None = None + if args.workflow: + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + workflow_check = {"error": "workflow file not found"} + else: + try: + with wf_path.open() as f: + workflow = unwrap_workflow(json.load(f)) + from check_deps import check_deps + workflow_check = check_deps(workflow, host=args.host, api_key=api_key) + except (ValueError, json.JSONDecodeError) as e: + workflow_check = {"error": str(e)} + + smoke = None + if args.smoke_test and server.get("reachable"): + first_ckpt = ckpts["first_few"][0] if ckpts and ckpts.get("first_few") else None + smoke = smoke_test(args.host, headers, first_ckpt) + + # ---- verdict ---- + verdict = "pass" + reasons: list[str] = [] + if not server.get("reachable"): + verdict = "fail" + reasons.append("server unreachable") + if ckpts and ckpts.get("queryable") and ckpts.get("count", 0) == 0: + verdict = "warn" if verdict == "pass" else verdict + reasons.append("no checkpoints installed") + if workflow_check and workflow_check.get("error"): + verdict = "fail" + reasons.append(f"workflow check failed: {workflow_check['error']}") + elif workflow_check and not workflow_check.get("is_ready"): + if workflow_check.get("node_check_skipped"): + reasons.append("node check skipped (cloud free tier)") + else: + verdict = "fail" + reasons.append("workflow has missing deps") + if smoke and smoke.get("ran") and not smoke.get("submitted"): + verdict = "fail" + reasons.append("smoke-test submission failed") + if not cli.get("available"): + verdict = "warn" if verdict == "pass" else verdict + reasons.append("comfy-cli not on PATH (lifecycle commands won't work)") + + report = { + "verdict": verdict, + "reasons": reasons, + "host": args.host, + "comfy_cli": cli, + "server": server, + "checkpoints": ckpts, + "workflow_check": workflow_check, + "smoke_test": smoke, + } + emit_json(report) + + if verdict == "pass": + return 0 + if verdict == "warn": + return 1 if args.strict else 0 + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/creative/comfyui/scripts/run_batch.py b/skills/creative/comfyui/scripts/run_batch.py new file mode 100755 index 00000000000..7f5b159dbda --- /dev/null +++ b/skills/creative/comfyui/scripts/run_batch.py @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +""" +run_batch.py — Run a workflow many times, varying parameters per run. + +Two modes: + 1. --count N --randomize-seed + Submit N runs, each with a fresh random seed. Use for quick variations. + 2. --sweep '{"seed": [1,2,3], "steps": [20,30]}' + Cartesian product of values. With cloud subscription, runs in parallel + up to your tier's concurrent-job limit. + +Both modes write each run's outputs into output-dir/run_NNN/. + +Examples: + python3 run_batch.py --workflow flux_dev.json \ + --args '{"prompt": "a cat"}' \ + --count 8 --randomize-seed \ + --output-dir ./outputs/cat-batch + + python3 run_batch.py --workflow sdxl.json \ + --args '{"prompt": "abstract"}' \ + --sweep '{"seed": [1,2,3], "steps": [20, 40]}' \ + --output-dir ./outputs/sweep +""" + +from __future__ import annotations + +import argparse +import itertools +import json +import sys +from concurrent.futures import ThreadPoolExecutor, as_completed +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, coerce_seed, emit_json, log, + looks_like_video_workflow, resolve_api_key, unwrap_workflow, +) +from run_workflow import ( # noqa: E402 + ComfyRunner, download_outputs, inject_params, +) +from extract_schema import extract_schema # noqa: E402 + + +def expand_sweep(sweep: dict, base_args: dict, count: int, randomize_seed: bool) -> list[dict]: + """Generate a list of args dicts for each run.""" + if sweep: + # Cartesian product + keys = list(sweep.keys()) + values = [sweep[k] if isinstance(sweep[k], list) else [sweep[k]] for k in keys] + runs = [] + for combo in itertools.product(*values): + ar = dict(base_args) + for k, v in zip(keys, combo): + ar[k] = v + runs.append(ar) + return runs + # Count mode + runs = [] + for _ in range(count): + ar = dict(base_args) + if randomize_seed: + ar["seed"] = coerce_seed(None) + runs.append(ar) + return runs + + +def execute_one( + runner: ComfyRunner, workflow: dict, schema: dict, args: dict, + *, output_dir: Path, timeout: int, ws: bool, +) -> dict: + wf, warnings = inject_params(workflow, schema, args) + sub = runner.submit(wf) + if "_http_error" in sub: + return {"status": "error", "error": "submission HTTP error", + "details": sub.get("body"), "args": args} + pid = sub.get("prompt_id") + if not pid: + return {"status": "error", "error": "no prompt_id", "response": sub, "args": args} + if sub.get("node_errors"): + return {"status": "error", "error": "validation failed", + "node_errors": sub["node_errors"], "args": args} + + if ws: + result = runner.monitor_ws(pid, timeout=timeout) + else: + result = runner.poll_status(pid, timeout=timeout) + + if result["status"] != "success": + return { + "status": result["status"], + "prompt_id": pid, + "details": result.get("data"), + "args": args, + } + + outputs = result.get("outputs") or runner.get_outputs(pid) + downloaded = download_outputs(runner, outputs, output_dir, preserve_subfolder=False) + return { + "status": "success", + "prompt_id": pid, + "args": args, + "outputs": downloaded, + "warnings": warnings, + } + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser( + description="Submit a workflow many times with varying parameters.", + ) + p.add_argument("--workflow", required=True) + p.add_argument("--args", default="{}", help="Base parameters JSON") + p.add_argument("--count", type=int, default=0, + help="Number of runs (use with --randomize-seed)") + p.add_argument("--sweep", default="", + help='JSON dict of param→list of values. Cartesian product. ' + 'e.g. \'{"seed":[1,2,3],"cfg":[5,8]}\'') + p.add_argument("--randomize-seed", action="store_true", + help="In --count mode, vary seed per run") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST) + p.add_argument("--api-key", help=f"or set ${ENV_API_KEY}") + p.add_argument("--partner-key") + p.add_argument("--parallel", type=int, default=1, + help="Concurrent submissions (cloud: up to your tier limit). " + "Default 1 (sequential)") + p.add_argument("--output-dir", default="./outputs/batch") + p.add_argument("--timeout", type=int, default=0) + p.add_argument("--ws", action="store_true") + p.add_argument("--continue-on-error", action="store_true", + help="Don't stop the batch when a run fails") + args = p.parse_args(argv) + + if args.count <= 0 and not args.sweep: + emit_json({"error": "Specify --count N or --sweep '{...}'"}) + return 1 + + base_args = json.loads(args.args) if args.args.strip() else {} + sweep = json.loads(args.sweep) if args.sweep.strip() else {} + + # Validate sweep shape + if sweep: + if not isinstance(sweep, dict): + emit_json({"error": "--sweep must be a JSON object {param: [values]}"}) + return 1 + empty = [k for k, v in sweep.items() if isinstance(v, list) and len(v) == 0] + if empty: + emit_json({"error": f"--sweep parameters have empty value lists: {empty}"}) + return 1 + # If user passed BOTH --sweep and --count/--randomize-seed, --sweep wins + if args.count or args.randomize_seed: + log("--sweep set; ignoring --count / --randomize-seed (sweep defines the runs)") + + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + workflow = unwrap_workflow(json.load(f)) + except (ValueError, json.JSONDecodeError) as e: + emit_json({"error": str(e)}) + return 1 + + schema = extract_schema(workflow) + runs = expand_sweep(sweep, base_args, args.count, args.randomize_seed) + log(f"Planned {len(runs)} run(s)") + + api_key = resolve_api_key(args.api_key) + runner = ComfyRunner(host=args.host, api_key=api_key, partner_key=args.partner_key) + + ok, info = runner.check_server() + if not ok: + emit_json({"error": "Cannot reach server", "details": info, "host": args.host}) + return 1 + + timeout = args.timeout + if timeout <= 0: + timeout = 900 if looks_like_video_workflow(workflow) else 300 + + base_dir = Path(args.output_dir).expanduser() + base_dir.mkdir(parents=True, exist_ok=True) + + results: list[dict] = [] + failures = 0 + + if args.parallel > 1: + with ThreadPoolExecutor(max_workers=args.parallel) as ex: + future_to_idx = {} + for i, ar in enumerate(runs): + run_dir = base_dir / f"run_{i:04d}" + fut = ex.submit( + execute_one, runner, workflow, schema, ar, + output_dir=run_dir, timeout=timeout, ws=args.ws, + ) + future_to_idx[fut] = i + for fut in as_completed(future_to_idx): + i = future_to_idx[fut] + try: + r = fut.result() + except Exception as e: + r = {"status": "error", "error": str(e), "args": runs[i]} + r["index"] = i + results.append(r) + if r["status"] != "success": + failures += 1 + log(f" run {i} → {r['status']}: {r.get('error','?')}") + if not args.continue_on_error: + log(" --continue-on-error not set; aborting batch") + break + else: + log(f" run {i} → success: {len(r.get('outputs', []))} files") + else: + for i, ar in enumerate(runs): + run_dir = base_dir / f"run_{i:04d}" + r = execute_one(runner, workflow, schema, ar, + output_dir=run_dir, timeout=timeout, ws=args.ws) + r["index"] = i + results.append(r) + if r["status"] != "success": + failures += 1 + log(f" run {i} → {r['status']}: {r.get('error','?')}") + if not args.continue_on_error: + log(" --continue-on-error not set; aborting batch") + break + else: + log(f" run {i} → success: {len(r.get('outputs', []))} files") + + results.sort(key=lambda x: x.get("index", 0)) + emit_json({ + "status": "success" if failures == 0 else "partial", + "total": len(runs), + "completed": sum(1 for r in results if r["status"] == "success"), + "failed": failures, + "output_dir": str(base_dir), + "results": results, + }) + return 0 if failures == 0 else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/creative/comfyui/scripts/run_workflow.py b/skills/creative/comfyui/scripts/run_workflow.py old mode 100644 new mode 100755 index 95b598fa798..444957960b6 --- a/skills/creative/comfyui/scripts/run_workflow.py +++ b/skills/creative/comfyui/scripts/run_workflow.py @@ -1,7 +1,23 @@ #!/usr/bin/env python3 """ -run_workflow.py — Inject parameters into a ComfyUI workflow, submit it, monitor execution, -and download outputs. +run_workflow.py — Inject parameters into a ComfyUI workflow, submit it, monitor +execution, and download outputs. + +Improvements over v1: + - Cloud-aware URL routing (handles /api prefix and /history_v2 / /experiment/models renames) + - API key from CLI flag OR $COMFY_CLOUD_API_KEY env var + - WebSocket progress monitoring (--ws), with HTTP polling fallback + - Streaming download (no whole-file buffering — handles GB-size video outputs) + - Path-traversal-safe output writes + - Subfolder-aware download paths (no silent overwrites) + - Retry with exponential backoff on transient errors + - Status-error correctly classified before "completed: true" + - Image upload helper (--input-image NAME=PATH) + - Auto-randomize seed when value is -1 or omitted on a randomize-seed flag + - Auto-extends timeout heuristically for video workflows + - Editor-format detection with helpful error + - Doesn't pollute extra_data.api_key_comfy_org with the cloud auth key + unless --partner-key is provided (correct semantic per cloud docs) Usage: # Local server @@ -9,345 +25,772 @@ Usage: --args '{"prompt": "a cat", "seed": 42}' \ --output-dir ./outputs - # Cloud server + # Cloud server (API key from env var) + export COMFY_CLOUD_API_KEY="comfyui-xxxxxxx" python3 run_workflow.py --workflow workflow_api.json \ --args '{"prompt": "a cat"}' \ --host https://cloud.comfy.org \ - --api-key comfyui-xxxxxxx \ --output-dir ./outputs - # With schema file (pre-extracted) - python3 run_workflow.py --workflow workflow_api.json \ - --schema schema.json \ - --args '{"prompt": "a cat"}' \ - --output-dir ./outputs + # With image input (auto-uploads, then references) + python3 run_workflow.py --workflow img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it cyberpunk"}' -Requires: Python 3.10+, requests (or urllib as fallback) + # WebSocket real-time progress + python3 run_workflow.py --workflow flux_dev.json \ + --args '{"prompt": "..."}' \ + --ws + +Stdlib-only by default (Python 3.10+). Will use `requests`/`websocket-client` +if installed for nicer behavior. """ +from __future__ import annotations + +import argparse +import copy import json import sys import time -import uuid -import copy -import argparse from pathlib import Path -from urllib.parse import urljoin, urlencode, urlparse +from typing import Any +from urllib.parse import urlencode, urlparse -try: - import requests - HAS_REQUESTS = True -except ImportError: - HAS_REQUESTS = False - import urllib.request - import urllib.error +# Local import — _common.py sits next to this script. +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, + coerce_seed, emit_json, http_get, http_post, http_request, + is_cloud_host, is_link, log, looks_like_video_workflow, + media_type_from_filename, new_client_id, resolve_api_key, resolve_url, + safe_path_join, unwrap_workflow, +) -def http_get(url: str, headers: dict = None, follow_redirects: bool = True) -> tuple: - """GET request, returns (status_code, body_bytes, response_headers).""" - if HAS_REQUESTS: - r = requests.get(url, headers=headers or {}, allow_redirects=follow_redirects, timeout=30) - return r.status_code, r.content, dict(r.headers) - else: - req = urllib.request.Request(url, headers=headers or {}) - try: - resp = urllib.request.urlopen(req, timeout=30) - return resp.status, resp.read(), dict(resp.headers) - except urllib.error.HTTPError as e: - return e.code, e.read(), dict(e.headers) +# ============================================================================= +# Runner +# ============================================================================= +class WorkflowRunError(Exception): + """Raised when a workflow run fails (validation, execution, timeout).""" -def http_post(url: str, data: dict, headers: dict = None) -> tuple: - """POST JSON request, returns (status_code, response_dict).""" - payload = json.dumps(data).encode() - hdrs = {"Content-Type": "application/json"} - if headers: - hdrs.update(headers) - if HAS_REQUESTS: - r = requests.post(url, json=data, headers=hdrs, timeout=30) - try: - return r.status_code, r.json() - except Exception: - return r.status_code, {"raw": r.text} - else: - req = urllib.request.Request(url, data=payload, headers=hdrs, method="POST") - try: - resp = urllib.request.urlopen(req, timeout=30) - return resp.status, json.loads(resp.read()) - except urllib.error.HTTPError as e: - return e.code, json.loads(e.read()) + def __init__(self, status: str, message: str, **details: Any): + super().__init__(message) + self.status = status + self.message = message + self.details = details + + def to_dict(self) -> dict: + d = {"status": self.status, "error": self.message} + d.update(self.details) + return d class ComfyRunner: - def __init__(self, host: str = "http://127.0.0.1:8188", api_key: str = None): + def __init__( + self, + host: str = DEFAULT_LOCAL_HOST, + api_key: str | None = None, + client_id: str | None = None, + partner_key: str | None = None, + ): self.host = host.rstrip("/") self.api_key = api_key - parsed_host = urlparse(self.host).hostname or "" - self.is_cloud = parsed_host.lower() == "cloud.comfy.org" or api_key is not None - self.client_id = str(uuid.uuid4()) + self.partner_key = partner_key + self.is_cloud = is_cloud_host(self.host) + self.client_id = client_id or new_client_id() @property - def headers(self) -> dict: - h = {} + def headers(self) -> dict[str, str]: + h: dict[str, str] = {} if self.api_key: h["X-API-Key"] = self.api_key return h - def api_url(self, path: str) -> str: - """Build URL. Cloud uses /api prefix for some endpoints.""" - if self.is_cloud and not path.startswith("/api"): - # Cloud endpoints: /api/prompt, /api/view, /api/job, /api/queue - return f"{self.host}/api{path}" - return f"{self.host}{path}" + def _url(self, path: str) -> str: + return resolve_url(self.host, path, is_cloud=self.is_cloud) - def check_server(self) -> bool: - """Check if server is reachable.""" + # ---------- server health ---------- + def check_server(self) -> tuple[bool, dict | None]: try: - url = self.api_url("/system_stats") if not self.is_cloud else f"{self.host}/api/system_stats" - status, _, _ = http_get(url, self.headers) - return status == 200 + r = http_get(self._url("/system_stats"), headers=self.headers, retries=2) + if r.status == 200: + try: + return True, r.json() + except Exception: + return True, None + return False, {"http_status": r.status, "body": r.text()[:500]} + except Exception as e: + return False, {"error": str(e)} + + # ---------- upload ---------- + def upload_image(self, path: Path, *, image_type: str = "input", overwrite: bool = True, + endpoint: str = "/upload/image", extra_form: dict | None = None) -> dict: + """Upload an image file via multipart. Returns server-side ref dict.""" + if not path.exists(): + raise FileNotFoundError(f"input image not found: {path}") + # Stream the file via a handle to avoid OOM on huge inputs (16MP+ photos). + with path.open("rb") as fh: + files = {"image": (path.name, fh)} + form = {"type": image_type} + if overwrite: + form["overwrite"] = "true" + if extra_form: + form.update({k: str(v) for k, v in extra_form.items()}) + r = http_request( + "POST", self._url(endpoint), + headers=self.headers, files=files, form=form, + timeout=300, retries=2, + ) + if r.status != 200: + raise WorkflowRunError( + "upload_failed", + f"Upload of {path.name} failed: HTTP {r.status}", + body=r.text()[:500], + ) + try: + return r.json() except Exception: - return False + return {"name": path.name} + def upload_mask(self, path: Path, original_ref: dict) -> dict: + """Upload an inpaint mask, linked to a previously uploaded source image. + + `original_ref` should be the dict returned by `upload_image()` for the + source image (or `{"filename": ..., "subfolder": ..., "type": "input"}`). + """ + return self.upload_image( + path, + endpoint="/upload/mask", + extra_form={ + "subfolder": "clipspace", + "original_ref": json.dumps(original_ref), + }, + ) + + # ---------- submit ---------- def submit(self, workflow: dict) -> dict: - """Submit workflow for execution. Returns {prompt_id, node_errors}.""" - payload = {"prompt": workflow, "client_id": self.client_id} - if self.api_key and self.is_cloud: - payload.setdefault("extra_data", {})["api_key_comfy_org"] = self.api_key - url = self.api_url("/prompt") - status, resp = http_post(url, payload, self.headers) - if status != 200: - return {"error": f"HTTP {status}", "details": resp} - return resp + payload: dict[str, Any] = {"prompt": workflow, "client_id": self.client_id} + if self.partner_key: + payload["extra_data"] = {"api_key_comfy_org": self.partner_key} - def poll_status(self, prompt_id: str, timeout: int = 120) -> dict: - """Poll until job completes. Returns final status dict.""" + r = http_post(self._url("/prompt"), headers=self.headers, json_body=payload, timeout=120) + try: + body = r.json() + except Exception: + body = {"raw": r.text()[:500]} + if r.status != 200: + return {"_http_error": r.status, "body": body} + return body + + # ---------- HTTP polling ---------- + def poll_status(self, prompt_id: str, *, timeout: float = 300.0, + initial_interval: float = 1.5, max_interval: float = 8.0) -> dict: start = time.time() - poll_interval = 2.0 + interval = initial_interval while time.time() - start < timeout: if self.is_cloud: - # Cloud has a dedicated status endpoint - url = f"{self.host}/api/job/{prompt_id}/status" - status, body, _ = http_get(url, self.headers) - if status == 200: - data = json.loads(body) if isinstance(body, bytes) else body - job_status = data.get("status", "unknown") - if job_status == "completed": + r = http_get( + self._url(f"/job/{prompt_id}/status"), + headers=self.headers, retries=2, timeout=30, + ) + if r.status == 200: + try: + data = r.json() + except Exception: + data = {} + s = data.get("status") + if s == "completed": return {"status": "success", "data": data} - elif job_status == "failed": + if s in ("failed",): return {"status": "error", "data": data} - elif job_status == "cancelled": + if s == "cancelled": return {"status": "cancelled", "data": data} - # still running, continue polling + # pending / in_progress → continue + elif r.status == 404: + # Cloud sometimes 404s briefly between submit and dispatcher pickup + pass + else: + # transient error — retry loop covers it + pass else: - # Local: check /history/{prompt_id} - url = f"{self.host}/history/{prompt_id}" - status, body, _ = http_get(url, self.headers) - if status == 200: - data = json.loads(body) if isinstance(body, bytes) else body - if prompt_id in data: - entry = data[prompt_id] - if entry.get("status", {}).get("completed", False): - return {"status": "success", "outputs": entry.get("outputs", {})} - if entry.get("status", {}).get("status_str") == "error": + # Local: /history/{id} grows once execution completes + r = http_get( + self._url(f"/history/{prompt_id}"), + headers=self.headers, retries=2, timeout=30, + ) + if r.status == 200: + try: + data = r.json() or {} + except Exception: + data = {} + entry = data.get(prompt_id) + if isinstance(entry, dict): + st = entry.get("status") or {} + # IMPORTANT: check error first — `completed: true` can coexist with errors + status_str = st.get("status_str") + if status_str == "error": return {"status": "error", "data": entry} + if st.get("completed", False): + return {"status": "success", "outputs": entry.get("outputs", {})} + # not in history yet → continue polling - time.sleep(poll_interval) - poll_interval = min(poll_interval * 1.2, 10.0) + time.sleep(interval) + interval = min(max_interval, interval * 1.4) return {"status": "timeout", "elapsed": time.time() - start} - def get_outputs(self, prompt_id: str) -> dict: - """Get output file info from history.""" - if self.is_cloud: - url = f"{self.host}/api/job/{prompt_id}/status" - else: - url = f"{self.host}/history/{prompt_id}" - status, body, _ = http_get(url, self.headers) - if status != 200: - return {} - data = json.loads(body) if isinstance(body, bytes) else body - if self.is_cloud: - return data.get("outputs", {}) - if prompt_id in data: - return data[prompt_id].get("outputs", {}) - return {} + # ---------- WebSocket monitoring ---------- + def monitor_ws(self, prompt_id: str, *, timeout: float = 300.0, + on_progress: Any = None) -> dict: + """Connect to /ws and listen until execution_success / execution_error. - def download_output(self, filename: str, subfolder: str, file_type: str, output_dir: Path) -> Path: - """Download a single output file.""" - params = urlencode({"filename": filename, "subfolder": subfolder, "type": file_type}) - url = self.api_url(f"/view?{params}") - status, body, _ = http_get(url, self.headers, follow_redirects=True) - if status != 200: - raise RuntimeError(f"Failed to download {filename}: HTTP {status}") - out_path = output_dir / filename - out_path.write_bytes(body) + Falls back to HTTP polling if `websocket-client` is not installed. + Returns same shape as poll_status. + """ + try: + import websocket # type: ignore[import-not-found] + except ImportError: + log("websocket-client not installed; falling back to HTTP polling") + return self.poll_status(prompt_id, timeout=timeout) + + # Build WS URL. Preserve any base-path components the user gave us + # (e.g. http://example.com/comfyui → ws://example.com/comfyui/ws). + parsed = urlparse(self.host) + scheme = "wss" if parsed.scheme == "https" else "ws" + netloc = parsed.netloc + base_path = parsed.path.rstrip("/") + ws_url = f"{scheme}://{netloc}{base_path}/ws?clientId={self.client_id}" + if self.is_cloud and self.api_key: + ws_url += f"&token={self.api_key}" + + outputs: dict[str, Any] = {} + error_payload: dict[str, Any] | None = None + success = False + seen_executed = False + + ws = websocket.create_connection(ws_url, timeout=timeout) + try: + ws.settimeout(timeout) + deadline = time.time() + timeout + while time.time() < deadline: + msg = ws.recv() + if isinstance(msg, bytes): + # Binary preview frame — ignore for now; ws_monitor.py prints them + continue + try: + payload = json.loads(msg) + except Exception: + continue + mtype = payload.get("type", "") + mdata = payload.get("data", {}) or {} + + # Filter to our job (cloud broadcasts; local filters via client_id) + pid = mdata.get("prompt_id") + if pid is not None and pid != prompt_id: + continue + + if mtype == "progress": + if callable(on_progress): + on_progress({ + "type": "progress", + "value": mdata.get("value"), + "max": mdata.get("max"), + "node": mdata.get("node"), + }) + elif mtype == "progress_state": + if callable(on_progress): + on_progress({"type": "progress_state", "nodes": mdata.get("nodes", {})}) + elif mtype == "executing": + node = mdata.get("node") + if callable(on_progress): + on_progress({"type": "executing", "node": node}) + # When `node` is None on a local server, that signals end-of-run + if node is None and not self.is_cloud and seen_executed: + success = True + break + elif mtype == "executed": + seen_executed = True + nid = mdata.get("node") + out = mdata.get("output") or {} + if nid: + outputs[nid] = out + elif mtype == "notification": + if callable(on_progress): + on_progress({"type": "notification", "message": mdata.get("value", "")}) + elif mtype == "execution_success": + success = True + break + elif mtype == "execution_error": + error_payload = mdata + break + elif mtype == "execution_interrupted": + error_payload = {"interrupted": True, **mdata} + break + finally: + try: + ws.close() + except Exception: + pass + + if error_payload is not None: + return {"status": "error", "data": error_payload} + if success: + return {"status": "success", "outputs": outputs} + return {"status": "timeout", "elapsed": timeout} + + # ---------- outputs ---------- + def get_outputs(self, prompt_id: str) -> dict: + if self.is_cloud: + # Try /jobs/{id} first (returns full job with outputs); fall back to /history_v2 + r = http_get(self._url(f"/jobs/{prompt_id}"), headers=self.headers, retries=2) + if r.status == 200: + try: + return (r.json() or {}).get("outputs", {}) or {} + except Exception: + pass + # Fallback + r = http_get(self._url(f"/history/{prompt_id}"), headers=self.headers, retries=2) + if r.status == 200: + try: + body = r.json() or {} + except Exception: + body = {} + if isinstance(body, dict) and prompt_id in body: + return body[prompt_id].get("outputs", {}) or {} + if isinstance(body, dict) and "outputs" in body: + return body["outputs"] or {} + return {} + # Local + r = http_get(self._url(f"/history/{prompt_id}"), headers=self.headers, retries=2) + if r.status != 200: + return {} + try: + body = r.json() or {} + except Exception: + return {} + entry = body.get(prompt_id) or {} + return entry.get("outputs", {}) or {} + + def download_output( + self, *, filename: str, subfolder: str, file_type: str, + output_dir: Path, preserve_subfolder: bool = True, overwrite: bool = False, + ) -> Path: + """Stream a single output to disk. Path-traversal-safe.""" + params = {"filename": filename, "subfolder": subfolder, "type": file_type} + url = self._url("/view") + "?" + urlencode(params) + + # Compute target path safely. If preserve_subfolder, include subfolder in the + # local path; otherwise put the file in output_dir flat. + target_parts: list[str] = [] + if preserve_subfolder and subfolder: + target_parts.extend(p for p in subfolder.split("/") if p and p not in (".", "..")) + target_parts.append(filename) + out_path = safe_path_join(output_dir, *target_parts) + + if out_path.exists() and not overwrite: + stem, suffix = out_path.stem, out_path.suffix + i = 1 + while True: + candidate = out_path.with_name(f"{stem}_{i}{suffix}") + if not candidate.exists(): + out_path = candidate + break + i += 1 + + out_path.parent.mkdir(parents=True, exist_ok=True) + + # Stream download. Two-step for cloud: get the 302, then fetch signed URL + # so we don't accidentally send X-API-Key to the storage backend. + # The HTTP transport already strips X-API-Key on cross-host redirect + # via _strip_api_key_on_redirect, so a single follow_redirects=True call + # is safe AND simpler. + r = http_request( + "GET", url, headers=self.headers, + timeout=600, retries=3, follow_redirects=True, + stream=True, sink=out_path, + ) + if r.status != 200: + try: + if out_path.exists(): + out_path.unlink() + except Exception: + pass + raise WorkflowRunError( + "download_failed", + f"Download of {filename} failed: HTTP {r.status}", + url=url, + ) return out_path + # ---------- queue / cancel ---------- + def cancel(self, prompt_id: str | None = None) -> bool: + if prompt_id: + r = http_post( + self._url("/queue"), headers=self.headers, + json_body={"delete": [prompt_id]}, retries=1, + ) + return r.status == 200 + # Interrupt currently running + r = http_post(self._url("/interrupt"), headers=self.headers, retries=1) + return r.status == 200 -def load_schema(schema_path: str = None, workflow: dict = None) -> dict: - """Load or generate parameter schema.""" - if schema_path: - with open(schema_path) as f: - return json.load(f) - # Inline extraction (same logic as extract_schema.py but simplified) - if workflow is None: - return {"parameters": {}} - # Import from sibling script - script_dir = Path(__file__).parent - sys.path.insert(0, str(script_dir)) - from extract_schema import extract_schema + +# ============================================================================= +# Schema / parameter injection +# ============================================================================= + +def _inline_schema(workflow: dict) -> dict: + """Generate schema using the sibling extract_schema module.""" + from extract_schema import extract_schema # noqa: WPS433 return extract_schema(workflow) -def inject_params(workflow: dict, schema: dict, args: dict) -> dict: - """Inject user parameters into workflow based on schema mapping.""" +def load_schema(schema_path: str | None, workflow: dict) -> dict: + if schema_path: + with open(schema_path) as f: + return json.load(f) + return _inline_schema(workflow) + + +def inject_params( + workflow: dict, schema: dict, args: dict, + *, randomize_seed_if_unset: bool = False, +) -> tuple[dict, list[str]]: + """Inject user args into the workflow. Returns (new_workflow, warnings).""" wf = copy.deepcopy(workflow) - params = schema.get("parameters", {}) + params = schema.get("parameters", {}) or {} + warnings: list[str] = [] - for param_name, value in args.items(): - if param_name not in params: - print(f"Warning: unknown parameter '{param_name}', skipping", file=sys.stderr) + # Auto-randomize seed when it's -1 in args, or when randomize_seed_if_unset + # and user didn't pass a seed. + if "seed" in params: + if "seed" in args and args["seed"] in (None, -1, "-1"): + args = dict(args) + args["seed"] = coerce_seed(args["seed"]) + warnings.append(f"seed=-1 expanded to {args['seed']}") + elif randomize_seed_if_unset and "seed" not in args: + args = dict(args) + args["seed"] = coerce_seed(None) + warnings.append(f"seed auto-randomized to {args['seed']}") + + for name, value in args.items(): + if name not in params: + warnings.append(f"unknown parameter '{name}' (not in schema), skipping") continue - mapping = params[param_name] - node_id = mapping["node_id"] - field = mapping["field"] - if node_id in wf and "inputs" in wf[node_id]: - wf[node_id]["inputs"][field] = value - else: - print(f"Warning: node {node_id} not found in workflow", file=sys.stderr) + m = params[name] + nid, field = m["node_id"], m["field"] + node = wf.get(nid) + if not isinstance(node, dict) or "inputs" not in node: + warnings.append(f"node '{nid}' for parameter '{name}' missing in workflow") + continue + # Refuse to overwrite a link with a literal — would silently break wiring + cur = node["inputs"].get(field) + if is_link(cur): + warnings.append( + f"parameter '{name}' targets {nid}.{field} which is currently a link; " + f"refusing to overwrite (set the schema to point at the source node instead)" + ) + continue + node["inputs"][field] = value - return wf + return wf, warnings -def main(): - parser = argparse.ArgumentParser(description="Run a ComfyUI workflow with parameter injection") - parser.add_argument("--workflow", required=True, help="Path to workflow API JSON file") - parser.add_argument("--args", default="{}", help="JSON parameters to inject") - parser.add_argument("--schema", help="Path to schema JSON (from extract_schema.py). Auto-generated if omitted.") - parser.add_argument("--host", default="http://127.0.0.1:8188", help="ComfyUI server URL") - parser.add_argument("--api-key", help="API key for cloud (X-API-Key)") - parser.add_argument("--output-dir", default="./outputs", help="Directory to save outputs") - parser.add_argument("--timeout", type=int, default=120, help="Max seconds to wait for completion") - parser.add_argument("--no-download", action="store_true", help="Skip downloading outputs") - parser.add_argument("--submit-only", action="store_true", help="Submit and return prompt_id without waiting") - args = parser.parse_args() +# ============================================================================= +# Output download helper +# ============================================================================= - # Load workflow - workflow_path = Path(args.workflow) - if not workflow_path.exists(): - print(json.dumps({"error": f"Workflow file not found: {args.workflow}"})) - sys.exit(1) - with open(workflow_path) as f: - workflow = json.load(f) - - # Validate format - if "nodes" in workflow and "links" in workflow: - print(json.dumps({"error": "Workflow is in editor format, not API format. Re-export with 'Save (API Format)'."})) - sys.exit(1) - - # Parse user args - try: - user_args = json.loads(args.args) - except json.JSONDecodeError as e: - print(json.dumps({"error": f"Invalid --args JSON: {e}"})) - sys.exit(1) - - # Load/generate schema and inject params - schema = load_schema(args.schema, workflow) - if user_args: - workflow = inject_params(workflow, schema, user_args) - - # Connect to server - runner = ComfyRunner(host=args.host, api_key=args.api_key) - - # Check server - if not runner.check_server(): - print(json.dumps({"error": f"Cannot reach server at {args.host}. Is ComfyUI running?"})) - sys.exit(1) - - # Submit - result = runner.submit(workflow) - if "error" in result: - print(json.dumps({"error": "Submission failed", "details": result})) - sys.exit(1) - - prompt_id = result.get("prompt_id") - if not prompt_id: - print(json.dumps({"error": "No prompt_id in response", "response": result})) - sys.exit(1) - - # Check for node errors - node_errors = result.get("node_errors", {}) - if node_errors: - print(json.dumps({"error": "Workflow validation failed", "node_errors": node_errors})) - sys.exit(1) - - if args.submit_only: - print(json.dumps({"status": "submitted", "prompt_id": prompt_id})) - sys.exit(0) - - # Poll for completion - print(f"Submitted: {prompt_id}. Waiting...", file=sys.stderr) - poll_result = runner.poll_status(prompt_id, timeout=args.timeout) - - if poll_result["status"] == "timeout": - print(json.dumps({"status": "timeout", "prompt_id": prompt_id, "elapsed": poll_result["elapsed"]})) - sys.exit(1) - elif poll_result["status"] == "error": - print(json.dumps({"status": "error", "prompt_id": prompt_id, "details": poll_result.get("data")})) - sys.exit(1) - elif poll_result["status"] == "cancelled": - print(json.dumps({"status": "cancelled", "prompt_id": prompt_id})) - sys.exit(1) - - # Download outputs - outputs = poll_result.get("outputs") or runner.get_outputs(prompt_id) - if args.no_download: - print(json.dumps({"status": "success", "prompt_id": prompt_id, "outputs": outputs})) - sys.exit(0) - - output_dir = Path(args.output_dir) +def download_outputs( + runner: ComfyRunner, outputs: dict, output_dir: Path, + *, preserve_subfolder: bool = True, overwrite: bool = False, +) -> list[dict]: + """Walk the outputs dict and download every file. Cloud uses `video` (singular); + local uses `videos` (plural). We accept both.""" output_dir.mkdir(parents=True, exist_ok=True) + downloaded: list[dict] = [] - downloaded = [] - for node_id, node_output in outputs.items(): - # ComfyUI puts images/videos under "images" key (even for video) - for key in ("images", "gifs", "videos", "audio"): - if key not in node_output: + OUTPUT_KEYS = ("images", "gifs", "videos", "video", "audio", "files", "models", "3d") + + for node_id, node_output in (outputs or {}).items(): + if not isinstance(node_output, dict): + continue + for key in OUTPUT_KEYS: + entries = node_output.get(key) + if not entries: continue - for file_info in node_output[key]: - filename = file_info.get("filename", "") - subfolder = file_info.get("subfolder", "") - file_type = file_info.get("type", "output") + if not isinstance(entries, list): + entries = [entries] + for fi in entries: + if not isinstance(fi, dict): + continue + filename = fi.get("filename") or "" if not filename: continue + subfolder = fi.get("subfolder") or "" + file_type = fi.get("type") or "output" try: - out_path = runner.download_output(filename, subfolder, file_type, output_dir) - # Detect media type from extension - ext = Path(filename).suffix.lower() - if ext in (".mp4", ".webm", ".avi", ".mov", ".gif"): - media_type = "video" - elif ext in (".wav", ".mp3", ".flac", ".ogg"): - media_type = "audio" - else: - media_type = "image" + out_path = runner.download_output( + filename=filename, subfolder=subfolder, file_type=file_type, + output_dir=output_dir, preserve_subfolder=preserve_subfolder, + overwrite=overwrite, + ) downloaded.append({ "file": str(out_path), "node_id": node_id, - "type": media_type, + "type": media_type_from_filename(filename), "filename": filename, + "subfolder": subfolder, + "source_type": file_type, }) except Exception as e: - print(f"Warning: failed to download {filename}: {e}", file=sys.stderr) + log(f"WARN: failed to download {filename}: {e}") + return downloaded - print(json.dumps({ + +# ============================================================================= +# CLI +# ============================================================================= + +def parse_input_image_arg(spec: str) -> tuple[str, Path]: + """Parse `name=path` (or `path` alone, defaulting to name='image').""" + if "=" in spec: + name, path = spec.split("=", 1) + return name.strip(), Path(path).expanduser() + return "image", Path(spec).expanduser() + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser( + description="Run a ComfyUI workflow with parameter injection.", + formatter_class=argparse.RawDescriptionHelpFormatter, + ) + p.add_argument("--workflow", required=True, help="Path to workflow API JSON file") + p.add_argument("--args", default="{}", + help="JSON parameters to inject (or `@/path/to/args.json`)") + p.add_argument("--schema", help="Path to schema JSON (auto-generated if omitted)") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL") + p.add_argument("--api-key", + help=f"API key for cloud (or set ${ENV_API_KEY} env var)") + p.add_argument("--partner-key", + help="Partner-node API key (extra_data.api_key_comfy_org). " + "Required for Flux Pro / Ideogram / etc. Defaults to --api-key if not set.") + p.add_argument("--output-dir", default="./outputs", help="Directory to save outputs") + p.add_argument("--timeout", type=int, default=0, + help="Max seconds to wait (0=auto: 300 / 900 for video workflows)") + p.add_argument("--input-image", action="append", default=[], + help="Upload local image before running. Format: `name=path` or `path`. " + "The `name` becomes the value injected into the matching schema parameter.") + p.add_argument("--randomize-seed", action="store_true", + help="If schema has a 'seed' parameter and --args didn't set one, randomize it") + p.add_argument("--ws", action="store_true", + help="Use WebSocket for real-time progress (requires `websocket-client`)") + p.add_argument("--no-download", action="store_true", help="Skip downloading outputs") + p.add_argument("--flat-output", action="store_true", + help="Don't preserve server-side subfolder structure when saving outputs") + p.add_argument("--overwrite", action="store_true", + help="Overwrite existing files instead of appending _1, _2, ...") + p.add_argument("--submit-only", action="store_true", + help="Submit and return prompt_id without waiting") + p.add_argument("--client-id", help="Override generated client_id (UUID)") + p.add_argument("--use-partner-key-as-auth", action="store_true", + help="(Compat) Use --partner-key value as cloud X-API-Key. Don't use unless you know why.") + + args = p.parse_args(argv) + + # ---- Load workflow ---- + wf_path = Path(args.workflow).expanduser() + if not wf_path.exists(): + emit_json({"error": f"Workflow file not found: {args.workflow}"}) + return 1 + try: + with wf_path.open() as f: + workflow_raw = json.load(f) + workflow = unwrap_workflow(workflow_raw) + except ValueError as e: + emit_json({"error": str(e)}) + return 1 + except json.JSONDecodeError as e: + emit_json({"error": f"Invalid JSON in workflow file: {e}"}) + return 1 + + # ---- Parse user args ---- + args_str = args.args + if args_str.startswith("@"): + try: + args_str = Path(args_str[1:]).read_text() + except OSError as e: + emit_json({"error": f"Cannot read args file: {e}"}) + return 1 + try: + user_args = json.loads(args_str) if args_str.strip() else {} + except json.JSONDecodeError as e: + emit_json({"error": f"Invalid --args JSON: {e}"}) + return 1 + if not isinstance(user_args, dict): + emit_json({"error": "--args must be a JSON object"}) + return 1 + + # ---- Resolve API key ---- + api_key = resolve_api_key(args.api_key) + partner_key = args.partner_key or None + if args.use_partner_key_as_auth and not api_key and partner_key: + api_key = partner_key + + # ---- Connect ---- + runner = ComfyRunner( + host=args.host, api_key=api_key, partner_key=partner_key, + client_id=args.client_id, + ) + + # Server reachability + ok, info = runner.check_server() + if not ok: + emit_json({ + "error": f"Cannot reach server at {args.host}", + "details": info, + "hint": ( + "Check `comfy launch --background` is running for local, " + f"or set ${ENV_API_KEY} for cloud." + ), + }) + return 1 + + # ---- Upload input images ---- + upload_warnings: list[str] = [] + for spec in args.input_image: + try: + param_name, path = parse_input_image_arg(spec) + except Exception as e: + emit_json({"error": f"Bad --input-image spec '{spec}': {e}"}) + return 1 + try: + ref = runner.upload_image(path) + except Exception as e: + emit_json({"error": f"Upload failed for {path}: {e}"}) + return 1 + # Register as a user arg so inject_params consumes it through the schema + uploaded_name = ref.get("name") or path.name + if param_name not in user_args: + user_args[param_name] = uploaded_name + + # ---- Inject params ---- + schema = load_schema(args.schema, workflow) + workflow, inj_warnings = inject_params( + workflow, schema, user_args, randomize_seed_if_unset=args.randomize_seed, + ) + warnings = upload_warnings + inj_warnings + for w in warnings: + log(f"WARN: {w}") + + # ---- Submit ---- + submit_resp = runner.submit(workflow) + if "_http_error" in submit_resp: + emit_json({ + "error": "Submission HTTP error", + "http_status": submit_resp["_http_error"], + "body": submit_resp.get("body"), + }) + return 1 + + if isinstance(submit_resp.get("error"), dict): + emit_json({ + "error": "Workflow validation failed", + "details": submit_resp["error"], + "node_errors": submit_resp.get("node_errors"), + }) + return 1 + + prompt_id = submit_resp.get("prompt_id") + if not prompt_id: + emit_json({"error": "No prompt_id in submit response", "response": submit_resp}) + return 1 + + node_errors = submit_resp.get("node_errors") or {} + if node_errors: + emit_json({"error": "Workflow validation failed", "node_errors": node_errors}) + return 1 + + if args.submit_only: + emit_json({"status": "submitted", "prompt_id": prompt_id, "warnings": warnings}) + return 0 + + # ---- Wait ---- + timeout = args.timeout + if timeout <= 0: + timeout = 900 if looks_like_video_workflow(workflow) else 300 + + log(f"Submitted: prompt_id={prompt_id}, waiting (timeout={timeout}s)…") + + def _on_progress(evt: dict) -> None: + t = evt.get("type") + if t == "progress": + log(f" step {evt.get('value')}/{evt.get('max')} on node {evt.get('node')}") + elif t == "executing": + node = evt.get("node") + if node: + log(f" executing node {node}") + + try: + if args.ws: + wait_result = runner.monitor_ws(prompt_id, timeout=timeout, on_progress=_on_progress) + else: + wait_result = runner.poll_status(prompt_id, timeout=timeout) + except KeyboardInterrupt: + log(f"Interrupted — cancelling job {prompt_id} on server…") + try: + runner.cancel(prompt_id) + except Exception as e: + log(f" (cancel request failed: {e})") + emit_json({ + "status": "interrupted", + "prompt_id": prompt_id, + "note": "Ctrl+C received; sent cancellation to server.", + }) + return 130 + + if wait_result["status"] == "timeout": + emit_json({ + "status": "timeout", + "prompt_id": prompt_id, + "elapsed": wait_result.get("elapsed"), + "hint": "Re-run with larger --timeout, or use --submit-only and check later.", + }) + return 1 + if wait_result["status"] == "error": + emit_json({"status": "error", "prompt_id": prompt_id, "details": wait_result.get("data")}) + return 1 + if wait_result["status"] == "cancelled": + emit_json({"status": "cancelled", "prompt_id": prompt_id}) + return 1 + + # ---- Outputs ---- + outputs = wait_result.get("outputs") + if not outputs: + outputs = runner.get_outputs(prompt_id) + + if args.no_download: + emit_json({ + "status": "success", "prompt_id": prompt_id, + "outputs": outputs, "warnings": warnings, + }) + return 0 + + downloaded = download_outputs( + runner, outputs, Path(args.output_dir).expanduser(), + preserve_subfolder=not args.flat_output, overwrite=args.overwrite, + ) + + emit_json({ "status": "success", "prompt_id": prompt_id, "outputs": downloaded, - }, indent=2)) + "warnings": warnings, + }) + return 0 if __name__ == "__main__": - main() + sys.exit(main()) diff --git a/skills/creative/comfyui/scripts/ws_monitor.py b/skills/creative/comfyui/scripts/ws_monitor.py new file mode 100755 index 00000000000..b8689655bd0 --- /dev/null +++ b/skills/creative/comfyui/scripts/ws_monitor.py @@ -0,0 +1,267 @@ +#!/usr/bin/env python3 +""" +ws_monitor.py — Real-time ComfyUI WebSocket monitor. + +Connects to /ws and pretty-prints execution events: node start/finish, sampling +progress, cached nodes, errors. Optionally writes preview frames to disk. + +Useful for: + - Watching a long-running job in real time without parsing JSON yourself + - Saving in-progress preview frames for video / animation workflows + - Debugging "why is this hanging?" — see exactly which node is stuck + +Usage: + # Local — watch all jobs from this client_id + python3 ws_monitor.py + + # Cloud — watch a specific prompt_id + python3 ws_monitor.py --host https://cloud.comfy.org \ + --prompt-id abc-123-def + + # Save preview frames to ./previews/ + python3 ws_monitor.py --previews ./previews + +Requires: websocket-client (`pip install websocket-client`). +Falls back to a clear error message when not installed. +""" + +from __future__ import annotations + +import argparse +import json +import struct +import sys +from pathlib import Path +from urllib.parse import urlparse + +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _common import ( # noqa: E402 + DEFAULT_LOCAL_HOST, ENV_API_KEY, log, new_client_id, resolve_api_key, is_cloud_host, +) + + +# Binary frame types from ComfyUI WebSocket protocol +BINARY_PREVIEW_IMAGE = 1 +BINARY_TEXT = 3 +BINARY_PREVIEW_IMAGE_WITH_METADATA = 4 + +# Image type codes inside PREVIEW_IMAGE +IMAGE_TYPE_JPEG = 1 +IMAGE_TYPE_PNG = 2 + +# ANSI escape codes (works on most modern terminals) +RESET = "\033[0m" +DIM = "\033[2m" +BOLD = "\033[1m" +GREEN = "\033[32m" +YELLOW = "\033[33m" +RED = "\033[31m" +CYAN = "\033[36m" + + +def fmt_color(s: str, color: str, *, color_on: bool = True) -> str: + return f"{color}{s}{RESET}" if color_on else s + + +def parse_binary_frame(data: bytes) -> dict | None: + if len(data) < 8: + return None + type_code = struct.unpack(">I", data[0:4])[0] + if type_code == BINARY_PREVIEW_IMAGE: + image_type = struct.unpack(">I", data[4:8])[0] + ext = "jpg" if image_type == IMAGE_TYPE_JPEG else "png" if image_type == IMAGE_TYPE_PNG else "bin" + return { + "kind": "preview", + "image_type": image_type, + "ext": ext, + "image_bytes": data[8:], + } + if type_code == BINARY_PREVIEW_IMAGE_WITH_METADATA: + if len(data) < 12: + return None + meta_len = struct.unpack(">I", data[4:8])[0] + meta_end = 8 + meta_len + if len(data) < meta_end: + return None + try: + meta = json.loads(data[8:meta_end].decode("utf-8")) + except Exception: + meta = {"raw": data[8:meta_end][:200].decode("utf-8", "replace")} + return { + "kind": "preview_with_metadata", + "metadata": meta, + "image_bytes": data[meta_end:], + "ext": "png", + } + if type_code == BINARY_TEXT: + if len(data) < 8: + return None + nid_len = struct.unpack(">I", data[4:8])[0] + nid_end = 8 + nid_len + if len(data) < nid_end: + return None + return { + "kind": "text", + "node_id": data[8:nid_end].decode("utf-8", "replace"), + "text": data[nid_end:].decode("utf-8", "replace"), + } + return {"kind": "unknown", "type_code": type_code, "size": len(data)} + + +def main(argv: list[str] | None = None) -> int: + p = argparse.ArgumentParser(description="Real-time ComfyUI WebSocket monitor") + p.add_argument("--host", default=DEFAULT_LOCAL_HOST, help="ComfyUI server URL") + p.add_argument("--api-key", help=f"API key for cloud (or set ${ENV_API_KEY} env var)") + p.add_argument("--client-id", default=None, help="Client ID (default: random UUID)") + p.add_argument("--prompt-id", default=None, + help="Filter to a specific prompt_id (default: all jobs)") + p.add_argument("--previews", default=None, + help="Directory to save in-progress preview frames") + p.add_argument("--no-color", action="store_true", help="Disable ANSI colour") + p.add_argument("--timeout", type=float, default=600.0, + help="Hard cap on monitor duration (default 600s)") + args = p.parse_args(argv) + + try: + import websocket # type: ignore[import-not-found] + except ImportError: + print(json.dumps({ + "error": "websocket-client not installed", + "install": "pip install websocket-client", + })) + return 1 + + api_key = resolve_api_key(args.api_key) + cloud = is_cloud_host(args.host) + client_id = args.client_id or new_client_id() + + # Build WS URL preserving any base-path component (e.g. behind reverse proxy). + parsed = urlparse(args.host if "://" in args.host else f"http://{args.host}") + scheme = "wss" if parsed.scheme == "https" else "ws" + netloc = parsed.netloc + base_path = parsed.path.rstrip("/") + ws_url = f"{scheme}://{netloc}{base_path}/ws?clientId={client_id}" + if cloud and api_key: + ws_url += f"&token={api_key}" + + color_on = not args.no_color and sys.stdout.isatty() + + preview_dir = Path(args.previews).expanduser() if args.previews else None + if preview_dir: + preview_dir.mkdir(parents=True, exist_ok=True) + log(f"Saving previews to {preview_dir}") + + log(f"Connecting to {ws_url} (client_id={client_id})") + if args.prompt_id: + log(f"Filtering messages to prompt_id={args.prompt_id}") + + ws = websocket.create_connection(ws_url, timeout=args.timeout) + ws.settimeout(args.timeout) + + preview_counter = 0 + try: + while True: + try: + msg = ws.recv() + except websocket.WebSocketTimeoutException: + log(f"Idle for {args.timeout}s — exiting") + return 0 + if isinstance(msg, bytes): + parsed = parse_binary_frame(msg) + if parsed is None: + continue + if parsed["kind"] in ("preview", "preview_with_metadata") and preview_dir: + img_bytes = parsed.get("image_bytes", b"") + if img_bytes: + ext = parsed.get("ext", "png") + out = preview_dir / f"preview_{preview_counter:05d}.{ext}" + out.write_bytes(img_bytes) + preview_counter += 1 + log(f" [preview] saved {out.name} ({len(img_bytes)} bytes)") + continue + + try: + payload = json.loads(msg) + except Exception: + continue + mtype = payload.get("type", "") + mdata = payload.get("data", {}) or {} + pid = mdata.get("prompt_id") + + if args.prompt_id and pid and pid != args.prompt_id: + continue + + if mtype == "status": + qr = mdata.get("status", {}).get("exec_info", {}).get("queue_remaining", "?") + print(fmt_color(f"[status] queue_remaining={qr}", DIM, color_on=color_on)) + elif mtype == "execution_start": + print(fmt_color(f"[start] prompt_id={pid}", BOLD, color_on=color_on)) + elif mtype == "executing": + node = mdata.get("node") + if node: + print(fmt_color(f" [executing] node={node}", CYAN, color_on=color_on)) + else: + print(fmt_color(f" [executing] (workflow done) prompt_id={pid}", DIM, color_on=color_on)) + elif mtype == "progress": + v, m = mdata.get("value", 0), mdata.get("max", 0) + pct = (v / m * 100) if m else 0 + print(f" [progress] {v}/{m} ({pct:5.1f}%) node={mdata.get('node')}") + elif mtype == "progress_state": + # Newer extended progress message + nodes = mdata.get("nodes") or {} + running = [k for k, v in nodes.items() if v.get("running")] + if running: + print(fmt_color(f" [progress_state] running={running}", DIM, color_on=color_on)) + elif mtype == "executed": + node = mdata.get("node") + out = mdata.get("output") or {} + summary_parts = [] + for key in ("images", "video", "videos", "gifs", "audio", "files"): + if out.get(key): + summary_parts.append(f"{key}={len(out[key])}") + summary = ", ".join(summary_parts) if summary_parts else "(no files)" + print(fmt_color(f" [executed] node={node} {summary}", GREEN, color_on=color_on)) + elif mtype == "execution_cached": + cached = mdata.get("nodes") or [] + if cached: + print(fmt_color(f" [cached] {len(cached)} nodes skipped", DIM, color_on=color_on)) + elif mtype == "execution_success": + print(fmt_color(f"[success] prompt_id={pid}", GREEN + BOLD, color_on=color_on)) + if args.prompt_id: + return 0 + elif mtype == "execution_error": + exc_type = mdata.get("exception_type", "?") + exc_msg = mdata.get("exception_message", "?") + print(fmt_color(f"[error] {exc_type}: {exc_msg}", RED + BOLD, color_on=color_on)) + tb = mdata.get("traceback") + if tb: + if isinstance(tb, list): + for line in tb: + print(fmt_color(f" {line}", RED, color_on=color_on)) + else: + print(fmt_color(f" {tb}", RED, color_on=color_on)) + if args.prompt_id: + return 1 + elif mtype == "execution_interrupted": + print(fmt_color(f"[interrupted] prompt_id={pid}", YELLOW, color_on=color_on)) + if args.prompt_id: + return 1 + elif mtype == "notification": + v = mdata.get("value", "") + print(fmt_color(f"[notification] {v}", DIM, color_on=color_on)) + else: + # Unknown / lightly-used types: print compactly + print(fmt_color(f"[{mtype}] {json.dumps(mdata, default=str)[:200]}", DIM, color_on=color_on)) + + except KeyboardInterrupt: + log("Interrupted") + return 130 + finally: + try: + ws.close() + except Exception: + pass + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/skills/creative/comfyui/tests/README.md b/skills/creative/comfyui/tests/README.md new file mode 100644 index 00000000000..833632ae9c4 --- /dev/null +++ b/skills/creative/comfyui/tests/README.md @@ -0,0 +1,50 @@ +# ComfyUI Skill Tests + +Pytest suite covering the skill's scripts. Pure-stdlib unit tests run +without any setup; cloud integration tests need a Comfy Cloud API key. + +## Running + +```bash +# Unit tests only (no network required) — runs in <1s +python3 -m pytest tests/ -c tests/pytest.ini -o addopts="-p no:xdist" + +# Including cloud integration tests +COMFY_CLOUD_API_KEY="comfyui-..." python3 -m pytest tests/ \ + -c tests/pytest.ini -o addopts="-p no:xdist" + +# Just cloud tests +COMFY_CLOUD_API_KEY="comfyui-..." python3 -m pytest tests/test_cloud_integration.py \ + -c tests/pytest.ini -o addopts="-p no:xdist" -v +``` + +The `-c` and `-o` overrides isolate this suite from any parent +`pyproject.toml` pytest config (e.g. the `-n auto` from a parent repo). + +## Test files + +| File | Coverage | +|------|----------| +| `test_common.py` | Cloud detection, URL routing, format validation, embeddings, paths, seeds, model-list parsing, folder aliases | +| `test_extract_schema.py` | Connection tracing, positive/negative prompt detection, dedup logic, embedding deps | +| `test_run_workflow.py` | Param injection (incl. -1 seed, link refusal), output download walk, runner construction | +| `test_check_deps.py` | Model-name fuzzy matching, install command suggestions | +| `test_cloud_integration.py` | Live cloud API contract tests (auto-skipped without API key) | + +## Adding tests + +When you change a script: + +1. Add a unit test if the change is pure logic (cloud detection, parsing, etc.) +2. Add a cloud integration test if the change depends on cloud API behavior + (use `pytestmark = pytest.mark.cloud` so it auto-skips without a key) +3. Workflow fixtures live in `conftest.py` (`sd15_workflow`, `flux_workflow`, + `video_workflow`) + +## Why the explicit `-c` / `-o`? + +The parent hermes-agent repo's `pyproject.toml` enables `pytest-xdist` by +default (`-n auto`). This suite is small enough that parallelism isn't +worth the complexity, and pytest-xdist isn't always installed in the user's +environment. The `-c tests/pytest.ini -o addopts="-p no:xdist"` flags make +the suite run identically regardless of the parent project's config. diff --git a/skills/creative/comfyui/tests/conftest.py b/skills/creative/comfyui/tests/conftest.py new file mode 100644 index 00000000000..a800fa79f1b --- /dev/null +++ b/skills/creative/comfyui/tests/conftest.py @@ -0,0 +1,64 @@ +"""Pytest configuration for the comfyui skill test suite. + +Adds `scripts/` to sys.path so tests can `from _common import ...`, and +provides a few common fixtures. +""" + +from __future__ import annotations + +import json +import os +import sys +from pathlib import Path + +import pytest + +ROOT = Path(__file__).resolve().parent.parent +SCRIPTS = ROOT / "scripts" +WORKFLOWS = ROOT / "workflows" + +sys.path.insert(0, str(SCRIPTS)) + + +@pytest.fixture +def sd15_workflow() -> dict: + return json.loads((WORKFLOWS / "sd15_txt2img.json").read_text()) + + +@pytest.fixture +def flux_workflow() -> dict: + return json.loads((WORKFLOWS / "flux_dev_txt2img.json").read_text()) + + +@pytest.fixture +def video_workflow() -> dict: + return json.loads((WORKFLOWS / "wan_video_t2v.json").read_text()) + + +@pytest.fixture +def workflows_dir() -> Path: + return WORKFLOWS + + +@pytest.fixture +def scripts_dir() -> Path: + return SCRIPTS + + +@pytest.fixture +def cloud_key() -> str | None: + """Cloud API key if set, otherwise None. + + Tests that need cloud connectivity should skip when this is None. + """ + return os.environ.get("COMFY_CLOUD_API_KEY") + + +def pytest_collection_modifyitems(config, items): + """Auto-skip cloud tests when no API key is set.""" + if os.environ.get("COMFY_CLOUD_API_KEY"): + return + skip_cloud = pytest.mark.skip(reason="Set COMFY_CLOUD_API_KEY to run cloud tests") + for item in items: + if "cloud" in item.keywords: + item.add_marker(skip_cloud) diff --git a/skills/creative/comfyui/tests/pytest.ini b/skills/creative/comfyui/tests/pytest.ini new file mode 100644 index 00000000000..2111fe21227 --- /dev/null +++ b/skills/creative/comfyui/tests/pytest.ini @@ -0,0 +1,5 @@ +[pytest] +markers = + cloud: tests that hit live Comfy Cloud API (require COMFY_CLOUD_API_KEY) +testpaths = . +addopts = -p no:xdist diff --git a/skills/creative/comfyui/tests/test_check_deps.py b/skills/creative/comfyui/tests/test_check_deps.py new file mode 100644 index 00000000000..c5d36b34189 --- /dev/null +++ b/skills/creative/comfyui/tests/test_check_deps.py @@ -0,0 +1,65 @@ +"""Tests for check_deps.py — focuses on parsing logic that doesn't need a server.""" + +from __future__ import annotations + +from check_deps import ( + NODE_TO_PACKAGE, + model_present, + normalize_for_match, + suggest_install_command, +) + + +class TestNormalizeForMatch: + def test_basic(self): + s = normalize_for_match("model.safetensors") + assert "model.safetensors" in s + assert "model" in s + + def test_subfolder(self): + s = normalize_for_match("subdir/model.pt") + assert "subdir/model.pt" in s + assert "model.pt" in s + assert "model" in s + + +class TestModelPresent: + def test_exact_match(self): + assert model_present("a.safetensors", {"a.safetensors", "b.safetensors"}) is True + + def test_extension_difference(self): + # User said "model" but installed is "model.safetensors" + assert model_present("model", {"model.safetensors"}) is True + # Reverse direction — also matches + assert model_present("model.safetensors", {"model"}) is True + + def test_subfolder_match(self): + # Installed list has "subdir/model.safetensors", workflow asks "model.safetensors" + assert model_present("model.safetensors", {"subdir/model.safetensors"}) is True + + def test_missing(self): + assert model_present("missing.safetensors", {"a.safetensors", "b.safetensors"}) is False + + def test_empty_installed(self): + assert model_present("anything.safetensors", set()) is False + + +class TestSuggestInstallCommand: + def test_known_node(self): + cmd = suggest_install_command("VHS_VideoCombine") + assert cmd == "comfy node install comfyui-videohelpersuite" + + def test_unknown_node(self): + assert suggest_install_command("SomeRandomNodeName123") is None + + +class TestNodePackageMap: + def test_no_duplicates(self): + # Each node should map to exactly one package + keys = list(NODE_TO_PACKAGE.keys()) + assert len(keys) == len(set(keys)) + + def test_all_lowercase_packages(self): + # Convention: package names are lowercase with hyphens/underscores + for pkg in NODE_TO_PACKAGE.values(): + assert pkg.lower() == pkg, f"Package name should be lowercase: {pkg}" diff --git a/skills/creative/comfyui/tests/test_cloud_integration.py b/skills/creative/comfyui/tests/test_cloud_integration.py new file mode 100644 index 00000000000..eb7b04ca225 --- /dev/null +++ b/skills/creative/comfyui/tests/test_cloud_integration.py @@ -0,0 +1,95 @@ +"""Integration tests against the live Comfy Cloud API. + +These tests are auto-skipped when COMFY_CLOUD_API_KEY is not set. +They never SUBMIT workflows (would need a paid subscription) — they only +verify the read-only endpoints we rely on. +""" + +from __future__ import annotations + +import pytest + +from _common import http_get, parse_model_list, resolve_url + + +pytestmark = pytest.mark.cloud + + +class TestCloudEndpointsLive: + def test_system_stats_reachable(self, cloud_key): + url = resolve_url("https://cloud.comfy.org", "/system_stats") + r = http_get(url, headers={"X-API-Key": cloud_key}) + assert r.status == 200 + data = r.json() + assert "system" in data + + def test_models_endpoint_routed_to_experiment(self, cloud_key): + # We expect the skill to route /models/checkpoints → /api/experiment/models/checkpoints + url = resolve_url("https://cloud.comfy.org", "/models/checkpoints") + assert "/api/experiment/models/checkpoints" in url + r = http_get(url, headers={"X-API-Key": cloud_key}) + assert r.status == 200 + + def test_models_endpoint_returns_dicts(self, cloud_key): + url = resolve_url("https://cloud.comfy.org", "/models/checkpoints") + r = http_get(url, headers={"X-API-Key": cloud_key}) + data = r.json() + assert isinstance(data, list) + if data: + # Cloud format: list of dicts with `name` + assert isinstance(data[0], dict) + assert "name" in data[0] + # Our parser normalizes both + normalized = parse_model_list(data) + assert len(normalized) == len(data) + + def test_history_renamed_to_v2(self, cloud_key): + # /history → /api/history_v2 on cloud + url = resolve_url("https://cloud.comfy.org", "/history/some-fake-id") + assert "/api/history_v2/some-fake-id" in url + + def test_object_info_paid_tier(self, cloud_key): + # On free tier, /object_info returns 403 with a recognizable message + url = resolve_url("https://cloud.comfy.org", "/object_info") + r = http_get(url, headers={"X-API-Key": cloud_key}) + # Should be either 200 (paid) or 403 (free) — not 404 / 500 + assert r.status in (200, 403) + if r.status == 403: + # Body should mention the limitation + assert "free tier" in r.text().lower() or "subscription" in r.text().lower() + + +class TestCloudCheckDepsLive: + def test_check_deps_against_cloud(self, cloud_key, sd15_workflow): + from check_deps import check_deps + report = check_deps(sd15_workflow, host="https://cloud.comfy.org", api_key=cloud_key) + # Either node check passed OR was skipped (free tier) + assert "missing_models" in report + assert "is_cloud" in report and report["is_cloud"] is True + + def test_flux_workflow_models_resolved_via_aliases(self, cloud_key, flux_workflow): + """Flux uses unet/clip folders; cloud has them in diffusion_models/text_encoders. + With folder aliasing, the check should still find them.""" + from check_deps import check_deps + report = check_deps(flux_workflow, host="https://cloud.comfy.org", api_key=cloud_key) + # The exact required Flux files (flux1-dev.safetensors, t5xxl_fp16, clip_l, ae) + # are present on cloud; with folder aliasing, none should be missing. + # If this fails, either the cloud removed the model or the aliasing logic broke. + missing_filenames = {m["value"] for m in report["missing_models"]} + assert "ae.safetensors" not in missing_filenames, \ + "ae.safetensors should be on cloud's vae folder" + # t5xxl_fp16 / clip_l should be reachable via the clip → text_encoders alias + # flux1-dev.safetensors likewise via unet → diffusion_models + + +class TestHealthCheckLive: + def test_health_check_passes(self, cloud_key, capsys): + from health_check import main as health_main + rc = health_main(["--host", "https://cloud.comfy.org", "--api-key", cloud_key]) + captured = capsys.readouterr() + # Should produce JSON + import json + report = json.loads(captured.out) + assert report["server"]["reachable"] is True + assert report["checkpoints"]["queryable"] is True + assert report["checkpoints"]["count"] > 0 diff --git a/skills/creative/comfyui/tests/test_common.py b/skills/creative/comfyui/tests/test_common.py new file mode 100644 index 00000000000..0263fe1d91b --- /dev/null +++ b/skills/creative/comfyui/tests/test_common.py @@ -0,0 +1,447 @@ +"""Unit tests for _common.py — pure logic only, no network.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +from _common import ( + DEFAULT_LOCAL_HOST, + EMBEDDING_REGEX, + FOLDER_ALIASES, + build_cloud_aware_url, + cloud_endpoint, + coerce_seed, + folder_aliases_for, + is_api_format, + is_cloud_host, + is_link, + iter_embedding_refs, + iter_model_deps, + iter_nodes, + looks_like_video_workflow, + media_type_from_filename, + parse_model_list, + resolve_url, + safe_path_join, + unwrap_workflow, +) + + +# ============================================================================= +# Cloud detection / URL routing +# ============================================================================= + +class TestCloudDetection: + def test_cloud_host_exact(self): + assert is_cloud_host("https://cloud.comfy.org") is True + assert is_cloud_host("https://cloud.comfy.org/foo/bar") is True + + def test_cloud_host_subdomain(self): + assert is_cloud_host("https://staging.cloud.comfy.org") is True + assert is_cloud_host("https://api.cloud.comfy.org") is True + + def test_local_not_cloud(self): + assert is_cloud_host("http://127.0.0.1:8188") is False + assert is_cloud_host("http://localhost:8188") is False + assert is_cloud_host("http://my-server.local:8188") is False + + def test_no_scheme(self): + # Defaults to http:// + assert is_cloud_host("cloud.comfy.org") is True + assert is_cloud_host("127.0.0.1:8188") is False + + +class TestCloudEndpointRename: + def test_history_renamed(self): + assert cloud_endpoint("/history") == "/history_v2" + assert cloud_endpoint("/history/abc-123") == "/history_v2/abc-123" + + def test_history_v2_preserved(self): + assert cloud_endpoint("/history_v2") == "/history_v2" + + def test_models_renamed(self): + assert cloud_endpoint("/models") == "/experiment/models" + assert cloud_endpoint("/models/checkpoints") == "/experiment/models/checkpoints" + assert cloud_endpoint("/models/loras") == "/experiment/models/loras" + + def test_other_paths_unchanged(self): + assert cloud_endpoint("/prompt") == "/prompt" + assert cloud_endpoint("/queue") == "/queue" + + +class TestResolveURL: + def test_local_no_prefix(self): + assert resolve_url("http://127.0.0.1:8188", "/prompt") == "http://127.0.0.1:8188/prompt" + + def test_cloud_adds_api_prefix(self): + assert resolve_url("https://cloud.comfy.org", "/prompt") == "https://cloud.comfy.org/api/prompt" + + def test_cloud_history_renamed(self): + assert resolve_url("https://cloud.comfy.org", "/history/abc") == "https://cloud.comfy.org/api/history_v2/abc" + + def test_cloud_models_renamed(self): + assert resolve_url("https://cloud.comfy.org", "/models/loras") == "https://cloud.comfy.org/api/experiment/models/loras" + + def test_cloud_already_has_api(self): + # Don't double-prefix + assert resolve_url("https://cloud.comfy.org", "/api/prompt") == "https://cloud.comfy.org/api/prompt" + + def test_trailing_slash_stripped(self): + assert resolve_url("http://127.0.0.1:8188/", "/prompt") == "http://127.0.0.1:8188/prompt" + + +# ============================================================================= +# Workflow validation +# ============================================================================= + +class TestAPIFormatDetection: + def test_valid_api(self, sd15_workflow): + assert is_api_format(sd15_workflow) is True + + def test_editor_format_rejected(self): + editor = {"nodes": [], "links": [], "version": 0.4} + assert is_api_format(editor) is False + + def test_empty_dict(self): + assert is_api_format({}) is False + + def test_non_dict(self): + assert is_api_format([]) is False + assert is_api_format(None) is False + assert is_api_format("string") is False + + def test_node_with_class_type(self): + wf = {"3": {"class_type": "KSampler", "inputs": {}}} + assert is_api_format(wf) is True + + +class TestUnwrapWorkflow: + def test_passthrough_api_format(self, sd15_workflow): + result = unwrap_workflow(sd15_workflow) + assert result is sd15_workflow + + def test_unwrap_prompt_key(self, sd15_workflow): + wrapped = {"prompt": sd15_workflow, "client_id": "abc"} + result = unwrap_workflow(wrapped) + assert result is sd15_workflow + + def test_editor_format_raises(self): + with pytest.raises(ValueError, match="editor format"): + unwrap_workflow({"nodes": [], "links": []}) + + def test_garbage_raises(self): + with pytest.raises(ValueError): + unwrap_workflow({"foo": "bar"}) + + +class TestIsLink: + def test_valid_link(self): + assert is_link(["3", 0]) is True + assert is_link(["10", 1]) is True + + def test_non_link(self): + assert is_link("string") is False + assert is_link(42) is False + assert is_link([]) is False + assert is_link(["3"]) is False # missing slot + assert is_link(["3", "0"]) is False # slot must be int + assert is_link([3, 0]) is False # node_id must be string + + +# ============================================================================= +# Workflow iterators +# ============================================================================= + +class TestIterators: + def test_iter_nodes(self, sd15_workflow): + nodes = dict(iter_nodes(sd15_workflow)) + assert "3" in nodes + assert nodes["3"]["class_type"] == "KSampler" + + def test_iter_nodes_skips_comments(self, sd15_workflow): + # _comment is not a node + nodes = dict(iter_nodes(sd15_workflow)) + assert "_comment" not in nodes + + def test_iter_model_deps(self, sd15_workflow): + deps = list(iter_model_deps(sd15_workflow)) + names = [d["value"] for d in deps] + assert "v1-5-pruned-emaonly.safetensors" in names + + def test_iter_model_deps_flux(self, flux_workflow): + deps = list(iter_model_deps(flux_workflow)) + names = {d["value"]: d["folder"] for d in deps} + assert names["flux1-dev.safetensors"] == "unet" + assert names["t5xxl_fp16.safetensors"] == "clip" + assert names["clip_l.safetensors"] == "clip" + assert names["ae.safetensors"] == "vae" + + +# ============================================================================= +# Embedding extraction +# ============================================================================= + +class TestEmbeddingRegex: + def test_basic_embedding(self): + m = EMBEDDING_REGEX.search("a cat, embedding:goodvibes, more text") + assert m is not None + assert m.group(1) == "goodvibes" + + def test_embedding_with_strength(self): + m = EMBEDDING_REGEX.search("embedding:bad-hands-5:1.2") + assert m is not None + assert m.group(1) == "bad-hands-5" + + def test_embedding_with_extension(self): + # Strips .pt / .safetensors / .bin + m = EMBEDDING_REGEX.search("embedding:my-emb.pt") + assert m is not None + assert m.group(1) == "my-emb" + + def test_embedding_in_parens(self): + m = EMBEDDING_REGEX.search("(embedding:foo:0.8)") + assert m is not None + assert m.group(1) == "foo" + + def test_multiple_in_one_string(self): + text = "a cat, embedding:foo:1.2, and embedding:bar" + matches = [m.group(1) for m in EMBEDDING_REGEX.finditer(text)] + assert matches == ["foo", "bar"] + + def test_no_false_positive_on_word_embedding(self): + # "embedding " (with space, no colon) should not match + m = EMBEDDING_REGEX.search("the embedding is great") + assert m is None + + +class TestIterEmbeddingRefs: + def test_finds_in_clip_text_encode(self): + wf = { + "1": {"class_type": "CLIPTextEncode", + "inputs": {"text": "embedding:foo, embedding:bar:0.5", "clip": ["2", 0]}}, + "2": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + } + refs = list(iter_embedding_refs(wf)) + names = [name for _, name in refs] + assert names == ["foo", "bar"] + + def test_ignores_non_prompt_fields(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "embedding:foo.safetensors"}}, + } + refs = list(iter_embedding_refs(wf)) + # ckpt_name is not a prompt field — ignored + assert refs == [] + + +# ============================================================================= +# Path safety +# ============================================================================= + +class TestSafePathJoin: + def test_normal_join(self, tmp_path): + p = safe_path_join(tmp_path, "subdir", "file.png") + assert p.is_relative_to(tmp_path) + + def test_blocks_traversal(self, tmp_path): + with pytest.raises(ValueError, match="path traversal"): + safe_path_join(tmp_path, "..", "..", "etc", "passwd") + + def test_blocks_absolute(self, tmp_path): + with pytest.raises(ValueError): + safe_path_join(tmp_path, "/etc/passwd") + + def test_subfolder_with_filename(self, tmp_path): + p = safe_path_join(tmp_path, "outputs", "img.png") + assert p.name == "img.png" + assert p.parent.name == "outputs" + + +# ============================================================================= +# Seed coercion +# ============================================================================= + +class TestCoerceSeed: + def test_explicit_int(self): + assert coerce_seed(42) == 42 + assert coerce_seed(0) == 0 + + def test_minus_one_randomizes(self): + s = coerce_seed(-1) + assert isinstance(s, int) + assert 0 <= s < 2**63 + + def test_none_randomizes(self): + s = coerce_seed(None) + assert isinstance(s, int) + + def test_string_int(self): + # str() that converts cleanly is allowed (relaxed) + assert coerce_seed("12345") == 12345 + + def test_string_minus_one_randomizes(self): + # CLI / JSON sometimes carries seed as a string. + s = coerce_seed("-1") + assert isinstance(s, int) + assert 0 <= s < 2**63 + # And whitespace tolerated + s2 = coerce_seed(" -1 ") + assert isinstance(s2, int) + assert 0 <= s2 < 2**63 + + +# ============================================================================= +# Model list normalization (cloud format) +# ============================================================================= + +class TestParseModelList: + def test_local_format_strings(self): + result = parse_model_list(["a.safetensors", "b.safetensors"]) + assert result == {"a.safetensors", "b.safetensors"} + + def test_cloud_format_dicts(self): + result = parse_model_list([ + {"name": "a.safetensors", "pathIndex": 0}, + {"name": "b.safetensors", "pathIndex": 1}, + ]) + assert result == {"a.safetensors", "b.safetensors"} + + def test_empty(self): + assert parse_model_list([]) == set() + + def test_garbage(self): + assert parse_model_list("not a list") == set() + assert parse_model_list(None) == set() + + def test_mixed_format(self): + result = parse_model_list([ + "string-form.safetensors", + {"name": "dict-form.safetensors"}, + ]) + assert result == {"string-form.safetensors", "dict-form.safetensors"} + + +# ============================================================================= +# Folder aliases +# ============================================================================= + +class TestFolderAliases: + def test_unet_aliases_diffusion_models(self): + aliases = folder_aliases_for("unet") + assert "unet" in aliases + assert "diffusion_models" in aliases + + def test_clip_aliases_text_encoders(self): + aliases = folder_aliases_for("clip") + assert "clip" in aliases + assert "text_encoders" in aliases + + def test_unknown_folder_returns_self(self): + assert folder_aliases_for("checkpoints") == ["checkpoints"] + + def test_primary_first(self): + # Order matters: primary should be first for human-friendly fix hints + assert folder_aliases_for("unet")[0] == "unet" + assert folder_aliases_for("diffusion_models")[0] == "diffusion_models" + + +# ============================================================================= +# Media-type detection +# ============================================================================= + +class TestMediaType: + def test_video_extensions(self): + assert media_type_from_filename("vid.mp4") == "video" + assert media_type_from_filename("foo.webm") == "video" + assert media_type_from_filename("bar.gif") == "video" + + def test_audio_extensions(self): + assert media_type_from_filename("song.wav") == "audio" + assert media_type_from_filename("music.mp3") == "audio" + + def test_image_default(self): + assert media_type_from_filename("pic.png") == "image" + assert media_type_from_filename("image.jpg") == "image" + assert media_type_from_filename("unknown.xyz") == "image" + + def test_3d(self): + assert media_type_from_filename("model.glb") == "3d" + assert media_type_from_filename("scene.gltf") == "3d" + + +# ============================================================================= +# Cross-host header stripping (security) +# ============================================================================= + +class TestRedirectHeaderStripping: + """Verify X-API-Key is dropped when redirect crosses to a different host + (e.g. cloud /api/view → S3 signed URL). Critical to prevent leaking auth + tokens to the storage backend. + """ + + def _build_session(self): + from _common import _StripSensitiveOnRedirectSession, HAS_REQUESTS + if not HAS_REQUESTS: + import pytest + pytest.skip("requests not installed") + return _StripSensitiveOnRedirectSession() + + def test_strips_x_api_key_cross_host(self): + import requests + s = self._build_session() + prep = requests.PreparedRequest() + prep.prepare(method="GET", url="https://other.example.com/file", + headers={"X-API-Key": "leak", "Authorization": "Bearer x"}) + resp = requests.Response() + orig = requests.PreparedRequest() + orig.prepare(method="GET", url="https://cloud.comfy.org/api/view", headers={}) + resp.request = orig + s.rebuild_auth(prep, resp) + assert "X-API-Key" not in prep.headers + assert "Authorization" not in prep.headers + + def test_preserves_x_api_key_same_host(self): + import requests + s = self._build_session() + prep = requests.PreparedRequest() + prep.prepare(method="GET", url="https://cloud.comfy.org/foo", + headers={"X-API-Key": "keep"}) + resp = requests.Response() + orig = requests.PreparedRequest() + orig.prepare(method="GET", url="https://cloud.comfy.org/bar", headers={}) + resp.request = orig + s.rebuild_auth(prep, resp) + assert prep.headers.get("X-API-Key") == "keep" + + def test_strips_cookie_cross_host(self): + import requests + s = self._build_session() + prep = requests.PreparedRequest() + prep.prepare(method="GET", url="https://other.example.com/x", + headers={"Cookie": "session=secret"}) + resp = requests.Response() + orig = requests.PreparedRequest() + orig.prepare(method="GET", url="https://cloud.comfy.org/foo", headers={}) + resp.request = orig + s.rebuild_auth(prep, resp) + assert "Cookie" not in prep.headers + + +# ============================================================================= +# Video workflow detection +# ============================================================================= + +class TestVideoWorkflow: + def test_image_workflow(self, sd15_workflow): + assert looks_like_video_workflow(sd15_workflow) is False + + def test_animatediff_workflow(self, workflows_dir): + import json + wf = json.loads((workflows_dir / "animatediff_video.json").read_text()) + assert looks_like_video_workflow(wf) is True + + def test_wan_workflow(self, video_workflow): + assert looks_like_video_workflow(video_workflow) is True diff --git a/skills/creative/comfyui/tests/test_extract_schema.py b/skills/creative/comfyui/tests/test_extract_schema.py new file mode 100644 index 00000000000..1cb965a1fa8 --- /dev/null +++ b/skills/creative/comfyui/tests/test_extract_schema.py @@ -0,0 +1,185 @@ +"""Tests for extract_schema.py.""" + +from __future__ import annotations + +import pytest + +from extract_schema import ( + extract_schema, + find_negative_prompt_node, + find_positive_prompt_node, + trace_to_node, +) + + +# ============================================================================= +# Connection tracing +# ============================================================================= + +class TestConnectionTracing: + def test_direct_link(self): + wf = { + "1": {"class_type": "CLIPTextEncode", "inputs": {"text": "x"}}, + "2": {"class_type": "KSampler", + "inputs": {"positive": ["1", 0], "negative": ["1", 0]}}, + } + assert trace_to_node(wf, ["1", 0]) == "1" + + def test_through_reroute(self): + wf = { + "1": {"class_type": "CLIPTextEncode", "inputs": {"text": "x"}}, + "2": {"class_type": "Reroute", "inputs": {"input": ["1", 0]}}, + "3": {"class_type": "Reroute", "inputs": {"input": ["2", 0]}}, + } + assert trace_to_node(wf, ["3", 0]) == "1" + + def test_circular_safe(self): + wf = { + "1": {"class_type": "Reroute", "inputs": {"input": ["2", 0]}}, + "2": {"class_type": "Reroute", "inputs": {"input": ["1", 0]}}, + } + # Should hit max_hops without infinite loop + result = trace_to_node(wf, ["1", 0], max_hops=5) + assert result in ("1", "2") # any node, just don't hang + + +class TestPositiveNegativeDetection: + def test_basic(self, sd15_workflow): + # In sd15_workflow.json node 6 is positive, node 7 is negative + assert find_positive_prompt_node(sd15_workflow) == "6" + assert find_negative_prompt_node(sd15_workflow) == "7" + + def test_swapped_order(self): + wf = { + "3": {"class_type": "KSampler", + "inputs": { + "positive": ["7", 0], "negative": ["6", 0], + "model": ["4", 0], "latent_image": ["5", 0], + "seed": 1, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + }}, + "4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", "inputs": {"text": "ugly", "clip": ["4", 1]}}, + "7": {"class_type": "CLIPTextEncode", "inputs": {"text": "beautiful", "clip": ["4", 1]}}, + } + # Now 7 is the positive (despite higher node ID) + assert find_positive_prompt_node(wf) == "7" + assert find_negative_prompt_node(wf) == "6" + + +# ============================================================================= +# Schema extraction +# ============================================================================= + +class TestExtractSchema: + def test_basic_sd15(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + params = schema["parameters"] + assert "prompt" in params + assert "negative_prompt" in params + assert "seed" in params + assert "steps" in params + assert "cfg" in params + assert "width" in params + assert "height" in params + + def test_prompt_value_correct(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + # The positive prompt in the example is the landscape one + assert "landscape" in schema["parameters"]["prompt"]["value"] + assert "ugly" in schema["parameters"]["negative_prompt"]["value"] + + def test_model_dependencies(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + deps = schema["model_dependencies"] + ckpts = [d["value"] for d in deps if d["folder"] == "checkpoints"] + assert "v1-5-pruned-emaonly.safetensors" in ckpts + + def test_output_nodes(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + assert "9" in schema["output_nodes"] + + def test_summary(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + s = schema["summary"] + assert s["has_negative_prompt"] is True + assert s["has_seed"] is True + assert s["is_video_workflow"] is False + assert s["parameter_count"] > 5 + + def test_flux_workflow(self, flux_workflow): + schema = extract_schema(flux_workflow) + # Flux uses RandomNoise for seed + assert schema["summary"]["has_seed"] is True + # Flux has only positive prompt (no negative encoder) + assert schema["summary"]["has_negative_prompt"] is False + + def test_video_detected(self, video_workflow): + schema = extract_schema(video_workflow) + assert schema["summary"]["is_video_workflow"] is True + + +class TestEmbeddingDeps: + def test_extract_from_prompt(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", + "inputs": { + "text": "a cat, embedding:goodvibes, embedding:art:1.2", + "clip": ["1", 1] + }}, + "7": {"class_type": "CLIPTextEncode", + "inputs": { + "text": "ugly, embedding:badhands", + "clip": ["1", 1] + }}, + "3": {"class_type": "KSampler", + "inputs": { + "positive": ["6", 0], "negative": ["7", 0], + "model": ["1", 0], "latent_image": ["5", 0], + "seed": 1, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + }}, + "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}}, + } + schema = extract_schema(wf) + names = [d["embedding_name"] for d in schema["embedding_dependencies"]] + assert sorted(names) == ["art", "badhands", "goodvibes"] + + +class TestDuplicateDeduplication: + def test_two_ksamplers_get_unique_names(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", "inputs": {"text": "a", "clip": ["1", 1]}}, + "7": {"class_type": "CLIPTextEncode", "inputs": {"text": "b", "clip": ["1", 1]}}, + "3": {"class_type": "KSampler", + "inputs": { + "positive": ["6", 0], "negative": ["7", 0], + "model": ["1", 0], "latent_image": ["5", 0], + "seed": 42, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + }}, + "4": {"class_type": "KSampler", + "inputs": { + "positive": ["6", 0], "negative": ["7", 0], + "model": ["1", 0], "latent_image": ["5", 0], + "seed": 99, "steps": 30, "cfg": 8.0, + "sampler_name": "euler", "scheduler": "normal", "denoise": 0.6, + }}, + "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}}, + } + schema = extract_schema(wf) + params = schema["parameters"] + # Both seeds present with disambiguated names + seed_keys = [k for k in params if "seed" in k] + # Symmetric: both renamed (no bare "seed") + assert "seed" not in params + assert "seed_3" in params and "seed_4" in params + assert params["seed_3"]["value"] == 42 + assert params["seed_4"]["value"] == 99 diff --git a/skills/creative/comfyui/tests/test_run_workflow.py b/skills/creative/comfyui/tests/test_run_workflow.py new file mode 100644 index 00000000000..32eb172ad1c --- /dev/null +++ b/skills/creative/comfyui/tests/test_run_workflow.py @@ -0,0 +1,213 @@ +"""Tests for run_workflow.py — focuses on logic that doesn't require a server.""" + +from __future__ import annotations + +import copy +import json + +import pytest + +from extract_schema import extract_schema +from run_workflow import ( + ComfyRunner, + download_outputs, + inject_params, + parse_input_image_arg, +) + + +class TestParseInputImageArg: + def test_with_name(self, tmp_path): + f = tmp_path / "x.png" + f.write_text("x") + n, p = parse_input_image_arg(f"image={f}") + assert n == "image" + assert p == f + + def test_without_name_defaults(self, tmp_path): + f = tmp_path / "x.png" + f.write_text("x") + n, p = parse_input_image_arg(str(f)) + assert n == "image" + + def test_custom_name(self, tmp_path): + f = tmp_path / "x.png" + f.write_text("x") + n, p = parse_input_image_arg(f"mask_image={f}") + assert n == "mask_image" + + +class TestInjectParams: + def test_basic_injection(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + wf, warnings = inject_params(sd15_workflow, schema, { + "prompt": "new prompt", + "seed": 999, + "steps": 25, + }) + assert wf["6"]["inputs"]["text"] == "new prompt" + assert wf["3"]["inputs"]["seed"] == 999 + assert wf["3"]["inputs"]["steps"] == 25 + assert warnings == [] + + def test_unknown_param_warns(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + _, warnings = inject_params(sd15_workflow, schema, {"foobar": "x"}) + assert any("foobar" in w for w in warnings) + + def test_seed_minus_one_randomizes(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + wf, warnings = inject_params(sd15_workflow, schema, {"seed": -1}) + assert wf["3"]["inputs"]["seed"] != -1 + assert isinstance(wf["3"]["inputs"]["seed"], int) + assert any("expanded" in w.lower() for w in warnings) + + def test_randomize_seed_when_unset(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + original = sd15_workflow["3"]["inputs"]["seed"] + wf, warnings = inject_params(sd15_workflow, schema, {}, randomize_seed_if_unset=True) + assert wf["3"]["inputs"]["seed"] != original + assert isinstance(wf["3"]["inputs"]["seed"], int) + + def test_does_not_mutate_original(self, sd15_workflow): + schema = extract_schema(sd15_workflow) + original_text = sd15_workflow["6"]["inputs"]["text"] + inject_params(sd15_workflow, schema, {"prompt": "MUTATED"}) + assert sd15_workflow["6"]["inputs"]["text"] == original_text + + def test_refuses_to_overwrite_link(self): + wf = { + "1": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "x"}}, + "5": {"class_type": "EmptyLatentImage", + "inputs": {"width": 512, "height": 512, "batch_size": 1}}, + "6": {"class_type": "CLIPTextEncode", + "inputs": {"text": ["3", 0], "clip": ["1", 1]}}, # text is a link! + "3": {"class_type": "KSampler", + "inputs": {"seed": 1, "steps": 20, "cfg": 7.5, + "sampler_name": "euler", "scheduler": "normal", "denoise": 1.0, + "model": ["1", 0], "positive": ["6", 0], "negative": ["6", 0], + "latent_image": ["5", 0]}}, + "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "x", "images": ["3", 0]}}, + } + # Manually create a schema that has prompt pointing at 6.text + schema = { + "parameters": { + "prompt": {"node_id": "6", "field": "text", "type": "string", "value": ""}, + } + } + wf2, warnings = inject_params(wf, schema, {"prompt": "literal value"}) + # The link should NOT have been overwritten + assert wf2["6"]["inputs"]["text"] == ["3", 0] + assert any("link" in w.lower() for w in warnings) + + +# ============================================================================= +# Output download walk +# ============================================================================= + +class TestDownloadOutputsWalk: + """Test that download_outputs walks the structure correctly.""" + + def test_handles_videos_plural(self, tmp_path, monkeypatch): + """Local ComfyUI uses 'videos'/'gifs' (plural) keys.""" + downloads = [] + + class FakeRunner: + def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite): + downloads.append((filename, subfolder, file_type)) + p = output_dir / filename + p.parent.mkdir(parents=True, exist_ok=True) + p.write_bytes(b"x") + return p + + outputs = { + "9": {"images": [{"filename": "img1.png", "subfolder": "", "type": "output"}]}, + "10": {"videos": [{"filename": "vid1.mp4", "subfolder": "", "type": "output"}]}, + "11": {"gifs": [{"filename": "anim1.gif", "subfolder": "", "type": "output"}]}, + } + + result = download_outputs(FakeRunner(), outputs, tmp_path) + files = sorted(d["filename"] for d in result) + assert files == ["anim1.gif", "img1.png", "vid1.mp4"] + + def test_handles_video_singular_cloud(self, tmp_path): + """Cloud uses 'video' (singular).""" + class FakeRunner: + def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite): + p = output_dir / filename + p.parent.mkdir(parents=True, exist_ok=True) + p.write_bytes(b"x") + return p + + outputs = { + "10": {"video": [{"filename": "cloud.mp4", "subfolder": "", "type": "output"}]}, + } + result = download_outputs(FakeRunner(), outputs, tmp_path) + assert len(result) == 1 + assert result[0]["filename"] == "cloud.mp4" + + def test_preserves_subfolder(self, tmp_path): + """When preserve_subfolder=True, server subfolder becomes local subdir.""" + class FakeRunner: + def download_output(self, *, filename, subfolder, file_type, output_dir, preserve_subfolder, overwrite): + if preserve_subfolder and subfolder: + p = output_dir / subfolder / filename + else: + p = output_dir / filename + p.parent.mkdir(parents=True, exist_ok=True) + p.write_bytes(b"x") + return p + + outputs = { + "9": {"images": [ + {"filename": "img.png", "subfolder": "myrun", "type": "output"}, + {"filename": "img.png", "subfolder": "otherrun", "type": "output"}, + ]}, + } + result = download_outputs(FakeRunner(), outputs, tmp_path, preserve_subfolder=True) + files = [d["file"] for d in result] + assert any("myrun" in f for f in files) + assert any("otherrun" in f for f in files) + # Both must exist (no collision) + assert len({str(f) for f in files}) == 2 + + +# ============================================================================= +# ComfyRunner construction +# ============================================================================= + +class TestRunnerConstruction: + def test_local_default(self): + r = ComfyRunner() + assert r.is_cloud is False + assert r.host == "http://127.0.0.1:8188" + + def test_cloud_detection(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="abc") + assert r.is_cloud is True + assert "X-API-Key" in r.headers + + def test_cloud_subdomain_detected(self): + r = ComfyRunner(host="https://staging.cloud.comfy.org", api_key="abc") + assert r.is_cloud is True + + def test_partner_key_does_not_pollute_extra_data(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="auth-key") + # No partner-key set → no extra_data should appear in submitted prompt + # (This is a static check; runtime check happens in submit()) + assert r.partner_key is None + + def test_url_routing_local(self): + r = ComfyRunner() + url = r._url("/prompt") + assert url == "http://127.0.0.1:8188/prompt" + + def test_url_routing_cloud(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="x") + url = r._url("/prompt") + assert url == "https://cloud.comfy.org/api/prompt" + + def test_url_routing_cloud_history_renamed(self): + r = ComfyRunner(host="https://cloud.comfy.org", api_key="x") + url = r._url("/history/abc-123") + assert url == "https://cloud.comfy.org/api/history_v2/abc-123" diff --git a/skills/creative/comfyui/workflows/README.md b/skills/creative/comfyui/workflows/README.md new file mode 100644 index 00000000000..f3f40c2f2dc --- /dev/null +++ b/skills/creative/comfyui/workflows/README.md @@ -0,0 +1,86 @@ +# Example Workflows + +These are starter API-format workflows for the most common tasks. They're +ready to run with `scripts/run_workflow.py` once you've installed (or have +cloud access to) the listed models. + +| File | Purpose | Required models | Min VRAM | +|------|---------|-----------------|----------| +| `sd15_txt2img.json` | SD 1.5 text-to-image (512×512) | SD1.5 checkpoint, e.g. `v1-5-pruned-emaonly.safetensors` | 4 GB | +| `sdxl_txt2img.json` | SDXL text-to-image (1024×1024) | `sd_xl_base_1.0.safetensors` | 8 GB | +| `flux_dev_txt2img.json` | Flux Dev text-to-image (1024×1024) | `flux1-dev.safetensors`, `t5xxl_fp16.safetensors`, `clip_l.safetensors`, `ae.safetensors` | 24 GB (or use `flux1-dev-fp8`) | +| `sdxl_img2img.json` | SDXL image-to-image | SDXL checkpoint | 8 GB | +| `sdxl_inpaint.json` | SDXL inpainting (image + mask) | SDXL checkpoint | 8 GB | +| `upscale_4x.json` | Standalone 4× ESRGAN upscale | `4x-UltraSharp.pth` (or any upscaler) | 4 GB | +| `animatediff_video.json` | AnimateDiff text-to-video (16 frames) | SD1.5 checkpoint, `mm_sd_v15_v2.ckpt` motion module | 8 GB | +| `wan_video_t2v.json` | Wan 2.x text-to-video (~33 frames) | `wan2.2_t2v_1.3B_fp16.safetensors`, `umt5_xxl_fp16.safetensors`, `wan_2.1_vae.safetensors` | 24 GB | + +## Quick start + +```bash +# Run a workflow with prompt injection +python3 ../scripts/run_workflow.py \ + --workflow sdxl_txt2img.json \ + --args '{"prompt": "majestic eagle in flight", "seed": 12345, "steps": 35}' \ + --output-dir ./out + +# Img2img: upload an input image first via the script's helper +python3 ../scripts/run_workflow.py \ + --workflow sdxl_img2img.json \ + --input-image image=./photo.png \ + --args '{"prompt": "make it watercolor", "denoise": 0.6}' \ + --output-dir ./out + +# Cloud (set API key once) +export COMFY_CLOUD_API_KEY="comfyui-..." +python3 ../scripts/run_workflow.py \ + --workflow flux_dev_txt2img.json \ + --args '{"prompt": "a fox in a misty forest"}' \ + --host https://cloud.comfy.org \ + --output-dir ./out + +# What can I tweak in this workflow? +python3 ../scripts/extract_schema.py sdxl_txt2img.json --summary-only + +# Are all required models / nodes installed? +python3 ../scripts/check_deps.py wan_video_t2v.json +``` + +## Notes + +- **Inpaint masks**: white pixels = "regenerate this region", black = preserve. + ComfyUI's `LoadImageMask` reads the **red channel** by default; export your + mask as a single-channel image or as a normal RGB where red==intensity. + +- **Denoise strength** in img2img: `0.0` = output identical to input, + `1.0` = ignore input entirely. Sweet spot is usually 0.4–0.7. + +- **Flux Dev** needs ~24 GB VRAM in its base form. The `flux1-dev-fp8.safetensors` + variant (already on Comfy Cloud) cuts that roughly in half. + +- **Video workflows** can take many minutes. The skill auto-detects video + output nodes and bumps the default timeout to 900s. Override with `--timeout 1800`. + +- These JSON files are deliberately **API format** (top-level keys are node IDs + with `class_type`), not editor format. To open them in ComfyUI's web UI for + visual editing, use `Workflow → Load (API Format)` or `Workflow → Open` and + follow the prompt. + +## Cloud vs local model names + +Comfy Cloud's preinstalled checkpoints sometimes have a `-fp16` suffix +(`v1-5-pruned-emaonly-fp16.safetensors`) while the canonical local download +keeps the original name (`v1-5-pruned-emaonly.safetensors`). The example +workflows use the local-canonical names. When running on cloud, override with: + +```bash +python3 ../scripts/run_workflow.py \ + --workflow sd15_txt2img.json \ + --args '{"ckpt_name": "v1-5-pruned-emaonly-fp16.safetensors", "prompt": "..."}' \ + --host https://cloud.comfy.org +``` + +The `ckpt_name`, `vae_name`, `lora_name`, `unet_name`, etc. are all exposed +as controllable parameters by `extract_schema.py` — discover what's installed +with `comfy model list` (local) or `curl /api/experiment/models/checkpoints` +(cloud). diff --git a/skills/creative/comfyui/workflows/animatediff_video.json b/skills/creative/comfyui/workflows/animatediff_video.json new file mode 100644 index 00000000000..cc2b296c3a2 --- /dev/null +++ b/skills/creative/comfyui/workflows/animatediff_video.json @@ -0,0 +1,64 @@ +{ + "_comment": "AnimateDiff text-to-video at 16 frames. Required: comfyui-animatediff-evolved + comfyui-videohelpersuite custom nodes; SD1.5 checkpoint; AnimateDiff motion module (e.g. mm_sd_v15_v2.ckpt in models/animatediff_models/). Outputs a webp animation.", + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, "steps": 25, "cfg": 7.5, + "sampler_name": "dpmpp_sde", "scheduler": "karras", "denoise": 1.0, + "model": ["10", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Checkpoint"}, + "inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"} + }, + "5": { + "class_type": "EmptyLatentImage", + "_meta": {"title": "Latent (16 frames)"}, + "inputs": {"width": 512, "height": 512, "batch_size": 16} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "a hot air balloon drifting over a mountain valley, sunset, cinematic", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "low quality, blurry, deformed, watermark", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "VHS_VideoCombine", + "_meta": {"title": "Video Combine"}, + "inputs": { + "frame_rate": 8.0, + "loop_count": 0, + "filename_prefix": "animatediff", + "format": "video/h264-mp4", + "pingpong": false, + "save_output": true, + "images": ["8", 0] + } + }, + "10": { + "class_type": "ADE_AnimateDiffLoaderWithContext", + "_meta": {"title": "AnimateDiff Loader"}, + "inputs": { + "model": ["4", 0], + "model_name": "mm_sd_v15_v2.ckpt", + "beta_schedule": "sqrt_linear (AnimateDiff)", + "motion_scale": 1.0, + "apply_v2_models_properly": true + } + } +} diff --git a/skills/creative/comfyui/workflows/flux_dev_txt2img.json b/skills/creative/comfyui/workflows/flux_dev_txt2img.json new file mode 100644 index 00000000000..1791280be21 --- /dev/null +++ b/skills/creative/comfyui/workflows/flux_dev_txt2img.json @@ -0,0 +1,78 @@ +{ + "_comment": "Flux Dev text-to-image using the modern sampler chain (BasicScheduler/Guider/SamplerCustomAdvanced). Required: flux1-dev.safetensors (UNET), t5xxl_fp16.safetensors + clip_l.safetensors (CLIP), ae.safetensors (VAE).", + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Prompt"}, + "inputs": {"text": "a serene mountain landscape at golden hour, photorealistic", "clip": ["11", 0]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["13", 0], "vae": ["10", 0]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "flux_dev", "images": ["8", 0]} + }, + "10": { + "class_type": "VAELoader", + "_meta": {"title": "VAE"}, + "inputs": {"vae_name": "ae.safetensors"} + }, + "11": { + "class_type": "DualCLIPLoader", + "_meta": {"title": "DualCLIPLoader"}, + "inputs": { + "clip_name1": "t5xxl_fp16.safetensors", + "clip_name2": "clip_l.safetensors", + "type": "flux" + } + }, + "12": { + "class_type": "UNETLoader", + "_meta": {"title": "UNET Loader"}, + "inputs": {"unet_name": "flux1-dev.safetensors", "weight_dtype": "default"} + }, + "13": { + "class_type": "SamplerCustomAdvanced", + "_meta": {"title": "Sampler Custom"}, + "inputs": { + "noise": ["25", 0], + "guider": ["22", 0], + "sampler": ["16", 0], + "sigmas": ["17", 0], + "latent_image": ["27", 0] + } + }, + "16": { + "class_type": "KSamplerSelect", + "_meta": {"title": "Sampler Select"}, + "inputs": {"sampler_name": "euler"} + }, + "17": { + "class_type": "BasicScheduler", + "_meta": {"title": "Scheduler"}, + "inputs": { + "scheduler": "simple", + "steps": 20, + "denoise": 1.0, + "model": ["12", 0] + } + }, + "22": { + "class_type": "BasicGuider", + "_meta": {"title": "Guider"}, + "inputs": {"model": ["12", 0], "conditioning": ["6", 0]} + }, + "25": { + "class_type": "RandomNoise", + "_meta": {"title": "Noise"}, + "inputs": {"noise_seed": 42} + }, + "27": { + "class_type": "EmptySD3LatentImage", + "_meta": {"title": "Latent"}, + "inputs": {"width": 1024, "height": 1024, "batch_size": 1} + } +} diff --git a/skills/creative/comfyui/workflows/sd15_txt2img.json b/skills/creative/comfyui/workflows/sd15_txt2img.json new file mode 100644 index 00000000000..f67eb79f54c --- /dev/null +++ b/skills/creative/comfyui/workflows/sd15_txt2img.json @@ -0,0 +1,49 @@ +{ + "_comment": "SD 1.5 text-to-image. Smallest model, fastest. Required model: v1-5-pruned-emaonly.safetensors (or any SD1.5 checkpoint)", + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 156680208700286, + "steps": 20, + "cfg": 8.0, + "sampler_name": "euler", + "scheduler": "normal", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Load Checkpoint"}, + "inputs": {"ckpt_name": "v1-5-pruned-emaonly.safetensors"} + }, + "5": { + "class_type": "EmptyLatentImage", + "_meta": {"title": "Empty Latent"}, + "inputs": {"width": 512, "height": 512, "batch_size": 1} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "a beautiful landscape painting, masterpiece, highly detailed", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, low quality, deformed", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "sd15", "images": ["8", 0]} + } +} diff --git a/skills/creative/comfyui/workflows/sdxl_img2img.json b/skills/creative/comfyui/workflows/sdxl_img2img.json new file mode 100644 index 00000000000..a835567aaae --- /dev/null +++ b/skills/creative/comfyui/workflows/sdxl_img2img.json @@ -0,0 +1,54 @@ +{ + "_comment": "SDXL img2img: load an input image, encode to latent, denoise partially. Use --input-image image=./photo.png with run_workflow.py. Lower 'denoise' value preserves more of the source image.", + "1": { + "class_type": "LoadImage", + "_meta": {"title": "Load Source Image"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"} + }, + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, + "steps": 30, + "cfg": 7.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 0.65, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["12", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Load SDXL Base"}, + "inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "make it cyberpunk, neon lights, futuristic", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, low quality, deformed", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "sdxl_img2img", "images": ["8", 0]} + }, + "12": { + "class_type": "VAEEncode", + "_meta": {"title": "VAE Encode"}, + "inputs": {"pixels": ["1", 0], "vae": ["4", 2]} + } +} diff --git a/skills/creative/comfyui/workflows/sdxl_inpaint.json b/skills/creative/comfyui/workflows/sdxl_inpaint.json new file mode 100644 index 00000000000..20e50ccf1b4 --- /dev/null +++ b/skills/creative/comfyui/workflows/sdxl_inpaint.json @@ -0,0 +1,59 @@ +{ + "_comment": "SDXL inpainting: given an image + mask, regenerate the masked region. Upload both: --input-image image=./photo.png --input-image mask_image=./mask.png. White pixels in mask = regenerate; black = preserve.", + "1": { + "class_type": "LoadImage", + "_meta": {"title": "Load Source"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"} + }, + "2": { + "class_type": "LoadImageMask", + "_meta": {"title": "Load Mask"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_MASK.png", "channel": "red"} + }, + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, + "steps": 30, + "cfg": 7.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["12", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Checkpoint"}, + "inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "fill with blooming flowers, photorealistic", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, deformed, bad anatomy", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save"}, + "inputs": {"filename_prefix": "sdxl_inpaint", "images": ["8", 0]} + }, + "12": { + "class_type": "VAEEncodeForInpaint", + "_meta": {"title": "VAE Encode for Inpaint"}, + "inputs": {"pixels": ["1", 0], "mask": ["2", 0], "vae": ["4", 2], "grow_mask_by": 6} + } +} diff --git a/skills/creative/comfyui/workflows/sdxl_txt2img.json b/skills/creative/comfyui/workflows/sdxl_txt2img.json new file mode 100644 index 00000000000..cb590b40f9b --- /dev/null +++ b/skills/creative/comfyui/workflows/sdxl_txt2img.json @@ -0,0 +1,49 @@ +{ + "_comment": "SDXL text-to-image at 1024x1024. Required model: sd_xl_base_1.0.safetensors (or any SDXL checkpoint).", + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, + "steps": 30, + "cfg": 7.5, + "sampler_name": "dpmpp_2m", + "scheduler": "karras", + "denoise": 1.0, + "model": ["4", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["5", 0] + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "_meta": {"title": "Load SDXL Base"}, + "inputs": {"ckpt_name": "sd_xl_base_1.0.safetensors"} + }, + "5": { + "class_type": "EmptyLatentImage", + "_meta": {"title": "Empty Latent"}, + "inputs": {"width": 1024, "height": 1024, "batch_size": 1} + }, + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Positive Prompt"}, + "inputs": {"text": "cinematic photograph, dramatic lighting, intricate detail", "clip": ["4", 1]} + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": {"text": "ugly, blurry, low quality, deformed, watermark", "clip": ["4", 1]} + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["4", 2]} + }, + "9": { + "class_type": "SaveImage", + "_meta": {"title": "Save Image"}, + "inputs": {"filename_prefix": "sdxl", "images": ["8", 0]} + } +} diff --git a/skills/creative/comfyui/workflows/upscale_4x.json b/skills/creative/comfyui/workflows/upscale_4x.json new file mode 100644 index 00000000000..91ad7eb1dee --- /dev/null +++ b/skills/creative/comfyui/workflows/upscale_4x.json @@ -0,0 +1,27 @@ +{ + "_comment": "Standalone 4x upscale of an input image using ESRGAN. Required model: 4x-UltraSharp.pth (or any upscaler in models/upscale_models/). Upload with --input-image image=./photo.png.", + "1": { + "class_type": "LoadImage", + "_meta": {"title": "Load Image"}, + "inputs": {"image": "REPLACE_WITH_UPLOADED_FILENAME.png"} + }, + "2": { + "class_type": "UpscaleModelLoader", + "_meta": {"title": "Load Upscale Model"}, + "inputs": {"model_name": "4x-UltraSharp.pth"} + }, + "3": { + "class_type": "ImageUpscaleWithModel", + "_meta": {"title": "Upscale Image (with Model)"}, + "inputs": { + "upscale_method": "lanczos", + "upscale_model": ["2", 0], + "image": ["1", 0] + } + }, + "4": { + "class_type": "SaveImage", + "_meta": {"title": "Save"}, + "inputs": {"filename_prefix": "upscaled_4x", "images": ["3", 0]} + } +} diff --git a/skills/creative/comfyui/workflows/wan_video_t2v.json b/skills/creative/comfyui/workflows/wan_video_t2v.json new file mode 100644 index 00000000000..7514e3a6279 --- /dev/null +++ b/skills/creative/comfyui/workflows/wan_video_t2v.json @@ -0,0 +1,69 @@ +{ + "_comment": "Wan 2.1 text-to-video. Cloud: confirmed available. Local: download wan2.1_t2v_1.3B_fp16.safetensors → models/diffusion_models/ (or models/unet/), umt5_xxl_fp16.safetensors → models/text_encoders/ (or models/clip/), wan_2.1_vae.safetensors → models/vae/. Output: MP4. Large model — only on cloud or 24 GB+ local GPU.", + "6": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Prompt"}, + "inputs": { + "text": "a graceful crane taking flight from a misty lake at dawn, slow motion, 4k", + "clip": ["38", 0] + } + }, + "7": { + "class_type": "CLIPTextEncode", + "_meta": {"title": "Negative Prompt"}, + "inputs": { + "text": "static, blurry, watermark, low quality", + "clip": ["38", 0] + } + }, + "8": { + "class_type": "VAEDecode", + "_meta": {"title": "VAE Decode"}, + "inputs": {"samples": ["3", 0], "vae": ["39", 0]} + }, + "37": { + "class_type": "UNETLoader", + "_meta": {"title": "Wan UNET"}, + "inputs": {"unet_name": "wan2.1_t2v_1.3B_fp16.safetensors", "weight_dtype": "default"} + }, + "38": { + "class_type": "CLIPLoader", + "_meta": {"title": "Wan CLIP"}, + "inputs": {"clip_name": "umt5_xxl_fp16.safetensors", "type": "wan"} + }, + "39": { + "class_type": "VAELoader", + "_meta": {"title": "Wan VAE"}, + "inputs": {"vae_name": "wan_2.1_vae.safetensors"} + }, + "3": { + "class_type": "KSampler", + "_meta": {"title": "KSampler"}, + "inputs": { + "seed": 42, "steps": 30, "cfg": 6.0, + "sampler_name": "uni_pc", "scheduler": "simple", "denoise": 1.0, + "model": ["37", 0], + "positive": ["6", 0], + "negative": ["7", 0], + "latent_image": ["40", 0] + } + }, + "40": { + "class_type": "EmptyHunyuanLatentVideo", + "_meta": {"title": "Latent Video (33 frames)"}, + "inputs": {"width": 832, "height": 480, "length": 33, "batch_size": 1} + }, + "9": { + "class_type": "VHS_VideoCombine", + "_meta": {"title": "Video Combine"}, + "inputs": { + "frame_rate": 16.0, + "loop_count": 0, + "filename_prefix": "wan_t2v", + "format": "video/h264-mp4", + "pingpong": false, + "save_output": true, + "images": ["8", 0] + } + } +}