CLI Flags and Supported Combinations

This document is the canonical map for VibeSys's CLI flag axes. Update it in the same PR whenever a flag, backend, domain, loop, runtime environment, or profiler behavior changes.

Start with Running VibeSys for the project layout and the choice between editing the input project or provisioning a copy. This document is the exhaustive flag reference.

Entry Point

Use the installed vibesys command. It launches the interactive TUI by default and runs headless with --headless, --help, validate, or when not attached to a TTY. Examples below use this command.

Agent configuration

--config PATH selects an explicit agent.toml. When the flag is omitted, VibeSys loads agent.toml from the process launch working directory if it exists, otherwise it uses built-in CLI defaults. It does not search parent directories.

An orchestration plugin declares its agent role IDs. Global model and thinking settings apply to every declared role; sparse role entries override only the named role:

[model]
name = "gpt-5.6-sol"

[thinking]
level = "high"

[agent.roles.orchestrator]
model = "gpt-5.6-sol"
reasoning_effort = "xhigh"

Every model must be a vetted model of the CLI provider: VibeSys runs only the provider and model pairs vs_agent.provider_policy.harnesses() returns: the models agentshim lists for each shipped provider (ProviderProfile.models). Without [model].name or --model, a run uses its provider's default model (the first one agentshim lists). An unvetted model in [model].name, --model, a role entry, or [agent.quota].fallback_model is rejected before the run builds anything, with the key and the vetted models. At a terminal, --allow-unvetted-model runs it anyway; runs started from the desktop app cannot opt out. A resume keeps the pair its run was started with.

Unknown role IDs are rejected against the selected orchestration before run resources open. The run manifest records the total resolved role map. Resume restores that map and rejects a plugin generation whose role IDs differ.

Rounds, workstreams, and attempts

A workstream is one independently scheduled unit of dynamic agent work. It may implement or continue a hypothesis, profile a revision, or merge candidate revisions. --max-in-flight M limits how many workstreams hold agent slots at once. --max-rounds N supplies N admissions per slot, so the dynamic search may admit at most N * M workstreams whose workspaces open. A workstream that fails parent verification or workspace admission is refunded and does not consume that budget; an operator stop before the workspace opens is not refunded. A workstream whose candidate waits for its trusted measurement parks and gives its agent slot to new work, so planning and implementing go on while measurements queue. --max-backlog K caps how many workstreams park at once (default: every waiting one, up to 32); --max-backlog 0 keeps each agent slot held until its workstream ends.

A dynamic round is the durable result of an implementation workstream recorded under a hypothesis. Profile and merge workstreams still appear in the iteration tree and consume an admission when their workspace opens, but they do not produce round records. An implementation workstream that never reaches a round record likewise produces no round event.

An attempt is the core-owned admission identity beneath a workstream and owns its workspace when one opens; it is not an implementer retry. Dynamic implementer retries are agent turns, bounded by --max-unmeasured-turns; --max-retries-per-round applies only to the serial agent policies. The strategy-neutral round_finished.data.attempts field currently reports 1 for a projected dynamic round and must not be read as its implementer-turn count.

Each recorded dynamic round produces one round_finished and one workstream_finished event. Both carry the same non-null envelope round_label (round-N), so consumers correlate the pair by (run_id, round_label). The count of either event is bounded by the N * M admission budget, not by --max-rounds alone; profile, merge, and pre-open workstreams are not included in that count.

Evaluation suspension bounds

Set the queue allowance in agent.toml:

[evaluation]
queue_allowance_seconds = 900
observe_interval_seconds = 10
observe_backoff_cap_seconds = 120
queue_wait_budget_seconds = 43200
queue_notice_interval_seconds = 600

observe_interval_seconds (default 10) is how often the core polls a running evaluation job. observe_backoff_cap_seconds (default 120, at least the interval) caps the poll delay after the job's state could not be read.

Time a measurement job spends waiting in the executor's queue (Slurm PENDING) does not count against the measurement's limit: each job is credited the queue time its polls observe, at most observe_backoff_cap_seconds per poll interval, so a scheduler that stops answering is not credited. queue_wait_budget_seconds (default 43200, 12 hours) bounds that queue time: a job still queued after it ends the measurement as queue-budget-spent. Every queue_notice_interval_seconds (default 600) of queue time the run reports the job as still queued, with the executor's pending reason when it gave one.

The value must be a positive integer and defaults to 900 seconds. A suspended continuation's absolute deadline is its evaluation submit time plus this allowance and the sum of declared timeouts for the stages that run, including framework setup when requested. Requested stages without a declared timeout cannot produce a suspension deadline. Queue estimates inform planning; they do not extend the deadline.

Backend retries

The backend, not the search, decides how often it tries again when a failure is not the candidate's. Declare its bounds in agent.toml:

[evaluation]
infrastructure_attempts = 3    # tries in all after a node fault or lost job
ambiguous_attempts = 2         # tries in all when the evidence cannot say whose fault it was
request_retry_limit = 3        # re-sends of a request the executor did not run
request_backoff_seconds = 1.0
request_backoff_cap_seconds = 30.0

The search sees one result per measurement, the last try's. A failure that is the candidate's own is a result and is never tried again. Each attempts value is at most 5 and request_retry_limit at most 10: caps in the core that no setting can lift. [run] also holds cancellation_bound_seconds, cleanup_bound_seconds and reconciliation_bound_seconds (60 each), the time core gives its own cleanup waits.

Provider quota and rate limits

When a provider runs out of quota, or stays rate limited after the agent library's own waiting, the run pauses (the same PAUSING/PAUSED states as an operator pause) and publishes a quota_paused event with the provider's diagnostic. Resuming sends the stopped turn again. To handle this without an operator, declare a policy in agent.toml:

[agent.quota]
policy = "wait"              # pause | wait | fail
wait_seconds = 14400         # with "wait"
retry_seconds = 300          # used when the provider gave no reset time

pause (the default) waits for the operator. wait resumes the run by itself once the provider's reported reset time (plus a few seconds) has passed, or every retry_seconds when it reported none, and fails the turn with the quota error once wait_seconds is spent or the reset lies beyond it. fail does not pause.

A run drives every role through one agent client on one provider, so it cannot switch to a fallback provider on a quota stop. fallback_provider, fallback_model and policy = "fallback" are accepted by the config schema but refused when a run starts, naming the key; use pause, wait or fail. A misspelled key, an unknown policy, or wait_seconds with pause or fail is rejected at startup, naming the key.

Run time budget

Optionally bound the wall-clock time of one run in agent.toml:

[run]
max_run_seconds = 86400

The value must be a positive integer. It is omitted by default, and the run is then unbounded in time (the round budget still applies). The core-driven dynamic strategy reads it as the run deadline.

Mental Model

Several flags look independent, but they combine into one execution contract:

Axis	Flag	Meaning
Search loop	--outer-loop	Which outer-loop policy runs: agent, profile-guided, plain, or evolve.
Evaluation interface	--interface	Agent loop only. Whether evaluator-owned code invokes the candidate directly or communicates with a service.
Compute backend	backends in vibesys.input.toml	Hardware/runtime target the task declares: cuda, metal, trainium, rocm, or cpu. The task decides; --backend only overrides it with another backend the task declares (see Compute backend).
Agent environment	none (always Docker)	Where the agent runs: a local Docker container, for every run. --docker was removed and is rejected.
Evaluation environment	--run-environment, --modal, --skypilot, --slurm-config	Where evaluation runs: the local container (default), Modal, SkyPilot, or Slurm. These never move the agent out of its local container.
Profiler	--profiler	Bottleneck evidence source: nsys, rocprof, torch, neuron, otel, macos_cpu, linux_cpu, or auto.
Domain	[agent].domain in vibesys.input.toml	Problem-space package used by the agent and evolve loops, such as llm-serving, microservices, or generic.
Modality	--modality	Per-task I/O contract, such as text_generation or speech_to_text.
Skills	--skills-dir, --extra-skills, --no-skills	Override the preset skill roots, stack extra skills on top of the presets, or disable skill loading.
Target project	--project (--input alias)	Candidate repository. Defaults to the current directory.
Task	--task NAME	Task below .vibesys/tasks; optional when exactly one exists.
Project provisioning	--runs-dir PATH	Copy a repository-shaped example nested below another Git root, or a project synthesized from --input-* flags. Standalone repository-native projects run in place.
Project repository	--repo, --repo-visibility, --local, --resume	Fresh copied projects use GitHub by default. Resume also accepts local project paths or GitHub repositories.
Client theme	--theme	Presentation only. Which semantic theme the interactive client renders with.

Do not treat these as simple toggles. Some combinations imply a startup contract, profiler, or sandbox capability. Language and artifact requirements come from the domain and input bundle, not the interface mode.

Outer Loops

Value	Behavior	Notes
agent	Orchestrator-driven loop with implementer, judge, and profiler roles.	Default. Supports --interface and --inner-loop.
profile-guided	Agent lifecycle with framework-owned component attribution and focus selection.	Requires [profile_guided] in the input manifest. Supports the agent flags.
plain	Issue-tracker loop with deterministic issue draining and perf evaluation.	`--tracker-backend local
evolve	Evolutionary search over candidate implementations.	Uses domain-aware mutator, judge, and profiler roles.

Run the commands below with vibesys. Use vibesys --outer-loop <kind> --help for loop-specific flags.

Canonical Projects

The candidate repository is the working project. --project defaults to the current directory:

cd /path/to/project
vibesys --task latency

# Equivalent without changing directory
vibesys --project /path/to/project --task latency

The project must contain .vibesys/tasks/<name>/OBJECTIVE.md, .vibesys/tasks/<name>/vibesys.input.toml, and the candidate source the agent will edit. A task name is required when the repository defines multiple tasks. It must be the root of its Git repository, or outside any Git repository so VibeSys can initialize one. A subdirectory of a containing Git repository is not a valid project root. An existing repository must have a baseline commit and a clean worktree. A directory outside Git is initialized with a baseline commit automatically.

All outer loops, runtime environments, profilers, and agent backends use the same project layout. A repository-shaped example nested below another Git root may use --runs-dir to materialize an isolated project. Standalone --input-* synthesis also uses this path.

The native sandbox requires bwrap on Linux or sandbox-exec on macOS. A missing confinement tool or an unsupported operating system stops the run rather than launching the agent without isolation. On Linux bwrap must also be able to create a user namespace; a present but blocked binary is treated as missing and reported at startup rather than failing once per round. Evaluator, checker, and benchmark paths are visible to agents but enforced read-only and protected by integrity checks. The agent also cannot write .git or .vibesys, and cannot read root .env* files or root agent.toml. Machine-local VibeSys state is stored outside the project. A run is rejected when a root .env* file or agent.toml is recoverable from Git refs or reflogs.

A task whose candidate is a container topology (docker compose, kind) declares [environment] docker_in_docker = true. The key selects the local Docker evaluation environment; --modal, --skypilot, Slurm, and --docker-image conflict with it and the error names the key. The sandbox runs under Sysbox with a Docker daemon of its own and never mounts the host socket; a host without sysbox-runc fails early. See Docker-in-Docker.

The host-sandbox variables below apply to the host environment (Metal on macOS) and to the confinement of agent GPU jobs on compute nodes. VIBESYS_AGENT_SANDBOX selects the Linux mechanism. auto (the default) and bwrap both require bubblewrap. landlock opts in to a weaker backend for hosts that block unprivileged user namespaces, which is the common reason bubblewrap cannot run without root (for example Ubuntu's kernel.apparmor_restrict_unprivileged_userns). The Landlock backend keeps the outer boundary — the project is writable and the rest of the host is denied for both read and write — but Landlock rules only ever add access, so it cannot carve a restriction out of the writable project. Read-only and hidden project paths are therefore not enforced under it, and each unenforced tier is logged at startup. Evaluator-input integrity still holds, because the accuracy gate independently diffs those paths against a trusted baseline and fails the round. Landlock is never selected automatically, and a project that sits inside a tree the backend must grant (such as /tmp) is refused rather than run unconfined. 0/false/off/no request no confinement, but VibeSys runs on the host require it, so they stop the run with SandboxUnavailableError instead of launching the agent unconfined. Only an embedder that builds an AgentClient with require_host_sandbox=False gets the disable. Any other value is rejected. VIBESYS_AGENT_SANDBOX_ALLOW is an os.pathsep-separated list of extra host paths granted to the agent read-only. See Sandboxing and confinement.

VibeSys initializes Git when needed and creates one vibesys-runs/<run-id> branch per run, cut from the branch checked out when the run starts (its home branch). A run that adopts a winner commits it on its own branch, with a message naming the candidate and its measurements, and never writes to the home branch. The run's branch stays checked out after it ends; the next run checks the home branch out before cutting its own, and a resume checks out the run's branch. Every run stays listed and resumable from any branch, because run history is read from the run branches. Agent-authored source stays at its normal project paths. Portable state remains in the project, while machine-local state defaults to ~/.vibesys:

.vibesys/
└── state/
    ├── .gitignore                         # contains /local/
    ├── project.json                       # committed project identity
    ├── runs/<run-id>/
    │   ├── run.json                       # committed sanitized run configuration
    │   ├── agent/rounds/NNNN.json         # agent-loop completed rounds
    │   ├── plain/                          # plain-loop portable cursor
    │   ├── evolve/                         # evolve population and policy state
    │   └── runtime/effective-objective.md  # objective plus run constraints
    ├── agent-visible/<run-id>/            # documents agents read by path, e.g. planner handoff notes
    └── local/runs/<run-id>/worktrees/     # ignored temporary evolve worktrees

~/.vibesys/projects/<project-key>/
├── current-run
└── runs/<run-id>/
    ├── agent/active.json
    ├── round-transaction.json             # during round commit/recovery
    └── logs/
~/.vibesys/cache/<architecture>/
└── evaluator-tools/<tool>/<spec-digest>/  # evaluator tools, shared by all projects

Set VIBESYS_STATE_HOME to an absolute directory to override ~/.vibesys. ~/.vibesys/cache/ holds immutable, content-addressed entries that every project on the host reuses, such as evaluator tools built from a pinned source revision. Entries are keyed by CPU architecture, so a state home on a filesystem shared by hosts of different architectures stays correct. Deleting the directory is safe: VibeSys rebuilds entries on the next run that needs them. VibeSys moves an existing .vibesys/state/local/ tree on first open, preserving the existing bytes and paths; temporary worktrees remain in the project.

The committed files contain portable configuration, fingerprints, metrics, and round outcomes. They exclude provider credentials, environment variables, absolute source paths, sessions, and raw logs. Resume restores the saved branch and run configuration:

# Resume the machine-local current run, or newest if unset
vibesys --resume

# Select a run explicitly
vibesys --resume <run-id>

A plain vibesys launch starts a new run. It does not silently resume the current or latest run. When run flags and --config are omitted during resume, their recorded values are restored. Explicit configuration changes are rejected, except that --max-rounds may increase the recorded total. The project worktree must be clean before VibeSys switches to the saved vibesys-runs/<run-id> branch.

Agent-loop review and memory policy

Flag	Default	Behavior
--judge-every N	3	Run an independent judge every Nth round. A candidate explicitly nominated by the implementer and the final round are always reviewed immediately. Canonical accuracy and benchmark commands run only after a judge PASS. The dynamic orchestration uses an implementation workstream's global schedule sequence for this cadence: it always reviews a nominated or supported candidate, and reviews another terminal outcome when that sequence is every Nth workstream. Its admission budget is --max-rounds times --max-in-flight, and a continued hypothesis uses another admission.
--official-eval-every N	3	Run configured framework-owned accuracy and benchmark gates every N accepted candidate checkpoints. Intermediate checkpoints remain provisional; orchestrator requests and the final round force immediate official evaluation. Retries, continuing hypotheses, and profiler-only rounds do not advance this cadence. Modal gates reuse one healthy deployment for the exact candidate commit, explicitly stop it after the final gate, and rely on zero minimum-warm replicas plus a short finite scaledown window as the crash backstop. Unchanged retries reuse a prior accuracy PASS when only a later gate failed. The dynamic orchestration ignores this cadence: its workstreams are parallel branches, so it evaluates every review-passed candidate.
--constraint TEXT	none	Add an operator-supplied workload invariant to every agent's objective without changing the input bundle. The framework commits the effective objective under the run's portable .vibesys/state/ and mounts it read-only in isolated environments, so candidate edits cannot erase it. Repeat for multiple constraints and repeat the same flags when resuming.

The designer's roadmap (its directions, their blockers and their statuses) is kept by VibeSys in the run's state, not in the project: the designer sees it in its prompt and changes it through its reply, and a resumed run keeps it.

Designer and judge invocations start with clean model sessions. The implementer session is keyed by the designer's stable hypothesis_id, so targeted experiments and debugging context persist while the causal claim is unchanged. The designer is not called again while that hypothesis is active. A round reported as continue is provisional when review is not due: neither the judge nor framework-owned official gates run that round. These independent-judge cadence rules apply to the default multi-agent inner loop; single-agent is retained as an ablation mode.

Evolve search policies

--search-policy vibesys (the default) uses VibeSys's scalar softmax or Pareto frontier selection. --search-policy openevolve imports pinned OpenEvolve 0.3.1 and delegates MAP-Elites archiving, island selection, population limits, and migration to its ProgramDatabase. It does not use OpenEvolve's one-shot LLM mutation path; VibeSys's coding agent continues to mutate the checked-out multi-file project.

Flag	Default	Meaning under --search-policy openevolve
--openevolve-population-size	1000	Maximum upstream program population.
--openevolve-archive-size	100	Elite archive size.
--openevolve-num-islands	5	Number of island populations sampled round-robin.
--openevolve-migration-interval	50	Per-island admitted generations between migrations.
--openevolve-migration-rate	0.1	Fraction of island elites copied during migration.

OpenEvolve state is part of the run's committed strategy state (the database's programs, with each passing child's patch text, so that state grows with the patches the database keeps) and is loaded on resume. See docs/contributing/openevolve.md for the adapter boundary and metric semantics. On resume the policy and saved settings are restored when these flags are omitted; partial explicit settings are merged with the saved values, flag-defined objectives are restored, and incompatible changes are rejected. On a new run, supplying any --openevolve-* setting also selects the OpenEvolve policy.

Copied Project Collections and Remote Repositories

Configure interactive defaults in agent.toml:

[repository]
# Optional GitHub user/org override. If omitted, use the account authenticated with `gh`.
# owner = "your-github-user"
visibility = "private"

Runs created with --runs-dir use GitHub by default. The owner can be any GitHub user or organization; when owner is omitted, VibeSys uses the account authenticated with gh. Interactive and headless launchers pass the same arguments directly to the engine, so provide the input and collection explicitly. Names default to <input-name>-<UTC timestamp>.

For headless use, the generated repository name is used automatically. Pass --repo NAME to override the name, or --repo OWNER/NAME to override the owner explicitly. Pass --local to keep a copied project in its collection without a GitHub repository. A direct project stays local unless --repo is explicit. Repositories use [repository].visibility unless --repo-visibility overrides it. Creation goes through the authenticated gh CLI.

The project repository records candidate history and portable .vibesys/state/runs/ state. Provider and agent logs under the configured VibeSys state home are excluded. On context shutdown, VibeSys pushes the already-authored run branch and retained evolve candidate refs. Publication never stages files or creates a synchronization commit. A non-fast-forward or authentication failure is reported rather than force-pushing.

With --runs-dir, --resume accepts collection project names, local project directories, GitHub OWNER/NAME pairs, and cloneable HTTPS/SSH URLs:

vibesys --runs-dir /work/vibesys-runs --resume 20260720-120000-example
vibesys --runs-dir /work/vibesys-runs --resume /work/experiments/example
vibesys --runs-dir /work/vibesys-runs --resume vibesys-playground/example
vibesys --runs-dir /work/vibesys-runs --resume https://github.com/my-org/example.git

Remote repositories are cloned into the selected collection. A local clone can live anywhere, but --runs-dir is still required for shared caches and any future runs. Resumed repositories with an origin are synchronized again after the run. --repo only creates a repository for a fresh experiment and cannot be combined with --resume.

Listing Tasks

Run vibesys tasks [PROJECT] [--json] to list the tasks a project defines under .vibesys/tasks/, one name per line, ordered by name. The project defaults to the current directory. Each task is validated the way a run selects it, so a task directory missing its OBJECTIVE.md or vibesys.input.toml fails the listing. --json prints one TaskList document (defined in src/entrypoints/tasks.py):

{"version": 1, "project_root": "/home/me/src/queue-rs", "tasks": [{"name": "mpmc"}, {"name": "spsc"}]}

A missing project, a project without .vibesys/tasks/, or an invalid task exits with status 1, prints the reason on stderr, and prints nothing on stdout.

Examples

vibesys example list [--json] lists the example catalog of a VibeSys source checkout (examples/registry.toml) by category, one example per line with its tasks. --json prints one ExampleList document (defined in src/entrypoints/example_catalog.py):

{"version": 1, "checkout": "/home/me/src/vibesys", "examples": [{"name": "data-structures/stack", "category": "data-structures", "tasks": ["mpmc", "mpsc", "spmc", "spsc"], "external_repo": false}]}

vibesys example new EXAMPLE DEST copies the example's tracked files into DEST, runs git init, commits them, and prints the vibesys --task ... line to run next. An external_repo example is cloned at the commit the checkout pins. DEST must be absent or empty and outside the checkout; otherwise, and for an unknown or ambiguous example, the command exits 1 with the reason on stderr and leaves DEST as it was. --checkout PATH selects another VibeSys checkout's catalog.

Repository Validation

Run vibesys validate [PROJECT] --task NAME to check a task's static harness contract without starting the interactive client, an optimization loop, or an agent.

The project is the positional argument. When omitted, it defaults to the current directory. --task may be omitted only when one task exists:

vibesys validate

Pass another project directly:

vibesys validate examples/model-serving/repositories/vllm \
  --task llama-3-8b-h100-long-prompts

The command applies the same strict schemas and path checks as a real run. It validates the task objective and manifest, accuracy and benchmark command paths or package entry points, and the optional benchmark-result contract. A valid task exits with status 0; an invalid task prints the failing contract and exits with status 1. Command-line usage errors exit with status 2. Validation does not execute the checker or benchmark. A project with a root vibesys.input.toml fails validation, naming the file and its place under .vibesys/tasks/<task>/.

Interface

--interface applies to the agent loop.

Value	Process boundary	Contract ownership
inprocess	Evaluator-owned code invokes the candidate directly inside an evaluator process.	The input defines the callable API or ABI, artifacts, ownership, and lifecycle.
service	Checker and benchmark communicate with a running candidate over its network interface.	The input defines the protocol, endpoints, startup behavior, and artifacts.

service does not automatically rewrite a checker or benchmark. The target inputs must already know how to probe the running service.

inprocess does not imply Python. A Python module imported by an accuracy checker and a C-ABI shared library loaded by a trusted adapter are both in-process candidates. Their exact requirements belong to domain/use-case prompts and input-owned candidate-contract documentation.

Compute Backends

Backend	Intended target	Sandbox support	Device handling	Default profiler behavior
cuda	NVIDIA GPU serving systems and kernel-writing tasks.	Docker, Modal.	Selects/reselects a GPU and can monitor contention.	Serving uses nsys in Docker and torch on Modal; kernel-writing uses ncu where supported by the run environment.
rocm	AMD GPU serving systems.	Docker, SkyPilot, Slurm.	Selects a visible ROCm device locally; a SkyPilot profile declares remote capacity.	Docker uses rocprof; SkyPilot uses none (only auto/none are supported there).
metal	Apple Silicon / MPS targets.	None: Docker on macOS cannot expose Metal/MPS, and agents no longer run on the host. A Metal run fails when the container is created, before agent work starts.	No device selection or monitor.	n/a
trainium	AWS Trainium / NeuronCore targets.	Docker; Modal unsupported.	Forwards /dev/neuron* in Docker; no per-device selection.	auto resolves to neuron.
cpu	CPU-only service/data-structure targets.	Docker.	No device selection or monitor.	Generic workloads on Linux select linux_cpu; macOS selects macos_cpu; other systems select no profiler.

When a backend rejects a runtime environment, it should fail before agent work starts with an actionable error.

Runtime Environment

Agents always run in a local Docker container. The flags below only choose where evaluation runs.

Flags	Evaluation runs	Notes
none, or --run-environment docker	In the local agent container.	Re-mounts read-only project paths read-only and overlays hidden paths with empty masks. Backend controls GPU/device passthrough.
--modal	On Modal: GPU-bound work dispatches through the candidate's own modal run from a local CPU Docker editor.	Mutually exclusive with --skypilot. Same overlays as the default.
--run-environment skypilot	On a SkyPilot cluster, from a local CPU Docker editor.	Requires portable task resources and an operator-owned cluster profile. Same overlays as the default. See Remote Slurm execution.
--run-environment slurm, or --slurm-config	On a Slurm cluster.	The editor runs in a local Docker container; trusted gates run on the host through a broker the container reaches over a socket. With [vibesys.agent_gpu] in the operator file (local transport only) the agent also runs its own GPU commands as Slurm jobs through vibesys-gpu; see GPU commands through Slurm. See Remote Slurm execution and Sandboxing.
--run-environment slurm-gpu (deprecated)	Alias for --run-environment slurm with local transport and [vibesys.agent_gpu], built by translating the old [slurm_gpu] file.	Logs one deprecation warning. --slurm-config names the old file. See the alias.

--docker and --run-environment local were removed. Both are rejected with an error instead of being ignored, because neither selects anything any more. A run recorded with the local environment before their removal continues in Docker when resumed: the run log says so and the run record is rewritten to docker.

A repository-native task may provide .vibesys/tasks/<task>/Dockerfile. Every environment whose agent runs in a local container (Docker, and the agent container of --modal, --skypilot and --run-environment slurm) builds it with the task directory as its complete context and the backend's default image as BASE_IMAGE, and runs the agent in the resulting image's ID. The run records a tag keyed to the task directory and a digest of every build input (honoring .dockerignore). No manifest field or pre-build step is required. --docker-image conflicts with this convention, and the host environment refuses it. Resume reuses the recorded image while it is in the local daemon (with a notice if the inputs changed), rebuilds it when it is gone and the inputs are unchanged, and refuses changed inputs once the image is gone, or a Dockerfile the record has no image for. docker_in_docker = true (below) still selects Docker only. A root Dockerfile is a candidate file and is not auto-detected. See Running VibeSys for why a task image's ENV leaks into Slurm GPU jobs.

--docker-image overrides the backend's default agent container image, including the local CPU editor image for Modal and SkyPilot.

The selected environment and its options (--docker-image, --modal-gpu, --modal-model-volume, --modal-app) are recorded in the run configuration. --modal is a boolean flag, so an omitted flag cannot be told apart from an explicit "off": on resume the recorded environment is therefore authoritative. Omitting the runtime-environment flags restores it, and passing a flag that contradicts the recording is rejected like any other immutable configuration field. The candidate's Modal entrypoint is declared by the task, not recorded, so it follows the current input bundle.

SkyPilot profile selection, profiles-file path, and executable are machine-local launch policy. Supply them again on resume when they are not available from the current CLI/config file.

Only v5 descriptor-backed runs can be resumed. Earlier run schemas are not supported by this CLI.

Profiler

Value	Intended use
auto	Let the runtime/backend pick the default profiler.
nsys	NVIDIA Nsight Systems. Requires a CUDA/NVIDIA profiling environment.
ncu	NVIDIA Nsight Compute for CUDA kernel-writing tasks. Selected by auto when the run environment supports it; use none to disable it.
rocprof	AMD rocprofv3 / rocprof-compute toolkit. Requires a ROCm profiling environment. Selected by auto for ROCm LLM-serving and kernel-writing tasks when the run environment supports it.
torch	PyTorch profiler. Used for in-process Python profiling and Modal GPU dispatch.
neuron	AWS Neuron profiler for Trainium.
otel	OpenTelemetry service, span, datastore, and critical-path latency for microservice benchmarks. Needs an input bundle that provisions instrumentation and a collector; auto selects it when the bundle's benchmark command declares both --telemetry-output and --trace-graph-json, and resolves to none otherwise.
macos_cpu	Instruments Time Profiler with a supported /usr/bin/sample fallback.
linux_cpu	Linux perf profiler for native and mixed-language CPU workloads.

When auto selects a host-native profiler that fails its local capability preflight, VibeSys logs the diagnostic and continues with none. An explicit macos_cpu or linux_cpu request remains a configuration error when its preflight fails.

--modal --profiler nsys is rejected by the CLI because Modal runs must use the torch profiler path.

Profiler prompts must match the interface, domain, and backend. In-process execution alone does not make the candidate Python or PyTorch-compatible; the selected domain must explicitly support Torch profiling. A CPU backend must not receive a GPU-kernel workflow.

The macOS backend verifies that the selected developer directory is full Xcode and asks xctrace for the Time Profiler template; the Command Line Tools shim is not considered functional Instruments. Captures are separate diagnostic runs, never scored results. They store exact commands, duration, warm-up, OS/CPU/tool data, target PID/topology, diagnostics, and the raw .trace or sample report. Attach failures, including SIP or privacy restrictions, are structured diagnostics. Optimized native builds should retain debug information; dsymutil, dwarfdump, nm, and atos can validate or resolve symbols. Reports must state when unavailable Apple hardware counters limit conclusions.

Domain and Modality

The profile-guided loop reads a task-owned attribution command from the input manifest. The command receives --vs-output PATH and must write protocol v1:

[profile_guided]
command = ["python", "profiler/attribute.py"]
timeout_seconds = 1800
result_protocol = 1
min_measured_rounds = 2
min_relative_improvement = 0.02

The result is JSON with version = 1, a cost_unit, and ranked components. Each component has name, nonnegative cost, share between zero and one, and an optional evidence string list. This configuration is task capability, not a modality. Other loops ignore it.

The command runs in the candidate workspace through the run's capture-command environment. Under --run-environment slurm with local transport and [vibesys.agent_gpu], that means a confined Slurm GPU job with the same partition, job-name prefix, concurrency cap, read_only_paths, host environment policy, and run-scoped launcher as vibesys-gpu. Its seconds timeout is rounded up for Slurm's minute allocation while VibeSys still cancels at the exact requested second. Without [vibesys.agent_gpu], and in every other run environment, capture keeps using the editor command runner. Local validation commands always use the editor runner; enabling agent GPU jobs does not move validation to Slurm.

Make the command and its dependencies visible in the environment selected above. When it fails, the run stops at the first capture and the error names the command, where it ran, and the bounded diagnostic tail (exit code 127 means a program it calls was not found). Before submission, VibeSys durably records the capture's stable Slurm job name. Cancellation drains the capture before its temporary output and candidate are removed. If Slurm cannot confirm the named job is gone, VibeSys reports an unresolved stop and retains those resources rather than deleting files a live job may still use. A resumed host cancels and queries the recorded name; it removes the candidate only after the scheduler confirms that job is absent.

[agent].domain in vibesys.input.toml supplies cross-cutting problem-space context for the agent and evolve loops. Registered domains include:

Domain	Meaning
llm-serving	LLM-serving guidance, including serving-system skills and judge gates.
microservices	Microservice workload guidance, lifecycle rules, and service-level evaluation context.
generic	No extra domain guidance. Useful for custom/non-LLM targets.

Each input bundle must declare [agent].domain; there is no CLI override for a bundle passed with --input. When synthesizing a bundle from --input-* flags instead, --input-domain sets [agent].domain for the generated manifest. New domains are added in source by registering a domain package with optional environment setup/teardown hooks.

--modality selects one of the modalities the run's domain declares (for llm-serving: text generation, the default, image or video generation, text-to-speech, speech-to-text and realtime audio); its text lives in that domain's pack. Other domains declare none and reject the flag; an example's own wire contract belongs in its OBJECTIVE.md. Domains and modalities may define language, toolchain, and artifact requirements. Interface-specific prose should describe only the direct-call or service boundary.

Client Theme

--theme selects the semantic theme the interactive client renders with. It is presentation only: it never reaches the agents, the project, or the recorded run state, and it is ignored in headless mode.

Theme	Appearance	Use for
dark	dark	Default dark palette.
light	light	The baseline palette inverted for light terminals.
solarized-dark	dark	Low-glare Solarized palette.
solarized-light	light	Low-glare Solarized palette.
catppuccin-mocha	dark	Softer, more expressive palette.
catppuccin-latte	light	Softer, more expressive palette.
high-contrast-dark	dark	Accessibility-focused; every foreground clears a 7:1 contrast ratio.
high-contrast-light	light	Accessibility-focused; every foreground clears a 7:1 contrast ratio.

Resolution order, highest first:

1. --theme <name> on the command line.
2. [tui].theme in agent.toml.
3. dark.

An unknown name is rejected before any process starts. Inside a running session, /theme opens the theme list as a keyboard selection (Up/Down to move, Enter to apply, Escape to close without switching) and /theme <name> switches immediately; that switch applies to the session only and does not edit agent.toml.

Themes define semantic roles — surfaces, text emphasis levels, borders, accents, status colors, conversation roles, and Markdown/code colors — rather than per-component colors, and every derived foreground is checked against its own background at build time. Status is never carried by color alone: agent phases show a marker glyph and the spelled-out status, todo items show a per-status marker, and the running round is the only one with an elapsed-time suffix.

vibesys --runs-dir /work/vibesys-runs --local \
  --input examples/model-serving/Llama-3-8B --theme solarized-light

Skills

Skill sources come from two flags, both repeatable, and each value may point at one skill directory containing SKILL.md, a parent tree containing multiple skills, or a single SKILL.md file:

• --skills-dir PATH replaces the built-in preset roots. When omitted, the preset resources/skills/ is the base.
• --extra-skills PATH stacks on top of the presets (or on top of --skills-dir when that is given). Use this to add your own skills while keeping the presets such as the llm-serving serving-systems skills. A same-named skill from --extra-skills overrides a preset one.

# presets + your own skill directory and a single SKILL.md file
vibesys --runs-dir /work/vibesys-runs --local --input <bundle> \
  --extra-skills ./my-skills \
  --extra-skills ./one-off-skill/SKILL.md

# use ONLY your skills, ignoring the presets
vibesys --runs-dir /work/vibesys-runs --local \
  --input <bundle> --skills-dir ./my-skills

Before a run starts, VibeSys discovers each SKILL.md under the candidate roots and validates its frontmatter. Optional .vibesys.toml sidecars can declare domain and backend applicability for a skill subtree:

[[rule]]
path = "skills"
backends = ["trainium"]
domains = ["llm-serving"]

Effective skill loading is the intersection of the declared constraints:

• unscoped skills load for every domain and --backend;
• skills matched by a sidecar rule with backends load only when the selected backend is in that list;
• skills matched by a sidecar rule with domains load only when the input bundle's [agent].domain is in that list;
• --skills-dir and --extra-skills add candidate roots, but routing metadata still filters the discovered skills;
• --no-skills disables all skill loading, including scoped skills, and overrides both --skills-dir and --extra-skills.

See Skill Metadata for the VibeSys-specific metadata contract and validation rules.

Target Inputs

Repository-native examples use this layout:

candidate-repository/
├── .vibesys/
│   └── tasks/<task>/
│       ├── OBJECTIVE.md
│       ├── vibesys.input.toml
│       ├── reference/
│       ├── accuracy_checker/
│       └── benchmark/
└── candidate source

For nontrivial callable APIs, ABIs, ownership rules, or service protocols, keep the normative implementation requirements in CANDIDATE_CONTRACT.md and link to it from OBJECTIVE.md. A shared evaluator may own this file when several input bundles use exactly the same contract. Keep evaluator internals and trust assumptions in a separate design document.

Launch from its root or select the root and task explicitly:

cd /path/to/project
vibesys --task <task> ...

vibesys --project /path/to/project --task <task> ...

The manifest declares direct commands or logical entry points from one exact evaluator package. Task commands run from the candidate repository root:

version = 1

[agent]
domain = "generic"

[accuracy]
entrypoint = "vibesys-queue"
args = ["check", "--workspace", "${PROJECT_ROOT}", "--scenario", "spsc"]

[benchmark]
entrypoint = "vibesys-queue"
args = ["benchmark", "--workspace", "${PROJECT_ROOT}", "--scenario", "spsc"]

[evaluator]
name = "vibesys-evaluator-queue"
version = "0.1.0"

[benchmark.result]
json_argument = "--output-json"
metric = "requests_per_second"

Direct command = [...] arrays may point at Python, shell, Go, Rust, C++, or any other task-owned program. Package entry points decouple task manifests from the package's install location. ${PROJECT_ROOT} in package arguments expands to the absolute candidate repository root.

Serving bundles that enable trusted captures declare a separate diagnostic load:

[profile]
command = ["python", "benchmark/profile.py", "--base-url", "http://127.0.0.1:VIBESYS_DYNAMIC_PORT/v1"]
timeout_seconds = 120

profile uses the same command-or-entrypoint contract as accuracy, including entrypoint args and an optional positive timeout_seconds. The bundle owns this short fixed workload in its trusted evaluator inputs. It must exercise serving without benchmark preflight, warmup or acceptance thresholds. A configured serving capture without the declaration fails with an error naming profile.command; VibeSys does not substitute the benchmark. The timeout bounds the load after service readiness, within the capture's overall deadline. Diagnostic captures do not change accuracy or benchmark acceptance.

The optional benchmark.result block opts a single-metric benchmark into trusted framework scoring: VibeSys appends json_argument, reads the resulting JSON, and requires a finite numeric field named by metric. For a JSON object, that field must be at the top level and is authoritative even when per-trial diagnostics repeat the same name. List-shaped results are accepted when they contain exactly one field with that name. Omit the result block for multi-profile or multi-objective benchmarks whose result cannot be represented by one scalar. Named profiles and benchmark parameter schemas are not part of manifest version 1.

benchmark.result_protocol is the alternative, and the two are mutually exclusive. It declares that the benchmark speaks the evaluator result protocol of that version, so VibeSys reads a complete metric row instead of scraping one named field:

[benchmark]
entrypoint = "vibesys-queue"
args = ["benchmark", "--workspace", "${PROJECT_ROOT}", "--scenario", "spsc"]
result_protocol = 2

VibeSys appends --vs-output with a path, and the benchmark writes a record stream there: a hello record declaring every metric it produces, then one result record carrying their values, or an error record explaining why it could not measure. The framework validates the row against the declaration and fails the round when they disagree, when a value is missing or not finite, or when a configured objective names a metric the benchmark does not produce. The protocol is specified in sdk/vs-evaluator/PROTOCOL.md, and sdk/vs-evaluator/vseval is a Go SDK for emitting it. Use this form for multi-objective benchmarks, which the scalar block cannot represent.

Task resources, including held-out evaluation sets, remain at their repository paths. VibeSys does not relocate them. .vibesys is read-only to coding agents, so task commands must write scratch data outside .vibesys (for example under /tmp or a candidate-excluded build directory).

For --modal runs, a task may declare the candidate deployment file:

[environment.modal]
entrypoint = "examples/deployment/service.py"

The entrypoint is relative to the candidate repository root, must remain inside that repository, and defaults to main.py when omitted. The manifest supplies task-owned deployment wiring; the operator still selects Modal with --modal.

Task-local evaluator source

The optional evaluator.source path is relative to the task directory and must resolve to a directory inside it. It is copied with its task and stays under .vibesys/tasks/, so it is read-only for the agent, and Git-backed integrity checks reject accuracy and benchmark gates after it is modified. Resumed runs use the evaluator snapshot committed in the canonical project.

Providing inputs without a bundle (--input-*)

For external usage without a project on disk, pass a task's contents as separate --input-* flags instead of --input. VibeSys synthesizes a one-task project under <runs-dir>/_inputs/<exp-name>/ (task default) and then loads it through the same path as --input, so every loop, resume, and evaluator behaves identically. The two forms are mutually exclusive: combining --input with any --input-* flag is rejected.

Required flags:

Flag	Maps to
--input-objective TEXT or --input-objective-file PATH	.vibesys/tasks/default/OBJECTIVE.md
--input-domain {llm-serving,generic,microservices}	[agent].domain
--input-accuracy-command CMD	[accuracy].command (shell-quoted argv)
--input-benchmark-command CMD	[benchmark].command (shell-quoted argv)

Optional flags:

Flag	Maps to
--input-accuracy-timeout SECONDS / --input-benchmark-timeout SECONDS	command timeout_seconds
--input-benchmark-metric NAME + --input-benchmark-result-arg OPT	[benchmark.result] (both required together)
--input-reference DIR	copied to .vibesys/tasks/default/reference/
--input-evaluator-dir DIR	contents copied into the project root, where the commands run
--input-evaluator-source DIR	[evaluator].source (staged inside the task)

The synthesized project stages --input-evaluator-source inside its task before loading the manifest, so the supplied local directory does not need to share a parent repository with the generated bundle. Git-pinned [[workspace.sources]] entries are not exposed as flags; use --input for those.

vibesys \
  --runs-dir /work/vibesys-runs \
  --input-objective-file ./OBJECTIVE.md \
  --input-domain llm-serving \
  --input-accuracy-command "python checker.py" \
  --input-benchmark-command "python benchmark.py" \
  --input-benchmark-metric requests_per_second \
  --input-benchmark-result-arg=--result-json \
  --input-evaluator-dir ./evaluator \
  --local

Common Commands

Default agent loop on local CUDA-compatible host:

vibesys \
  --runs-dir /work/vibesys-runs \
  --local \
  --outer-loop agent \
  --backend cuda \
  --interface inprocess \
  --input examples/model-serving/Llama-3-8B

Docker CUDA run:

vibesys --runs-dir /work/vibesys-runs --local \
  --outer-loop agent --backend cuda ...

Modal GPU run:

vibesys --runs-dir /work/vibesys-runs --local \
  --outer-loop agent --backend cuda --modal --profiler torch ...

Trainium run:

vibesys --runs-dir /work/vibesys-runs --local \
  --outer-loop agent --backend trainium --profiler auto ...

Over-the-wire service target:

vibesys \
  --runs-dir /work/vibesys-runs \
  --local \
  --outer-loop agent \
  --interface service \
  --input examples/<target>

CPU-only target:

vibesys --runs-dir /work/vibesys-runs --local \
  --outer-loop agent --backend cpu --interface service ...

CPU runs use the default Docker agent environment.

Maintenance Rule

When adding or changing a flag:

1. Update this document.
2. Add or update validation for unsupported combinations.
3. Add prompt-rendering tests for combinations that change generated instructions.
4. Keep README focused on quickstart guidance and link here for details.