Skip to content

feat: Add optional setup command to helix.yaml #11

@amrit110

Description

@amrit110

Summary

Add support for an optional setup command in helix.yaml that runs once before the first experiment in a session. This allows helices to declare one-time setup steps (e.g. downloading model weights, caching datasets) as part of their configuration, making helix run fully self-contained without requiring manual pre-run steps.

Motivation

Currently, some helices require a one-time setup step to be run manually before helix run can work — for example, uv run prepare.py in helix-inference-opt downloads model weights and caches datasets. There is no way to declare this in helix.yaml, so users must remember to run it themselves on each new machine. This is a poor experience and easy to forget.

Proposed Change

Add an optional setup section to helix.yaml:

setup:
  command: uv run prepare.py

Behaviour

  • If setup.command is provided, the runner executes it once at the start of helix run, before the agent is invoked or any experiments are run.
  • If setup.command is absent, the runner proceeds as today (no change in behaviour).
  • The command runs in the repo root, inheriting the same environment as the evaluate command.
  • If the setup command exits non-zero, helix run should abort with a clear error message.

Schema Changes

In config.py, add a new optional SetupConfig model and field on HelixConfig:

class SetupConfig(BaseModel):
    command: str

class HelixConfig(BaseModel):
    ...
    setup: Optional[SetupConfig] = None

Runner Changes

In runner.py, after _preflight() and before launching the agent, add a setup step:

if self.config.setup:
    # run setup.command, stream output, abort on non-zero exit

Example helix.yaml

name: inference-opt
setup:
  command: uv run prepare.py
metrics:
  primary:
    name: tokens_per_sec
    optimize: maximize
  evaluate:
    command: uv run infer.py

Acceptance Criteria

  • SetupConfig added to config.py with command field
  • HelixConfig.setup is optional (existing helices without it are unaffected)
  • Runner executes setup command once per helix run session before the agent starts
  • Setup output is streamed to the terminal so the user can see progress
  • Non-zero exit aborts the session with a clear error
  • Docs/template updated to show the new field

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions