Skip to content

RSDK-13711 Specify test version#239

Open
EvanDorsky wants to merge 36 commits into
viamrobotics:mainfrom
EvanDorsky:specify-test-version
Open

RSDK-13711 Specify test version#239
EvanDorsky wants to merge 36 commits into
viamrobotics:mainfrom
EvanDorsky:specify-test-version

Conversation

@EvanDorsky
Copy link
Copy Markdown
Member

@EvanDorsky EvanDorsky commented May 15, 2026

This PR re-engineers the test suite to enable testing against a specified version of viam-agent instead of stable.

Interface/infra changes

  • new config value: viam_agent_test to specify the version under test
  • installAgent now uses the local install.sh (just like uninstallAgent) instead of pulling the install script from gcp.
  • installAgent now checks that viam-agent is running at the specified version, instead of just running at all. If the version isn't correct, it installs the specified version.
  • There is a new way to specify the version in test cases, called "the version under test"
  • The after suite hook now pins agent back to the version under test, instead of stable
  • serialclient.EnsureOnline was re-added to the pre-scenario hook (this fixes some instability caused by the provisioning tests leaving the network adapter in a weird state)
  • new code to retrieve specific versions based on the high level version spec, and gcs url based on specific versions (detailed below)

The agent version under test can be specified in a number of ways:

  • pr.xxx will test a PR by number, if it has the dev-release label (so that artifacts are built) e.g. "pr.227"
  • "dev" will test the tip of main
  • "stable" will test the latest stable release
  • "x.y.z" will test any release version e.g. "0.27.3"
  • "file:///path/to/agent" will test an arbitrary binary (that you manually copy onto the pi)

Each feature test now prints the versions under test before running:

=================================================
=================================================
    Testing Agent Version: dev (0.27.3-dev.8)

          Stable Agent Version: 0.27.3
         Stable Server Version: 0.123.0
=================================================
=================================================

Test changes

Tests have been modified to account for the new syntax requirements to specify versions (like stable, the version under test, an old version):

  • viam-agent is installed -> viam-agent is installed at the version under test
  • the viam-agent systemd unit is running -> the viam-agent systemd unit is running with the version under test

The upgrade and downgrade tests have been changed so they make sense with this new flow:

  • upgrade.feature upgrades from stable to the version under test
  • downgrade.feature downgrades from the version under test to stable

Changes beyond tests

This PR makes a small modification to install.sh - it adds $AGENT_CUSTOM_URL, which will be used instead of $URL as the download target for the agent binary if it $AGENT_CUSTOM_URL is passed in. This should be totally transparent if not used, but it enables the test to install any version (instead of relying on agent to update itself via pinning, which isn't really the same thing).

EvanDorsky and others added 30 commits April 21, 2026 13:50
viamrobotics#232)

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants