Skip to content

Upgrade --contains filter to accommodate always-run tests #142

@bonachea

Description

@bonachea

Julienne has a handy --contains feature that lets users filter the tests to execute at runtime. However there are sometimes test dependencies where test "A" MUST run in order for later test "B" to work at all, and unfortunately --contains does not currently accommodate such cases.

The most immediate example of this is Caffeine's prif_init test, which initializes the library. It must run first to initialize the library before anything other tests can function. Unfortunately this means that running Caffeine's Julienne driver with an argument like --contains prif_co_sum is guaranteed to fail, because the prif_co_sum test cannot operate unless the earlier prif_init test runs first.

Some possible solutions:

  1. Hack the client code to force the mandatory test to always run, regardless of Julienne's --contains argument; this is the stop-gap approach I plan to deploy in the near-term.
  2. Upgrade the --contains options to accept a list of string-matching filters, so the user can instead type something like --contains prif_init,prif_co_sum (or some other reasonable syntax). This puts the responsibility on the end user to understand the inter-test dependencies, but perhaps that's reasonable because this feature seems geared for developers anyhow.
  3. Add an optional argument to the test_fixture_t constructor to specify that a given fixture is mandatory and always runs (ignoring any --contains filter).
  4. Deploy a full-blown general infrastructure to express an inter-test dependency DAG and ensure that --contains respects those dependencies. I'm NOT in favor of this approach, because it seems like alot more work to implement in Julienne (and possibly also to use) and would be overkill for my simple use case.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions