Skip to content

BasicMetaschemaTest.testExistsWithVariable times out intermittently on CI #533

@david-waltermire

Description

@david-waltermire

Summary

The test BasicMetaschemaTest.testExistsWithVariable intermittently times out after 60 seconds on CI, causing build failures. This is likely a spurious test concurrency issue.

Failure Details

[ERROR] BasicMetaschemaTest.testExistsWithVariable » Timeout testExistsWithVariable() timed out after 60 seconds

Example failure: https://github.com/metaschema-framework/metaschema-java/actions/runs/20066549687/job/57556545409

Root Cause Analysis

The test has several characteristics that may contribute to the timeout:

  1. External Network Dependency: Downloads the OSCAL complete metaschema from GitHub:

    IBindingMetaschemaModule module = bindingContext.loadMetaschema(
        new URL("https://raw.githubusercontent.com/usnistgov/OSCAL/main/src/metaschema/oscal_complete_metaschema.xml"));
  2. Recursive Import Loading: Uses recurse-depth to load all transitive imports:

    IMetapathExpression allImportsExpression = IMetapathExpression.compile(
        "recurse-depth(/METASCHEMA,'for $import in ./import return doc(resolve-uri($import/@href))/METASCHEMA')",
        staticContext);
  3. Likely Concurrency Issue: Despite the test class having @Execution(value = ExecutionMode.SAME_THREAD), there may be shared state or resource contention with other tests running in parallel that causes the timeout. This is likely a spurious failure that doesn't occur consistently.

Affected Code

  • File: databind/src/test/java/gov/nist/secauto/metaschema/databind/codegen/BasicMetaschemaTest.java
  • Test method: testExistsWithVariable() (lines 186-223)

Investigation Needed

  1. Identify any shared mutable state accessed by this test
  2. Check for resource contention (e.g., caching, static initialization)
  3. Determine if the @Execution(SAME_THREAD) annotation is being honored
  4. Review if there are any lazy-initialized singletons that could cause blocking

Environment

  • CI Platform: GitHub Actions
  • Test Framework: JUnit 5 with parallel execution enabled
  • Note: Test class has @Execution(value = ExecutionMode.SAME_THREAD) but this may not prevent all concurrency issues with tests in other classes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    Status

    Done

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions