🧪 Test gap analysis — 4 gaps found in Python/Node runtimes and upload-pipeline-artifact

[github-actions]

Test Gap Analysis

Test suite snapshot: 1,190 unit tests, 80 integration tests (tests/compiler_tests.rs), 3 init tests, 8 MCP HTTP tests — 1,278 total. All pass. ✅

Previous issues #376 and #392 remain open. The gaps below are new — introduced by the Python/Node runtime additions and upload-pipeline-artifact tool, none of which were covered by the prior issues.

---

Priority Gaps

Module	Function/Path	Why It Matters	Suggested Test
runtimes/python/mod.rs	generate_python_install() — 0 unit tests, no integration test for Python runtime path	Wrong versionSpec interpolation silently produces broken pipelines. Version fallback (3.x) and WithOptions path are untested	Unit test default/pinned version; integration test with runtimes: python: true
runtimes/python/mod.rs	generate_pip_authenticate() — 0 unit tests	Function always emits a PipAuthenticate@1 step; no test verifies it emits a valid YAML task block	Snapshot test the YAML output
runtimes/node/mod.rs	generate_node_install(), generate_npm_authenticate(), generate_ensure_npmrc() — 0 unit tests, no integration test	generate_ensure_npmrc conditionally uses feed-url or falls back to https://registry.npmjs.org/; both branches are untested. No pipeline compilation test exists for the Node runtime path	Unit test each fn; integration test with runtimes: node: true and runtimes: node: {feed-url: ...}
tests/mcp_http_tests.rs	test_mcp_initialize_and_tools_list — upload-pipeline-artifact not asserted	~20 tools are registered in src/mcp.rs; only 5 are asserted. A broken MCP registration for upload-pipeline-artifact would go completely undetected	Add assert!(body.contains("upload-pipeline-artifact"), ...)
src/execute.rs	No execute-level test for upload-pipeline-artifact dispatch (line 353)	Every other complex tool has at least one test_execute_malformed_* or test_execute_*_missing_context test. The dispatch arm at line 353 is unreachable by the current test suite	Mirror test_execute_malformed_update_wiki_page_returns_err for this tool

---

Suggested Test Cases

1. Python runtime — unit tests in src/runtimes/python/mod.rs

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_generate_python_install_default_version() {
        let config = PythonRuntimeConfig::Enabled(true);
        let step = generate_python_install(&config);
        assert!(step.contains("versionSpec: '3.x'"), "should default to 3.x, got: {step}");
        assert!(step.contains("UsePythonVersion@0"), "should use UsePythonVersion task");
        assert!(step.contains("Install Python 3.x"), "should set displayName");
    }

    #[test]
    fn test_generate_python_install_pinned_version() {
        let config = PythonRuntimeConfig::WithOptions(PythonOptions {
            version: Some("3.12".into()),
            ..Default::default()
        });
        let step = generate_python_install(&config);
        assert!(step.contains("versionSpec: '3.12'"), "should use pinned version, got: {step}");
        assert!(step.contains("Install Python 3.12"));
    }

    #[test]
    fn test_generate_pip_authenticate_emits_task() {
        let step = generate_pip_authenticate();
        assert!(step.contains("PipAuthenticate@1"), "should emit PipAuthenticate task");
        assert!(step.contains("artifactFeeds"), "should include artifactFeeds input");
    }
}

2. Node runtime — unit tests in src/runtimes/node/mod.rs

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_generate_node_install_default_version() {
        let config = NodeRuntimeConfig::Enabled(true);
        let step = generate_node_install(&config);
        assert!(step.contains("versionSpec: '22.x'"), "should default to 22.x, got: {step}");
        assert!(step.contains("NodeTool@0"));
    }

    #[test]
    fn test_generate_node_install_pinned_version() {
        let config = NodeRuntimeConfig::WithOptions(NodeOptions {
            version: Some("20.x".into()),
            ..Default::default()
        });
        let step = generate_node_install(&config);
        assert!(step.contains("versionSpec: '20.x'"), "got: {step}");
    }

    #[test]
    fn test_generate_npm_authenticate_emits_task() {
        let step = generate_npm_authenticate();
        assert!(step.contains("npmAuthenticate@0"));
    }

    #[test]
    fn test_generate_ensure_npmrc_default_registry() {
        let config = NodeRuntimeConfig::Enabled(true);
        let step = generate_ensure_npmrc(&config);
        assert!(step.contains("https://registry.npmjs.org/"), "should fallback to npm registry, got: {step}");
        assert!(step.contains("Ensure .npmrc exists"));
    }

    #[test]
    fn test_generate_ensure_npmrc_custom_feed_url() {
        let config = NodeRuntimeConfig::WithOptions(NodeOptions {
            feed_url: Some("(pkgs.dev.azure.com/redacted),
            ..Default::default()
        });
        let step = generate_ensure_npmrc(&config);
        assert!(step.contains("pkgs.dev.azure.com"), "should use custom feed URL, got: {step}");
    }
}

3. Python/Node runtime integration tests in tests/compiler_tests.rs

#[test]
fn test_python_runtime_compiled_output() {
    let input = "---\nname: \"Python Test\"\ndescription: \"Tests Python runtime\"\nruntimes:\n  python: true\nsafe-outputs:\n  noop: {}\n---\n\n## Test\n";
    let compiled = compile_pipeline_from_str(input, CompileTarget::Standalone).unwrap();
    assert!(compiled.contains("UsePythonVersion@0"), "should have Python install step");
    assert!(compiled.contains("versionSpec: '3.x'"), "should default to Python 3.x");
}

#[test]
fn test_python_runtime_pinned_version_compiled_output() {
    let input = "---\nname: \"Python 312 Test\"\ndescription: \"Tests pinned Python\"\nruntimes:\n  python:\n    version: \"3.12\"\nsafe-outputs:\n  noop: {}\n---\n\n## Test\n";
    let compiled = compile_pipeline_from_str(input, CompileTarget::Standalone).unwrap();
    assert!(compiled.contains("versionSpec: '3.12'"), "should use pinned version");
}

#[test]
fn test_node_runtime_compiled_output() {
    let input = "---\nname: \"Node Test\"\ndescription: \"Tests Node runtime\"\nruntimes:\n  node: true\nsafe-outputs:\n  noop: {}\n---\n\n## Test\n";
    let compiled = compile_pipeline_from_str(input, CompileTarget::Standalone).unwrap();
    assert!(compiled.contains("NodeTool@0"), "should have Node install step");
    assert!(compiled.contains("versionSpec: '22.x'"), "should default to Node 22.x");
}

4. MCP tools list — assert upload-pipeline-artifact is registered

// In tests/mcp_http_tests.rs, add to test_mcp_initialize_and_tools_list:
assert!(
    body.contains("upload-pipeline-artifact"),
    "Should list upload-pipeline-artifact tool, body: {body}"
);
// Note: also still missing from #376/#392:
// assert!(body.contains("upload-workitem-attachment"), ...);
// assert!(body.contains("upload-build-attachment"), ...);

5. Execute dispatch — upload-pipeline-artifact

#[tokio::test]
async fn test_execute_malformed_upload_pipeline_artifact_returns_err() {
    let dir = tempfile::tempdir().unwrap();
    let path = dir.path().join("out.ndjson");
    let line = r#"{"tool":"upload-pipeline-artifact","params":{"not_valid":true}}"#;
    std::fs::write(&path, line).unwrap();

    let result = execute_safe_outputs(
        dir.path(),
        &path,
        &ExecutionContext::default(),
        false,
    )
    .await;
    assert!(result.is_err() || result.unwrap().iter().any(|r| r.is_err()),
        "Malformed upload-pipeline-artifact should fail");
}

---

Coverage Summary

Module	Public Fns	Tests	Coverage Estimate
runtimes/python/mod.rs	generate_python_install, generate_pip_authenticate + 4 getters	0	~0% for generators
runtimes/node/mod.rs	generate_node_install, generate_npm_authenticate, generate_ensure_npmrc + 4 getters	0	~0% for generators
tests/mcp_http_tests.rs	test_mcp_initialize_and_tools_list	asserts 5/~20 registered tools	upload-pipeline-artifact unasserted
src/execute.rs — line 353	upload-pipeline-artifact dispatch arm	0 execute-level tests	0% for this arm

---

This issue was created by the automated test gap finder. Previous runs: #376 (2026-05-01, still open), #392 (2026-05-04, still open). Modules audited this cycle: runtimes/python, runtimes/node, mcp_http_tests, execute.rs (upload dispatch). Total tests found: 1,278 (up from 1,252 at #392 creation).

Generated by Test Gap Finder · ● 1.1M · ◷