Skip to content

start_debugging needs additional parameters for real-world .NET test framework execution #81

@mikejohnstonPremierinc

Description

@mikejohnstonPremierinc

Feature Request:

Problem

When bug #80 is fixed and createTestDebugConfig becomes reachable for coreclr, the auto-generated config will be:

args: ['test', '--filter', `FullyQualifiedName~${testName}`, '--no-build']

This is insufficient for real-world .NET test automation frameworks. Enterprise test projects require additional dotnet test parameters that have no corresponding start_debugging input:

Required Parameter Purpose Example
--settings <path> Specifies runsettings file (environment URL, credentials, timeouts) --settings RunSettings/project.qa.runsettings
Project path (.csproj) Specifies which test project to run (solutions contain multiple projects) dotnet test MyApp.UITest/MyApp.UITest.csproj
Additional --filter criteria Browser selection, fixture type, device exclusion --filter "Name=MyTest&TestCategory=Remote.chrome&FullyQualifiedName!~iPhone"
--environment / env vars Runtime environment variables needed by the test framework BROWSER=chrome, credential injection

Real-World Example

Here's what a real dotnet test command looks like for an enterprise NUnit/Selenium test framework:

dotnet test MyApp.UITest/MyApp.UITest.csproj \
  --filter "Name=LoginTest&TestCategory=Local.chrome&FullyQualifiedName!~iPhone" \
  --settings MyApp.UITest/RunSettings/myapp.qa.runsettings

Breaking this down:

  • MyApp.UITest.csproj - The solution has 4+ projects; only one is the test project
  • Name=LoginTest - The specific test method (this is what testName maps to today)
  • TestCategory=Local.chrome - Run with Local Chrome fixture only (not Remote, not Edge, not Firefox)
  • FullyQualifiedName!~iPhone - Exclude iPhone emulation device tests (Chrome has two variants: standard + iPhone emulation)
  • --settings ...qa.runsettings - Points to the QA environment config containing application URL, credentials, timeouts, and other environment-specific settings

Without these parameters, the test either:

  1. Fails immediately (no runsettings = no application URL to navigate to)
  2. Runs ALL browser/fixture combinations simultaneously (chrome + edge + remote + local = resource explosion)
  3. Runs against the wrong project or fails to find the test entirely

Proposed Solution

Add these parameters to start_debugging:

// Option A: Named parameters (better UX for AI agents)
start_debugging({
  fileFullPath: "path/to/TestFile.cs",
  testName: "LoginTest",
  workingDirectory: "path/to/workspace",
  projectPath: "MyApp.UITest/MyApp.UITest.csproj",        // NEW
  testSettings: "MyApp.UITest/RunSettings/myapp.qa.runsettings",  // NEW
  testFilterExpression: "Name=LoginTest&TestCategory=Local.chrome&FullyQualifiedName!~iPhone",  // NEW (overrides testName-based filter)
  environmentVariables: { "BROWSER": "chrome" }            // NEW
})

// Option B: Generic additionalArgs (simpler implementation, covers all cases)
start_debugging({
  fileFullPath: "path/to/TestFile.cs",
  testName: "LoginTest",
  workingDirectory: "path/to/workspace",
  additionalTestArgs: [                                    // NEW
    "--settings", "MyApp.UITest/RunSettings/myapp.qa.runsettings",
    "--", "NUnit.NumberOfTestWorkers=1"
  ],
  testProjectPath: "MyApp.UITest/MyApp.UITest.csproj"     // NEW
})

Option A is more explicit and easier for AI agents to reason about. Option B is more flexible and handles edge cases we haven't thought of.

Either way, the generated args array in createTestDebugConfig would become:

case 'coreclr':
    const args = ['test'];
    if (projectPath) args.push(projectPath);
    args.push('--filter', testFilterExpression || `FullyQualifiedName~${testName}`);
    if (testSettings) args.push('--settings', testSettings);
    args.push('--no-build');
    if (additionalTestArgs) args.push(...additionalTestArgs);
    return {
        type: 'coreclr', request: 'launch',
        name: `DebugMCP .NET Test: ${testName}`,
        program: 'dotnet',
        args: args,
        console: 'integratedTerminal',
        cwd: cwd,
        stopAtEntry: false,
        env: environmentVariables || {}
    };

Why This Matters for AI-Assisted Debugging

The value proposition of DebugMCP is autonomous AI debugging - the agent decides what to debug and does it without human intervention. For .NET test projects, an AI agent already knows (from project context):

  • Which .csproj contains the test
  • Which runsettings file to use (from prior test runs or project conventions)
  • Which browser/fixture to target (from the test fixture attributes)
  • What filter expression to construct

The agent has all this information. It just needs a way to PASS it to start_debugging. Without these parameters, the only option is the configurationName workaround (manually creating launch.json configs), which:

  • Requires per-project setup that doesn't scale
  • Has hardcoded paths that break across environments
  • Cannot dynamically target different tests without modifying files each time
  • Defeats the "zero configuration" promise of DebugMCP

Environment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions