[Goal] [Tests] Reach 10,000 unit tests and 100% test coverage

[Nucs]

Overview

Expand NumSharp's test suite from ~4,300 to 10,000 tests with systematic coverage tracking.

Current State

Metric	Value
Actual tests	~4,323
Target	10,000
Gap	~5,677 tests needed
Test files	290
Test classes	178
Source files	417
Source LOC	~42K
ILKernelGenerator LOC	~21K (28 partial files)
OpenBugs tests	124 (excluded from CI)

Problem

1. Coverage gaps: Many operations lack dtype/layout coverage (see KERNEL_API_AUDIT.md)
2. Dynamic IL code: ~21K lines in ILKernelGenerator uses DynamicMethod - standard coverage tools (Coverlet, OpenCover) cannot instrument dynamically generated code
3. Type matrix: 12 dtypes × N operations × M layouts = combinatorial explosion not fully tested

Proposal

Phase 1: Coverage Infrastructure

• Add Coverlet to CI pipeline for static code coverage
• Create KernelCoverageTracker for IL kernel behavioral coverage
• Establish coverage baseline report

Phase 2: Test Matrix Expansion (~3,000 tests)

• Parameterize existing tests with all 12 dtypes using [Arguments]
• Add layout variants (contiguous, strided, broadcast) to core tests
• Create dtype coverage tests for each operation category

Phase 3: Edge Case Batteries (~1,500 tests)

• Empty array handling for all operations
• Scalar input/output tests
• NaN/Inf propagation tests (float types)
• Negative index and slice tests
• Broadcast shape combination tests

Phase 4: NumPy Parity Tests (~1,000 tests)

• Port tests from NumPy's own test suite (src/numpy/)
• Add verified-against-NumPy tests for each function
• Document any intentional [Misaligned] differences

Phase 5: IL Kernel Coverage (~500 tests)

• Binary ops: all dtype pairs × all operations
• Unary ops: all dtypes × all functions
• Reduction ops: axis/no-axis × keepdims variants
• Comparison ops: all dtype pairs

Dynamic Code Coverage Strategy

Since DynamicMethod IL can't be instrumented by profilers:

1. Kernel Hit Tracking: Add #if DEBUG counters in ILKernelGenerator to track which kernels execute
2. Test Matrix Validation: Ensure all (dtype × layout × operation) combinations are tested
3. Behavioral Verification: Tests verify output matches NumPy (implicit coverage)
4. Coverage Report: Generate report showing kernel path coverage separate from Coverlet

Test Organization

test/NumSharp.UnitTest/
├── Coverage/                    # NEW: Coverage tracking infrastructure
│   ├── KernelCoverageTracker.cs
│   └── CoverageReportTests.cs
├── DtypeMatrix/                 # NEW: All dtype combinations
│   ├── BinaryOps/
│   ├── UnaryOps/
│   └── Reductions/
├── LayoutMatrix/                # NEW: Memory layout variations
│   ├── ContiguousTests.cs
│   ├── StridedTests.cs
│   └── BroadcastTests.cs
└── EdgeCases/                   # NEW: Systematic edge cases
    ├── EmptyArrayTests.cs
    ├── ScalarTests.cs
    └── NaNPropagationTests.cs

Coverage Tool Analysis

Tool	Dynamic Code Support	Notes
Coverlet	❌ No	Standard .NET coverage, instruments assemblies at build time
OpenCover	❌ No	Already referenced in project, Windows only
dotCover	❌ No	JetBrains commercial tool
AltCover	❌ No	Open source alternative

Reality: "100% coverage" will show ~60-70% in Coverlet reports because ~21K lines of ILKernelGenerator emit code at runtime. Behavioral coverage can reach 100%, but reported coverage won't reflect it.

Definition of Done

• 10,000+ tests passing
• Static code coverage > 80% (via Coverlet)
• IL kernel coverage tracking shows all paths exercised
• All 12 dtypes tested for each operation
• All 3 memory layouts tested for core operations
• No untested NumPy API functions
• Coverage reports integrated into CI

Test Multiplication Quick Wins

Strategy	Tests Added	Effort
Dtype parameterization	~3,000+	Low (add [Arguments] to existing tests)
Layout parameterization	~1,500+	Medium (contiguous/strided/broadcast)
Edge case batteries	~1,000+	Medium (empty, scalar, NaN, indices)
Missing operation tests	~500+	High (new test files)

Related

• docs/KERNEL_API_AUDIT.md - tracks test gaps per operation
• OpenBugs files - 124 known failing tests to fix