Skip to content

Register/disposition compiled-but-never-run tests (push line coverage past 90%) #6370

@hjmjohnson

Description

@hjmjohnson

Summary

While working toward pushing ITK's line coverage past 90% (the CDash itk-coverage build currently reports 89.51%, ~760 covered lines short), I found a systemic source of silent coverage loss: test source files that are compiled into their module test drivers but are never invoked by any itk_add_test. They build, count as "present," but never run — so the code they would exercise shows as uncovered.

itkOrientImageFilter was the motivating example: 0% coverage on CDash despite two existing test files, purely because neither was registered with itk_add_test. A toolkit-wide scan found ~30 more.

This issue tracks registering (or otherwise dispositioning) each one. Verified history shows these are move/reorg oversights, not deliberate removals: none ever had an itk_add_test (git log -G = 0 historical invocations), none have GoogleTest replacements, and Nonunit/IntegratedTest/test/CMakeLists.txt even carried explicit # missing itk_add_test() call comments.

Detection method (reproducible)

For each Modules/**/test/CMakeLists.txt: strip #-comments, collect itkFoo.cxx source tokens that exist on disk, skip GoogleTest bodies (TEST( / itkGTest.h), require the file to define int <basename>(int …), then flag if <basename> never appears in the CMake except as <basename>.cxx.

Known blind spots (fix before re-running): tests whose names are built in a foreach/variable (e.g. itkDiscrete${loop_var}Curvature…) are not dormant but look it; and a commented-out itk_add_test means intentionally disabled, not missing.

Done — registered, build + run locally, all pass (15 tests)
  • Filtering/ImageGrid: itkOrientImageFilterTest (useImageDirection=0) + legacy itkOrientImageFilterTest2
  • Core/Common: itkObjectStoreTest, itkImageComputeOffsetAndIndexTest, itkImageRegionExplicitTest, itkMetaProgrammingLibraryTest, itkPromoteType
  • Nonunit/IntegratedTest: itkFilterImageAddTest, itkReleaseDataFilterTest (the two with # missing itk_add_test() comments)
  • IO/JPEG2000: itkJPEG2000ImageIOFactoryTest01, itkJPEG2000ImageIOTest00
  • Segmentation/LevelSetsv4: itkLevelSetEquationChanAndVeseInternalTermTest, itkLevelSetEquationRegionTermTest, itkLevelSetEquationTermBaseTest, itkLevelSetEquationTermContainerTest
Skipped — fail when run or need investigation (NOT just a registration)
  • Core/QuadEdgeMesh/itkQuadEdgeMeshBasicLayerTest — runtime failure: Erroneous GetDestination() on edge number 0. Was expecting 2 but got 0. Behavior drift since it last ran.
  • Segmentation/LevelSetsv4/itkLevelSetEquationChanAndVeseExternalTermTest — runs to "CV External term created" then returns EXIT_FAILURE (internal assertion/value issue).
  • Nonunit/Review/itkWarpJacobianDeterminantFilterTestITKReview not built in the default configuration; needs the module enabled to verify.
Needs fixtures / parameters before registering
  • Filtering/LabelMap (3): itkStatistics{KeepNObjects,Opening,Relabel}LabelMapFilterTest1 — need 6 args (input + feature image + output + reverseOrdering + a valid statistics-attribute enum + count) and no baseline exists (would need one published).
  • IO/JPEG2000: itkJPEG2000ImageIOTest01/02/04 — need .j2k inputs + args.
  • IO/DCMTK/itkDCMTKImageIOMultiFrameImageTest — needs a multi-frame DICOM fixture.
  • Video/BridgeVXL/itkVXLVideoIOTest — needs VXL + a video fixture.
  • Core/ImageFunction / ImageGradient / Registration/Metricsv4 (4 *SpeedTest) — benchmarks; likely intentionally not run in CI (confirm before enabling).
Scanner false positives (already running / intentionally disabled — leave alone)
  • Filtering/QuadEdgeMeshFiltering (4): itkDiscrete{Gaussian,Maximum,Mean,Minimum}CurvatureQuadEdgeMeshFilterTest — already registered via foreach(loop_var ${CURV_TESTS}).
  • Filtering/IsotropicWavelets (2): itkZeroDCImageFilterTest, itkWaveletFrequencyFilterBankGeneratorDownsampleTestitk_add_test blocks are commented out (deliberate disable).
  • Bridge/VtkGlue/runViewImage — commented-out interactive viewer (needs a display).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions