Bug: levels/norm conflict in plot_hist_with_contours when both provided

[blalterman]

Summary

When both levels and a custom norm (e.g., BoundaryNorm) are passed to plot_hist_with_contours, unexpected color mapping occurs because matplotlib's contourf() interprets both parameters independently.

Root Cause

In plot_contours (lines 1012-1013), there's logic to auto-create a BoundaryNorm from levels when norm is None:

if (norm is None) and (levels is not None) and (len(levels) >= 2):
    norm = mpl.colors.BoundaryNorm(levels, 256, clip=True)

However, in plot_hist_with_contours, this logic is absent. When a user provides:

• levels=[0.1, 0.3, 0.5, 0.7, 0.9]
• norm=BoundaryNorm([vmin, ..., vmax], ncolors)

The levels and norm.boundaries may not match, causing:

• Contour lines at different values than color transitions
• Inconsistent colorbar ticks

Workaround

Pass only levels to plot_hist_with_contours (omit norm):

hist.plot_hist_with_contours(
    levels=np.linspace(vmin, vmax, 11),
    # Do not pass norm - let matplotlib handle it
)

Or use plot_contours directly which handles this correctly:

hist.plot_contours(use_contourf=True, levels=np.linspace(...))

Environment

• solarwindpy version: current main
• matplotlib: 3.x

[blalterman]

Investigation and Remediation Plan

1. Evidence-Based Root Cause Analysis

Code Evidence

plot_contours method (lines 1010-1024):

levels = self._get_contour_levels(levels)

if (norm is None) and (levels is not None) and (len(levels) >= 2):
    norm = mpl.colors.BoundaryNorm(levels, 256, clip=True)

contour_fcn = ax.contour
if use_contourf:
    contour_fcn = ax.contourf

if levels is None:
    args = [XX, YY, C]
else:
    args = [XX, YY, C, levels]

qset = contour_fcn(*args, linestyles=linestyles, cmap=cmap, norm=norm, **kwargs)

plot_hist_with_contours method (lines 645-672):

levels = self._get_contour_levels(levels)

contour_fcn = ax.contourf if use_contourf else ax.contour

if levels is None:
    args = [XX_centers, YY_centers, C_centers]
else:
    args = [XX_centers, YY_centers, C_centers, levels]

qset = contour_fcn(*args, linestyles=linestyles, cmap=cmap, norm=norm, **contour_kwargs)

Bug Identification

Aspect	plot_contours	plot_hist_with_contours
Auto-creates BoundaryNorm from levels when norm is None	✅ Yes (line 1012-1013)	❌ No
Passes levels separately to contour function	✅ Yes	✅ Yes
Conflict when both norm and levels provided	⚠️ Potential	⚠️ Definite

2. Proposed Fix

Option A+C (Recommended): Extract levels from BoundaryNorm + Warn on conflict

1. Create helper method _reconcile_levels_and_norm(levels, norm):

   • If norm is a BoundaryNorm and levels is None, extract levels from norm.boundaries
   • If norm is None and levels is provided, create BoundaryNorm from levels
   • If both are provided with mismatched values, issue UserWarning

2. Apply to both methods for consistent behavior

Helper Implementation:

def _reconcile_levels_and_norm(self, levels, norm):
    import warnings

    # Extract levels from BoundaryNorm if not provided
    if levels is None and isinstance(norm, mpl.colors.BoundaryNorm):
        levels = list(norm.boundaries)

    # Create BoundaryNorm from levels if norm not provided
    elif norm is None and levels is not None and len(levels) >= 2:
        norm = mpl.colors.BoundaryNorm(levels, 256, clip=True)

    # Warn on mismatch
    elif levels is not None and isinstance(norm, mpl.colors.BoundaryNorm):
        if not np.allclose(levels, norm.boundaries):
            warnings.warn(
                f"levels ({levels}) do not match norm.boundaries "
                f"({list(norm.boundaries)}). This may cause unexpected "
                "color mapping. Consider providing only one of levels or norm.",
                UserWarning,
                stacklevel=3
            )

    return levels, norm

3. Test-Driven Development Plan

Test ID	Input	Expected Output
U1	levels=None, norm=None	(None, None)
U2	levels=[0.1, 0.5, 0.9], norm=None	([0.1, 0.5, 0.9], BoundaryNorm)
U3	levels=None, norm=BoundaryNorm([0.1, 0.5, 0.9], 256)	([0.1, 0.5, 0.9], norm)
U4	levels=[0.1, 0.5, 0.9], norm=BoundaryNorm([0.1, 0.5, 0.9], 256)	No warning
U5	levels=[0.2, 0.6], norm=BoundaryNorm([0.1, 0.5, 0.9], 256)	Warning issued
U6	levels=None, norm=LogNorm(0.1, 1.0)	(None, norm) (no extraction)
U7	levels=[0.5] (single level)	([0.5], None) (no BoundaryNorm)

4. Atomic Commits

1. test: add failing tests for levels/norm reconciliation
2. feat(hist2d): add _reconcile_levels_and_norm helper
3. refactor(hist2d): use _reconcile_levels_and_norm in plot_contours
4. refactor(hist2d): use _reconcile_levels_and_norm in plot_hist_with_contours
5. test: add regression tests for existing behavior
6. docs(hist2d): update docstrings for levels/norm interaction

[blalterman]

Root Cause Analysis: pcolormesh/contourf Color Scale Mismatch

Evidence-Based Root Cause

Code Flow (hist2d.py lines 580-672):

# Line 604: norm extracted from kwargs or default
norm = kwargs.pop("norm", default_norm)

# Line 615: pcolormesh uses norm for color mapping
pc = ax.pcolormesh(..., norm=norm, ...)

# Line 645: levels obtained from user or defaults
levels = self._get_contour_levels(levels)

# Line 670-672: contourf uses levels (positions) AND norm (colors)
qset = contour_fcn(*args, ..., norm=norm, ...)

Bug Mechanism:

Scenario	pcolormesh	contourf	Result
levels=[...], norm=None	Auto-scale from data	Use explicit levels, auto-scale colors	❌ MISMATCH
levels=[...], norm=BoundaryNorm(levels)	Use norm	Use levels + norm	✅ Consistent

When levels is passed without norm:

1. pcolormesh receives norm=None, auto-scales colors from data range
2. contourf receives explicit levels for positions but norm=None for colors
3. matplotlib creates independent color scales → visible discontinuity

Proposed Fix: _reconcile_levels_and_norm() Helper

def _reconcile_levels_and_norm(self, levels, norm):
    # Case 1: Extract levels from BoundaryNorm
    if levels is None and isinstance(norm, mpl.colors.BoundaryNorm):
        levels = list(norm.boundaries)

    # Case 2: Create BoundaryNorm from levels
    elif norm is None and levels is not None and len(levels) >= 2:
        norm = mpl.colors.BoundaryNorm(levels, 256, clip=True)

    # Case 3: Warn on mismatch
    elif levels is not None and isinstance(norm, mpl.colors.BoundaryNorm):
        if not np.allclose(levels, norm.boundaries):
            warnings.warn("levels/norm.boundaries mismatch", UserWarning)

    return levels, norm

Test Plan

Unit Tests (8): Input/output types, shapes, values for helper
Smoke Tests (7): All parameter combinations run without error
Integration Tests (5): pcolormesh.norm == contourf.norm after reconciliation

Atomic Commits

1. test: add failing tests for levels/norm reconciliation
2. feat(hist2d): add _reconcile_levels_and_norm helper
3. refactor(hist2d): use helper in plot_contours
4. refactor(hist2d): use helper in plot_hist_with_contours
5. test: add integration tests for color consistency
6. docs(hist2d): document levels/norm interaction

Immediate Workaround

Pass BOTH levels and matching norm:

levels = np.linspace(vmin, vmax, 17)
norm = mcolors.BoundaryNorm(levels, ncolors=256, extend="both")
hist.plot_hist_with_contours(norm=norm, levels=levels, ...)

Full analysis: docs/issues/hist2d_levels_norm_root_cause.md