Add density mode to render_points#679
Merged
Merged
Conversation
Surfaces a discoverable density visualization on top of the existing datashader plumbing: `density=True` switches `render_points` to a 2-D count aggregation, with `density_how` controlling the count-to-color mapping (linear, log, cbrt, eq_hist). The aggregation primitives already existed -- `cvs.points(..., agg=ds.count())` for plain density and `ds.by(col, ds.count())` for per-category density -- they were just unreachable without an explicit user-facing toggle. This change wires them up and exposes `how=` through the shade helpers. Under `density=True`, `method` is forced to `"datashader"`; passing `method="matplotlib"` raises. Continuous or literal colors are rejected (the validator catches the literal-color case early; the render path catches continuous-color before `.compute()` materializes the data). `size`, `transfunc`, `norm.vmin/vmax`, and `datashader_reduction` are warn-and-ignored because they do not interact meaningfully with a count-based density.
The shared blobs fixture only has ~200 points scattered across a 500x500 canvas, so density renders as nearly invisible single-pixel dots. Add a dedicated sdata_dense_points fixture (~60k points across three Gaussian clusters, with a categorical "gene" column) so the test_plot_density_* cases produce a meaningful visual baseline. Also fixes test_plot_density_categorical_with_groups, which previously passed palette="tab20" alongside a single-element groups list -- our palette parser treats single-string + single-group as a literal color and rejected "tab20" with "Unknown color".
Generated from CI artifact for hatch-test.py3.11-stable on the sdata_dense_points fixture.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #679 +/- ##
==========================================
+ Coverage 77.71% 77.79% +0.08%
==========================================
Files 11 11
Lines 3652 3693 +41
Branches 863 877 +14
==========================================
+ Hits 2838 2873 +35
- Misses 487 490 +3
- Partials 327 330 +3
🚀 New features to boost your workflow:
|
For ``color=None``, the dispatch in ``_render_points`` was sending the
aggregate to ``_ds_shade_categorical``, which uses ``color_vector[0]``
as a single hex color and only modulates alpha across counts. That
collapses any sequential colormap (``viridis`` and friends) to a flat
hue and makes a "density" plot look like a uniform smudge with low
opacity.
Plain density now routes through ``_ds_shade_continuous`` so the
configured cmap is used as a real intensity gradient over the count
aggregate. ``density_how`` ("linear" / "log" / "cbrt" / "eq_hist")
finally produces visibly different results.
The categorical density branch is unchanged: per-category color keys
still pass through ``_ds_shade_categorical`` and modulate alpha per
gene, which is the correct behaviour for "where does each category
concentrate".
Regenerated from CI after routing plain density through the continuous shade path so the viridis cmap actually produces a gradient over counts (previously alpha-only on a single hue).
For categorical density the only signal available to encode count is per-pixel alpha (hue is fixed per category by the color key). The existing min_alpha of ~254 (set so scatter plots render every point at full opacity) clamps that signal to a single value, so all three gene clusters were rendering as flat, uniform blobs. Under density=True the categorical shade path now uses min_alpha=40 (~15%): low enough to give the count-driven alpha real range, high enough to keep sparse-edge pixels visible under density_how="linear". Plain density (continuous shade path) is unaffected -- its viridis gradient already encodes count via hue.
Regenerated from CI after dropping the min_alpha floor for density: each gene cluster now shows a real opacity gradient (darker centers, lighter edges) instead of a flat hue cloud.
Consolidate 8 single-assertion unit tests into 2 parametrized ones (rejections and warn-and-ignores) and drop 3 low-value cases (forces_datashader, chaining_composes, default_path_unchanged) that either duplicate other tests or assert tautologies. Folds the "no warnings at default" check into the combined defaults test. Drop two redundant visual tests: density_how_log (linear and eq_hist already bracket the intensity-mapping range) and density_categorical_with_groups (groups filtering shares no code with density; covered elsewhere). Down from 12 + 5 to 4 + 3 tests with the same coverage of decision points.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
density: boolanddensity_how: Literal["linear","log","cbrt","eq_hist"]torender_pointsfor 2-D count density via datashader.colorproduces per-category density viads.by(col, ds.count()); no color produces plainds.count().how=through the shade helpers;density=Trueforcesmethod="datashader". Continuous or literal colors are rejected;size,transfunc,norm.vmin/vmax, anddatashader_reductionare warn-and-ignored under density.Why this design
The aggregation primitives already existed in
_datashader.py— they were just unreachable without an explicit user-facing toggle. This change exposes them as a single boolean (density) plus one knob for the count-to-color mapping (density_how). Defaultdensity_how="linear"keeps the colorbar axis as a count, which is the intuitive first-time-user expectation;log/cbrt/eq_histare available for dynamic-range compression.API