NaNs Are Returned by non NaN Downsamplers

[mike-iqmo]

When using the non NaN version of the samplers, I am getting NaNs in the downsampled data.

My understanding from the documentation was that MinMaxLTTBDownsampler would omit all NaN values.

Some code demonstrating this below

n=10_000
y = np.arange(n, dtype=np.float64)
for i in range(1,100):
    y[i+100] = np.nan

sampled=MinMaxLTTBDownsampler().downsample(y,n_out=1000)
print(f"MinMaxLTTBDownsampler:{[i for i in sampled if np.isnan(y[i])]}")

sampled_nan=NaNMinMaxLTTBDownsampler().downsample(y,n_out=1000)
print(f"NaNMinMaxLTTBDownsampler:{[i for i in sampled_nan if np.isnan(y[i])]}")

That will print

MinMaxLTTBDownsampler:[101, 111, 121, 131, 141, 151, 161, 171, 181, 191]
NaNMinMaxLTTBDownsampler:[101, 111, 121, 131, 141, 151, 161, 171, 181, 191]

[jvdd]

Good catch! The code didn't handle the case where the bin is full of NaNs (as is the case in your example).

#83 now prunes NaN indices after downsampling (in Rust) for the omit NaN downsamplers