Skip to content

fix, H5cout sp_t operator<< calloc overflow when dataspace has no hyperslab selection #210

@steven-varga

Description

@steven-varga

Problem

ASan reports `calloc parameters overflow: count * size (-4 * 8)` in `H5cout.hpp:93` triggered from `test/H5cout.cpp:15` (test case `operator<< for sp_t prints rank and dims`).

Root cause: `H5Sget_select_hyper_nblocks` returns `hssize_t` and yields `-1` when the dataspace has no hyperslab selection. The code declares:

```cpp
hsize_t nblocks = H5Sget_select_hyper_nblocks(id); // -1 wraps to UINT64_MAX
hsize_t ncoordinates = 2 * rank * nblocks; // 2 * 2 * UINT64_MAX wraps to -4 (as signed)
std::calloc(ncoordinates, sizeof(hsize_t)); // ASan: calloc overflow
```

Fix

  • Declare `nblocks` as `hssize_t` (keep the signed return type).
  • Guard the calloc allocation and block-iteration loop with `if (nblocks > 0)`.

Observed

CI run 25950680215 on staging — `asan / ubuntu-24.04 / clang-20` fails at the Test step.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions