Skip to content

fix: avoid okio resize overflow for files larger than 2 GB#140

Merged
linroid merged 1 commit into
mainfrom
fix/preallocate-large-files
May 20, 2026
Merged

fix: avoid okio resize overflow for files larger than 2 GB#140
linroid merged 1 commit into
mainfrom
fix/preallocate-large-files

Conversation

@linroid
Copy link
Copy Markdown
Owner

@linroid linroid commented May 20, 2026

Summary

Fixes a NegativeArraySizeException when starting a download larger than
2 GB (reported on a 2.72 GB Ubuntu server ISO — preallocate(2_918_598_656)).

Root cause

PathFileAccessor.preallocate() called okio's FileHandle.resize() to
grow the file to the target size. okio 3.16.4 (JvmFileHandle.kt:30)
implements grow as:

val delta = size - currentSize
if (delta > 0) {
  protectedWrite(currentSize, ByteArray(delta.toInt()), 0, delta.toInt())
}

For a 2.72 GB grow, delta.toInt() overflows to -1376368640, throwing
NegativeArraySizeException(-1376368640) — the exact value seen in the
crash log. Beyond the overflow, allocating a multi-GB byte array purely
to preallocate disk space is wasteful even when it doesn't crash.

iOS isn't affected — okio's UnixFileHandle.protectedResize uses
ftruncate(), which takes a Long and doesn't allocate.

Fix

Extend the file with a single sparse-byte write at (size - 1) when
growing — same approach already used in ContentUriFileAccessor.preallocate.
The filesystem allocates space lazily as ranges are written. Shrinks
continue to use resize() (okio's shrink path uses RandomAccessFile.setLength
and is safe).

Test Plan

  • New JVM regression test PathFileAccessorPreallocateTest:
    • preallocate_smallFile — sanity check
    • preallocate_above2GB_doesNotOverflow — fails on main, passes here
    • preallocate_zero_isNoOp
  • All existing library:core:jvmTest tests still pass
  • Multi-platform compile (iosArm64, js, wasmWasi) still works
  • Manual: download a > 2 GB file end-to-end

PathFileAccessor.preallocate() called okio's FileHandle.resize() to
grow the file to the target size. okio 3.16.4 implements that grow
path by allocating a ByteArray((size - current).toInt()) and writing
it whole, which overflows Int and throws NegativeArraySizeException
once the delta exceeds 2 GB. The reported failure was
preallocate(2_918_598_656) for an Ubuntu server ISO.

Switch to extending the file with a single sparse-byte write at
(size - 1) when growing, mirroring ContentUriFileAccessor's approach.
This also avoids the gratuitous multi-GB byte-array allocation on
the happy path. Shrinks still use resize() (okio's shrink path uses
RandomAccessFile.setLength and is safe).

Adds a regression test that preallocates ~2.72 GB and asserts the
file ends at the requested size.
@github-actions
Copy link
Copy Markdown
Contributor

Test Results

1 046 tests  +3   1 046 ✅ +3   14s ⏱️ -1s
   95 suites +1       0 💤 ±0 
   95 files   +1       0 ❌ ±0 

Results for commit 16c2ddb. ± Comparison against base commit 729b118.

@linroid linroid merged commit d4a3706 into main May 20, 2026
6 checks passed
@linroid linroid deleted the fix/preallocate-large-files branch May 20, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant