Testsuite crashes on sparc64 due to unaligned access (Bus error)

[glaubitz]

The testsuite for bitarray crashes on sparc64 due to unaligned access (Bus error):

Successfully built bitarray-3.7.2-cp313-cp313-linux_sparc64.whl
I: pybuild plugin_pyproject:144: Unpacking wheel built for python3.13 with "installer" module
I: pybuild pybuild:334: mkdir -p /build/reproducible-path/python-bitarray-3.7.2/debian/buildhtml/doc && cp /build/reproducible-path/python-bitarray-3.7.2/README.rst /build/reproducible-path/python-bitarray-3.7.2/CHANGE_LOG /build/reproducible-path/python-bitarray-3.7.2/update_doc.py /build/reproducible-path/python-bitarray-3.7.2/debian/buildhtml; cp /build/reproducible-path/python-bitarray-3.7.2/doc/*.rst /build/reproducible-path/python-bitarray-3.7.2/debian/buildhtml/doc; cd /build/reproducible-path/python-bitarray-3.7.2/debian/buildhtml; PYTHONPATH=/build/reproducible-path/python-bitarray-3.7.2/.pybuild/cpython3_3.13_bitarray/build python3.13 update_doc.py; pandoc -r gfm -w html5 -o README.html -s --toc --metadata title="BitArray Documentation" README.rst; mv /build/reproducible-path/python-bitarray-3.7.2/debian/buildhtml/README.html /build/reproducible-path/python-bitarray-3.7.2
already up-to-date
   dh_auto_test -a -O--buildsystem=pybuild
I: pybuild base:311: cd /build/reproducible-path/python-bitarray-3.7.2/.pybuild/cpython3_3.13_bitarray/build; python3.13 -c 'import bitarray; bitarray.test()'
..............Bus error
E: pybuild pybuild:389: test: plugin pyproject failed with: exit code=138: cd /build/reproducible-path/python-bitarray-3.7.2/.pybuild/cpython3_3.13_bitarray/build; {interpreter} -c 'import bitarray; bitarray.test()'
dh_auto_test: error: pybuild --test -i python{version} -p 3.13 returned exit code 13

Full build log available in: https://buildd.debian.org/status/fetch.php?pkg=python-bitarray&arch=sparc64&ver=3.7.2-1&stamp=1762556188&raw=0

This is usually easy to debug and fix with the help of GDB.

Access to a SPARC machine running both Solaris and Linux can be obtained through the GCC Compile Farm: https://gcc.gnu.org/wiki/CompileFarm

[ilanschnell]

Thank you for the bug report! I have some ideas what might cause the bus error. Unfortunately I don't have time at the moment to look into this more, but I hope to look into it in December.

[glaubitz]

I really appreciate the open and kind reply. Thank you!

Let me try whether I can get you a backtrace real quick which might help down the problem faster.

[glaubitz]

Here is a quick backtrace from the core dump:

(gdb) bt
#0  0xfff8000101103fd4 in builtin_bswap64 (word=<optimized out>) at bitarray/bitarray.h:206
#1  shift_r8le (buff=buff@entry=0xba9a7c "\006\352\027ZY\356\357\221\f\356\241\351F\233\312\360\037\217\375\377\377[ym&\317\353\216\361\006\305d", n=<optimized out>, 
    k=k@entry=5) at bitarray/_bitarray.c:230
#2  0xfff8000101106920 in shift_r8 (self=<optimized out>, a=<optimized out>, b=149, k=5) at bitarray/_bitarray.c:293
#3  0xfff800010110a2f0 in copy_n (self=0xfff8000102871ac0, a=101, other=0xfff800010100aba0, b=0, n=1086) at bitarray/_bitarray.c:349
#4  0xfff800010110ec40 in setslice_bitarray (self=0xfff8000102871ac0, slice=<optimized out>, other=0xfff800010100aba0) at bitarray/_bitarray.c:2262
#5  assign_slice (self=0xfff8000102871ac0, slice=<optimized out>, value=<optimized out>) at bitarray/_bitarray.c:2346
#6  bitarray_ass_subscr (self=0xfff8000102871ac0, item=<optimized out>, value=<optimized out>) at bitarray/_bitarray.c:2567
#7  0x000000000015fc58 in PyObject_SetItem ()
#8  0x00000000002ea520 in _PyEval_EvalFrameDefault ()
#9  0x0000000000181aac in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

The code in question is this one in shift_r8le():

#if HAVE_BUILTIN_BSWAP64 && PY_BIG_ENDIAN
        *p = builtin_bswap64(*p);
        *p <<= k;
        *p = builtin_bswap64(*p);
#else
        *p <<= k;
#endif

[glaubitz]

I managed to fix all cases of unaligned access. He is a very rough proof of concept patch:

Index: python-bitarray-3.7.2/bitarray/_bitarray.c
===================================================================
--- python-bitarray-3.7.2.orig/bitarray/_bitarray.c
+++ python-bitarray-3.7.2/bitarray/_bitarray.c
@@ -227,9 +227,12 @@ shift_r8le(unsigned char *buff, Py_ssize
     while (w--) {                 /* shift in word-range(0, w) */
         uint64_t *p = ((uint64_t *) buff) + w;
 #if HAVE_BUILTIN_BSWAP64 && PY_BIG_ENDIAN
-        *p = builtin_bswap64(*p);
-        *p <<= k;
-        *p = builtin_bswap64(*p);
+        uint64_t tmp;
+        memcpy(&tmp, p, sizeof(uint64_t));
+        tmp = builtin_bswap64(tmp);
+        tmp <<= k;
+        tmp = builtin_bswap64(tmp);
+        memcpy(p, &tmp, sizeof(uint64_t));
 #else
         *p <<= k;
 #endif
@@ -261,7 +264,10 @@ shift_r8be(unsigned char *buff, Py_ssize
         *p >>= k;
         *p = builtin_bswap64(*p);
 #else
-        *p >>= k;
+        uint64_t tmp;
+        memcpy(&tmp, p, sizeof(uint64_t));
+        tmp >>= k;
+        memcpy(p, &tmp, sizeof(uint64_t));
 #endif
         if (w)                    /* add shifted byte from next lower word */
             buff[8 * w] |= buff[8 * w - 1] << (8 - k);
Index: python-bitarray-3.7.2/bitarray/bitarray.h
===================================================================
--- python-bitarray-3.7.2.orig/bitarray/bitarray.h
+++ python-bitarray-3.7.2/bitarray/bitarray.h
@@ -273,8 +273,11 @@ popcnt_words(uint64_t *w, Py_ssize_t n)
     Py_ssize_t cnt = 0;
 
     assert(n >= 0 && ((uintptr_t) w) % 4 == 0);
-    while (n--)
-        cnt += popcnt_64(*w++);
+    while (n--) {
+        uintptr_t tmp;
+        memcpy(&tmp, w++, sizeof(uintptr_t));
+        cnt += popcnt_64(tmp);
+    }
     return cnt;
 }

With this patch, the package builds without any problems and passes the testsuite:

Successfully built bitarray-3.7.2-cp313-cp313-linux_sparc64.whl
I: pybuild plugin_pyproject:144: Unpacking wheel built for python3.13 with "installer" module
I: pybuild pybuild:334: mkdir -p /home/glaubitz/python-bitarray/python-bitarray-3.7.2/debian/buildhtml/doc && cp /home/glaubitz/python-bitarray/python-bitarray-3.7.2/README.rst /home/glaubitz/python-bitarray/python-bitarray-3.7.2/CHANGE_LOG /home/glaubitz/python-bitarray/python-bitarray-3.7.2/update_doc.py /home/glaubitz/python-bitarray/python-bitarray-3.7.2/debian/buildhtml; cp /home/glaubitz/python-bitarray/python-bitarray-3.7.2/doc/*.rst /home/glaubitz/python-bitarray/python-bitarray-3.7.2/debian/buildhtml/doc; cd /home/glaubitz/python-bitarray/python-bitarray-3.7.2/debian/buildhtml; PYTHONPATH=/home/glaubitz/python-bitarray/python-bitarray-3.7.2/.pybuild/cpython3_3.13_bitarray/build python3.13 update_doc.py; pandoc -r gfm -w html5 -o README.html -s --toc --metadata title="BitArray Documentation" README.rst; mv /home/glaubitz/python-bitarray/python-bitarray-3.7.2/debian/buildhtml/README.html /home/glaubitz/python-bitarray/python-bitarray-3.7.2
already up-to-date
   dh_auto_test -a -O--buildsystem=pybuild
I: pybuild base:311: cd /home/glaubitz/python-bitarray/python-bitarray-3.7.2/.pybuild/cpython3_3.13_bitarray/build; python3.13 -c 'import bitarray; bitarray.test()'
bitarray is installed in: /home/glaubitz/python-bitarray/python-bitarray-3.7.2/.pybuild/cpython3_3.13_bitarray/build/bitarray
bitarray version: 3.7.2
sys.version: 3.13.9 (main, Oct 15 2025, 14:56:22) [GCC 15.2.0]
sys.prefix: /usr
pointer size: 64 bit
sizeof(size_t): 8
sizeof(bitarrayobject): 80
HAVE_BUILTIN_BSWAP64: 1
default bit-endianness: big
machine byte-order: big
Py_DEBUG: 0
DEBUG: 1
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
----------------------------------------------------------------------
Ran 597 tests in 3.152s

OK

[ilanschnell]

Thank you @glaubitz! I'm wondering if it is possible to "pre-" align the memory in way that additional memcpy's are avoided. I thought I'm already doing this in shift_r8 by calling to_aligned. I remember fixing a similar issue this way.

[glaubitz]

Sorry for the late reply!

Thank you @glaubitz! I'm wondering if it is possible to "pre-" align the memory in way that additional memcpy's are avoided.

I know that C++ has aligned_alloc but I don't know if something similar exists in C.

I thought I'm already doing this in shift_r8 by calling to_aligned. I remember fixing a similar issue this way.

I can try to use this approach in the cases above as well.