Skip to content

Conversation

@bschoenmaeckers
Copy link
Member

@bschoenmaeckers bschoenmaeckers commented Oct 15, 2025

Adds a new constructor on PyBytes that allows creating a python bytes object using a std::io::Write writer. I've exposed the writer as &mut dyn Write dyn object so we do not have to expose the internal writer implementation. We can change this to &mut PyBytesWriter at a later stage if that seems necessary.

closes #4003

@bschoenmaeckers bschoenmaeckers force-pushed the byteswriter branch 5 times, most recently from de94a5a to 89a3e09 Compare October 17, 2025 13:20
@bschoenmaeckers
Copy link
Member Author

How can I make pyo3-ffi-check happy? It is complaining that PyBytesWriter does not exist on pre-1.15. This is expected but I cannot let it point to the existing _PyBytesWriter because the layout is different.

  error[E0412]: cannot find type `PyBytesWriter` in module `bindings`
    --> src\main.rs:70:5
     |
  70 |     pyo3_ffi_check_macro::for_all_structs!(check_struct);
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `bindings`

@bschoenmaeckers bschoenmaeckers force-pushed the byteswriter branch 3 times, most recently from 244df75 to 5318908 Compare January 16, 2026 14:04
@bschoenmaeckers bschoenmaeckers marked this pull request as ready for review January 16, 2026 14:43
@bschoenmaeckers bschoenmaeckers changed the title Add BytesWriter Add PyBytes::new_with_writer Jan 16, 2026
@bschoenmaeckers bschoenmaeckers force-pushed the byteswriter branch 8 times, most recently from 85b55d9 to 7bfa6f4 Compare January 16, 2026 16:25
@bschoenmaeckers
Copy link
Member Author

A possible future extension would be to implement std::io::Seek on PyBytesWriter or std::io::Write on Cursor<PyBytesWriter>

Copy link
Member

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice, thank you! It's cool to see these newer APIs improving ergonomics / solving cases which couldn't be done previously.

return crate::PyBytes_AsString(op);
}

opaque_struct!(pub PyBytesWriter);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably missing a cfg here?

Suggested change
opaque_struct!(pub PyBytesWriter);
#[cfg(Py_3_15)]
opaque_struct!(pub PyBytesWriter);

#[inline]
pub unsafe fn PyBytesWriter_Create(
size: crate::Py_ssize_t,
) -> *mut crate::PyBytesWriter {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, the compat APIs produce a PyBytesWriter too, which exists before 3.15.

Maybe we should have pyo3_ffi::compat::PyBytesWriter opaque struct defined for pre-3.15? (On 3.15 it can be a re-export.)

Copy link
Member Author

@bschoenmaeckers bschoenmaeckers Jan 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could not get ffi-check to pass, with a public exposed struct that does not have a definition in the generated binding. So my only working solution so far was to force it to ignore this struct on pre-3.15.

For reference this is the error

  error[E0412]: cannot find type `PyBytesWriter` in module `bindings`
    --> src\main.rs:70:5
     |
  70 |     pyo3_ffi_check_macro::for_all_structs!(check_struct);
     |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ not found in `bindings`

@bschoenmaeckers bschoenmaeckers force-pushed the byteswriter branch 3 times, most recently from 8b744fc to 07f9d6c Compare January 19, 2026 11:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support resize on PyBytes when initialised via with_new

2 participants