Allows custom signatures for #[getter] and #[setter] in order to provide type hints

[varchasgopalaswamy]

Without the automatic type stub generation, it makes sense that the signature of getters and setters should not be overriden. However, with stub generation there can be a need to provide custom type hints. This PR relaxes the requirement that custom signatures not be allowed for getters and setters to enable this use case.

[Tpt]

Thank you for this!

Allowing signature might be slightly confusing to the user: the getter/setter are not exposed as method to cpython so things like argument name do not exist outside of stub generation. But indeed, it would be very convenient for stubs.

It might be nice to have some testing:

• add a class with such custom signature in pytests/src/pyclasses.rs to generate the stubs and check they are valid (run nox -s test-introspection to run the introspection tests)
• maybe make sure very wrong signatures (e.g. extra arguments) are still prevented by the macro with a compile error test in pyo3/tests/test_compile_error.rs

[Tpt]

Do we have a use case for text_signature? I don't think it's even possible to expose them to cpython

[varchasgopalaswamy]

Allowing signature might be slightly confusing to the user: the getter/setter are not exposed as method to cpython so things like argument name do not exist outside of stub generation. But indeed, it would be very convenient for stubs.

A fair point, but I guess the alternative would be another attribute macro? I thought about it, but I reasoned that the tradeoff is the mental load of remembering more attribute names would be worse.

I'll push up some tests when I get some more free time!

[Tpt]

I thought about it, but I reasoned that the tradeoff is the mental load of remembering more attribute names would be worse.

Yes! Agreed.

I'll push up some tests when I get some more free time!

Thank you!

[Icxolu]

I'm not sure we should so this if we can not make it work the same as for other functions as well. I feel like it would be very surprising if one for example specifies a signature with a default value which is then not respected, or overwriting a text_signature which is then not exposed to Python. Additionally these do not accept arbitrary signatures, so we would to cover that as well.

I feel like a general mechanism to override/adapt the type hints would be a better fit for this functionality. That would make it clear that it only affects type hint generation and not runtime behavior.

[varchasgopalaswamy]

@Icxolu that's also a fair point. But doesnt signature already check to make sure that it's consistent with the function it's applied to? Getters and setters do not have arguments other than self, so you should get a compiler error when that happens, no?

The cost of adding a new attribute would then be having multiple ways to specify the same thing, which I think is something python gets right in strongly recommending against. By this I mean that signature already allows one to specify type hints, so if we add e.g. hints, that's two ways to specify type hints.

• Should it only work on getters/setters? I think that would be quite counter intuitive
• Should it be mutually exclusive with signature? I think that's acceptable, if a little clunky
• Should it override/be overriden by signature? I think that's a bad idea
• Should signature not take type hints? I think that's also not great, since it's not how python would work and I think be annoying for people coming from python

[Icxolu]

But doesnt signature already check to make sure that it's consistent with the function it's applied to? Getters and setters do not have arguments other than self, so you should get a compiler error when that happens, no?

It does that for normal functions, but it might not do it for these, because it currently can't be placed there. Also a setter does have a second argument, namely the value to be set.

The cost of adding a new attribute would then be having multiple ways to specify the same thing, which I think is something python gets right in strongly recommending against. By this I mean that signature already allows one to specify type hints, so if we add e.g. hints, that's two ways to specify type hints.

I generally agree that we should not offer competing ways to do the same thing. These are all good questions and I don't have direct answer to which is best. I just think that we may need to explore some designs here first, especially since stub generation is still pretty experimental.

[davidhewitt]

What's the use case when this is needed?

For types exposed by PyO3, the type hints should hopefully be good enough. For custom types, maybe a FromPyObject or IntoPyObject implementation just needs the TYPE_HINT constant applied there? Doing it on the traits would hopefully make all functions have the right hint without needing to override their signatures one by one.

[Tpt]

For types exposed by PyO3, the type hints should hopefully be good enough

There is likely still a small need when the field type is something like Py<PyAny> even if it's actually only a very specific Python type. But 100% agreed that for most cases the built-in type hint generation should be enough.

An other option is to encourage users to use a "new type" approach, writing struct MyType(Py<PyAny>) and implement FromPyObject and IntoPyObject on them (or PyTypeCheck?), removing the need for this pattern. But it would be a bit inconsistent with regular functions that allow custom type annotations in their signatures.

[davidhewitt]

Agreed.

For what it's worth, I think I'm ok with having #[pyo3(signature)] permitted for #[getter] and #[setter], as long as we have full checks to enforce that the signature makes sense.

[varchasgopalaswamy]

What's the use case when this is needed?

My use case is exactly what @Tpt laid out. I'm returning a Py<PyAny> (actually, it's exactly what I asked a question about a month ago! #5675) that is only meant for the python side of the application to use. I recognize this is not a common use case and that it would be preferable in most cases to have pyo3 infer the type.

For what it's worth, I think I'm ok with having #[pyo3(signature)] permitted for #[getter] and #[setter], as long as we have full checks to enforce that the signature makes sense.

Great! I'll work on some tests for this over the next few weeks when I get some free time to do so.