ServiceOptions (and maybe others?) Extensions not serialized in descriptor correctly

[chriselion]

Summary

I've only tried this for ServiceOptions, so not sure if it's specific to them, or a general problem. I noticed a difference in how they behave compared to "vanilla" protobuf.

Reproduction Steps

Minimal repro:

# in option_example.proto
syntax = "proto3";

package betterproto.repro;

import "google/protobuf/descriptor.proto";

message ExampleOptions {
  string foo = 1 [targets = TARGET_TYPE_SERVICE];
}

extend google.protobuf.ServiceOptions {
  optional ExampleOptions example_options = 50000;
}

service MyService {
    option (example_options).foo = "http://localhost/";
}

Compiled with

$ uv run python -m grpc_tools.protoc --python_betterproto2_out generated/ --python_betterproto2_opt=google_protobuf_descriptors -I. option_example.proto

In an interpreter:

>>> from generated.google import protobuf  # work around issue 150
>>> from generated.betterproto.repro import OPTION_EXAMPLE_PROTO_DESCRIPTOR
>>> len(OPTION_EXAMPLE_PROTO_DESCRIPTOR.services_by_name["MyService"].GetOptions().Extensions)
0
>>> OPTION_EXAMPLE_PROTO_DESCRIPTOR.services_by_name["MyService"].GetOptions()

>>> OPTION_EXAMPLE_PROTO_DESCRIPTOR.services_by_name["MyService"].GetOptions().SerializeToString()
b'\x82\xb5\x18\x13\n\x11http://localhost/'

so there's something there (based on the SerializeToString() output), but I think it's in an unknown field.

Expected Results

Compiling to "vanilla" protobuf:

uv run python -m grpc_tools.protoc --python_out generated_pb2/ --grpc_python_out generated_pb2/ -I. option_example.proto

and exploring in the interpreter:

>>> from generated_pb2.option_example_pb2 import DESCRIPTOR
>>> len(DESCRIPTOR.services_by_name["MyService"].GetOptions().Extensions)
1
>>> DESCRIPTOR.services_by_name["MyService"].GetOptions()
[betterproto.repro.example_options] {
  foo: "http://localhost/"
}

>>> DESCRIPTOR.services_by_name["MyService"].GetOptions().SerializeToString()
b'\x82\xb5\x18\x13\n\x11http://localhost/'

So the contents of SerializeToString() look identical, but the Extensions are non-empty, and GetOptions() shows the contents correctly.

Actual Results

The loaded Extensions should be non-empty to that I can retrieve the values.

System Information

$ uv run python -m grpc_tools.protoc --version                                                                                                   
libprotoc 31.1

$ uv run python --version                     
Python 3.12.11

$ uv pip show betterproto2 
Name: betterproto2
Version: 0.9.0
Location: [snip]/.venv/lib/python3.12/site-packages
Requires: python-dateutil, typing-extensions
Required-by: betterproto2-compiler

$ uv pip show betterproto2_compiler
Name: betterproto2-compiler
Version: 0.9.0
Location: [snip]/.venv/lib/python3.12/site-packages
Requires: betterproto2, jinja2, ruff, typing-extensions
Required-by:

$ uv pip show protobuf
Name: protobuf
Version: 6.32.1
Location: [snip]/.venv/lib/python3.12/site-packages
Requires:
Required-by: grpcio-tools

Checklist

• I have searched the issues for duplicates.
• I have shown the entire traceback, if possible.
• I have verified this issue occurs on the latest prelease of betterproto which can be installed using pip install -U --pre betterproto, if possible.

[chriselion]

I spent some time digging into this; it's not due to the serialized Descriptor directly, but how it's registered with the DescriptorPool(s).

Here's a small test case:

from generated.google import protobuf
from generated.betterproto.repro import OPTION_EXAMPLE_PROTO_DESCRIPTOR

from google.protobuf import descriptor_pool as _descriptor_pool
from google.protobuf import descriptor_pb2 as google_dot_protobuf_dot_descriptor__pb2

# Also register the descriptor with protobuf's default pool
# Without this line, the assert will fail below.
_descriptor_pool.Default().AddSerializedFile(OPTION_EXAMPLE_PROTO_DESCRIPTOR.serialized_pb)

sd = OPTION_EXAMPLE_PROTO_DESCRIPTOR.services_by_name["MyService"]
print(sd.GetOptions())
assert len(sd.GetOptions().Extensions)

If I register the descriptor with the default protobuf DescriptorPool (in addition it getting registered with betterproto's pool), the Extensions are visible. But if I don't do that, they're not.

So this is probably less of a bug with the generated betterproto2 code, and more of a flaw in how it interacts with the protobuf library.

My plan to work around this is to use protobuf for codegen, but still use betterproto for the actual client/server stub code.