docs(types): document Field(exclude=True) and @model_serializer for nested wire isolation (#630)

bokelley · claude · web-flow · commit 4912af9f3f3f · 2026-05-10T12:01:57.000-04:00
* docs(types): document Field(exclude=True) and @model_serializer for nested wire isolation Pydantic v2's Rust-backed serializer does not call Python model_dump() overrides on nested child instances — a gap that was driving adopters to write mechanical parent-level dispatch boilerplate (~59 overrides in at least one downstream integration). Neither Field(exclude=True) nor @model_serializer(mode='wrap') require parent-level workarounds; both work correctly at every nesting depth via Pydantic's own pipeline. This commit: - Adds a top-level serialization note to extending-types.md explaining the Pydantic v2 nesting behavior - Adds a "Field(exclude=True) — Recommended" section with a working nested example - Adds a "@model_serializer" section for custom Python-level logic - Adds a migration guide replacing the 59-style parent-dispatch pattern - Adds a serialize_as_any note for runtime-type field inclusion - Updates Best Practices #1 to prefer Field(exclude=True) over call-site exclude={} - Adds a one-line comment on AdCPBaseModel.model_dump() cross-referencing the doc so source readers hit the explanation at the declaration https://claude.ai/code/session_01P7MQW9tW7z4rYm13zghrVC * docs(types): fix pre-PR review blockers in extending-types.md - @model_serializer: correct nesting claim — serializer only fires when serialized directly or when parent calls model_dump(serialize_as_any=True); update example to use user-defined source_label field (no non-existent render_url), add serialize_as_any=True demonstration - Field(exclude=True): fix import (adcp.types.base not adcp.types), switch example to Creative/CreativePayload (verifiable fields), remove invalid Product constructor that would raise ValidationError - Migration section: rename GetCreativesResponse (non-existent) to MyCreativePayload - Best Practices #1: fix CreateMediaBuySuccess → CreateMediaBuySuccessResponse (correct export name), add imports https://claude.ai/code/session_01P7MQW9tW7z4rYm13zghrVC * docs(types): fix pre-existing wrong type names throughout extending-types.md CreateMediaBuySuccess is not exported from adcp — the correct name is CreateMediaBuySuccessResponse. WebhookPayload is also not exported — McpWebhookPayload is the correct replacement. All 10+ import and usage sites in the pre-existing patterns corrected; class name suffixes (CreateMediaBuySuccessExtended) preserved. https://claude.ai/code/session_01P7MQW9tW7z4rYm13zghrVC --------- Co-authored-by: Claude <noreply@anthropic.com>
diff --git a/docs/extending-types.md b/docs/extending-types.md
@@ -4,13 +4,161 @@ ADCP types represent the standardized protocol schema. However, your implementat
 
 This guide shows how to extend ADCP types safely while maintaining protocol compliance.
 
+> **Pydantic v2 serialization note:** Pydantic v2 uses a Rust-backed serializer. When a parent
+> model calls `model_dump()`, Pydantic serializes nested child instances using its own compiled
+> pipeline — it does **not** call Python-level `model_dump()` overrides on child objects. This
+> means that if you override `model_dump()` on a child class, that override will not fire when
+> the child is serialized as part of a parent. Use `Field(exclude=True)` for field-level
+> exclusion (works automatically at every nesting depth) or `@model_serializer` with
+> `serialize_as_any=True` for custom Python logic (covered below).
+
+## Field-Level Exclusion with `Field(exclude=True)` — Recommended
+
+The simplest and most reliable way to keep internal fields off the wire. Fields annotated with
+`Field(exclude=True)` are excluded by Pydantic's own serializer at **every nesting depth** — no
+call-site `exclude={}` plumbing, no parent-model override required.
+
+```python
+from typing import Any
+from pydantic import Field
+from adcp import Creative
+from adcp.types.base import AdCPBaseModel
+
+
+class InternalCreative(Creative):
+    """Creative extended with seller-internal fields."""
+    internal_approval_id: str | None = Field(default=None, exclude=True)
+    seller_notes: str | None = Field(default=None, exclude=True)
+
+
+class CreativePayload(AdCPBaseModel):
+    """User-defined payload — creatives declared as base type."""
+    creatives: list[Creative]
+
+
+resp = CreativePayload(
+    creatives=[
+        InternalCreative(
+            creative_id="c-1",
+            variants=[],
+            internal_approval_id="approv-42",
+            seller_notes="approved by legal",
+        )
+    ]
+)
+
+wire = resp.model_dump()
+# {"creatives": [{"creative_id": "c-1", "variants": []}]}
+# internal_approval_id and seller_notes are absent — no parent override needed.
+```
+
+`Field(exclude=True)` works with `model_dump()`, `model_dump_json()`, and all standard Pydantic
+serialization options including `exclude_none=True`.
+
+## Custom Serialization Logic with `@model_serializer`
+
+For cases where you need Python-level transformation logic beyond field exclusion — reshaping
+output, conditional inclusion, derived computed fields — use Pydantic's
+`@model_serializer(mode='wrap')`.
+
+**Important:** When a parent field is annotated as the base type (`creatives: list[Creative]`),
+Pydantic's Rust serializer uses the declared type's schema and the subclass `@model_serializer`
+does **not** fire automatically. You must pass `serialize_as_any=True` to the parent's
+`model_dump()` call to apply subclass serializers from a parent. If you control the field
+annotation you can also declare it as the concrete subclass type.
+
+```python
+from typing import Any
+from pydantic import SerializationInfo, model_serializer
+from adcp import Creative
+from adcp.types.base import AdCPBaseModel
+
+
+class InternalCreative(Creative):
+    """Creative with a normalized source_label field."""
+    source_label: str | None = None
+
+    @model_serializer(mode="wrap")
+    def _serialize(self, handler: Any, info: SerializationInfo) -> dict[str, Any]:
+        result = handler(self, info)
+        # Normalize source_label to lowercase before it hits the wire.
+        if result.get("source_label"):
+            result["source_label"] = result["source_label"].lower()
+        return result
+
+
+# Direct serialization: serializer fires.
+c = InternalCreative(creative_id="c-1", variants=[], source_label="HD_VIDEO")
+c.model_dump()  # {"creative_id": "c-1", "variants": [], "source_label": "hd_video"}
+
+# Nested in a parent with a base-type annotation:
+class CreativePayload(AdCPBaseModel):
+    creatives: list[Creative]  # declared as base type
+
+payload = CreativePayload(creatives=[c])
+payload.model_dump()
+# {"creatives": [{"creative_id": "c-1", "variants": []}]}
+# source_label absent, serializer skipped — Pydantic uses Creative's declared schema.
+
+payload.model_dump(serialize_as_any=True)
+# {"creatives": [{"creative_id": "c-1", "variants": [], "source_label": "hd_video"}]}
+# serializer fired, source_label present and normalized.
+```
+
+## Migrating from Manual `model_dump()` Dispatch Overrides
+
+A common pattern in early SDK integrations is writing a parent override that manually re-calls
+`model_dump()` on each child list:
+
+```python
+# ❌ Fragile: every new response type needs this boilerplate, and missing one is silent.
+class MyCreativePayload(AdCPBaseModel):
+    creatives: list[Creative]
+
+    def model_dump(self, **kwargs: Any) -> dict[str, Any]:
+        result = super().model_dump(**kwargs)
+        if "creatives" in result and self.creatives:
+            result["creatives"] = [c.model_dump(**kwargs) for c in self.creatives]
+        return result
+```
+
+This requires ~10 lines per response type, must be written for every parent, and silently
+produces wrong output if a new child list field is added without updating the override.
+
+**Migration — field exclusion only:**
+
+```python
+# ✅ Delete the parent override entirely. Move exclusion to the child via Field(exclude=True).
+class InternalCreative(Creative):
+    internal_approval_id: str | None = Field(default=None, exclude=True)
+
+# MyCreativePayload needs no model_dump() override — Pydantic handles it at all depths.
+```
+
+**Migration — custom Python logic:**
+
+```python
+# ✅ Move the logic to the child via @model_serializer.
+#    Call model_dump(serialize_as_any=True) on the parent to apply it.
+class InternalCreative(Creative):
+    @model_serializer(mode="wrap")
+    def _serialize(self, handler: Any, info: SerializationInfo) -> dict[str, Any]:
+        result = handler(self, info)
+        # ... custom logic here ...
+        return result
+
+# Parent: no model_dump() override; caller passes serialize_as_any=True.
+payload = MyCreativePayload(creatives=[InternalCreative(creative_id="c-1", variants=[])])
+wire = payload.model_dump(serialize_as_any=True)
+```
+
 ## Basic Pattern: Subclassing Response Types
 
 ```python
-from adcp import CreateMediaBuySuccess
+from adcp import CreateMediaBuySuccessResponse
 from pydantic import ConfigDict, Field
 
-class CreateMediaBuySuccessExtended(CreateMediaBuySuccess):
+class CreateMediaBuySuccessExtended(CreateMediaBuySuccessResponse):
     """Extended with internal tracking fields."""
     workflow_step_id: str | None = Field(None, description="Internal workflow step ID")
     created_at: str | None = Field(None, description="Internal timestamp")
@@ -31,7 +179,7 @@ internal_response = CreateMediaBuySuccessExtended(
 )
 
 # Serialize to ADCP spec before sending over wire
-adcp_response = CreateMediaBuySuccess.model_validate(
+adcp_response = CreateMediaBuySuccessResponse.model_validate(
     internal_response.model_dump(exclude={'workflow_step_id', 'created_at', 'internal_notes'})
 )
 ```
@@ -57,10 +205,10 @@ class InternalResponseWrapper(BaseModel, Generic[T]):
     model_config = ConfigDict(extra='allow')
 
 # Usage
-from adcp import CreateMediaBuySuccess
+from adcp import CreateMediaBuySuccessResponse
 
-wrapper = InternalResponseWrapper[CreateMediaBuySuccess](
-    response=CreateMediaBuySuccess(
+wrapper = InternalResponseWrapper[CreateMediaBuySuccessResponse](
+    response=CreateMediaBuySuccessResponse(
         media_buy_id="mb_123",
         buyer_ref="ref_456",
         packages=[]
@@ -70,7 +218,7 @@ wrapper = InternalResponseWrapper[CreateMediaBuySuccess](
 )
 
 # Access ADCP response
-adcp_response = wrapper.response  # Type: CreateMediaBuySuccess
+adcp_response = wrapper.response  # Type: CreateMediaBuySuccessResponse
 
 # Access internal fields
 workflow_id = wrapper.workflow_step_id
@@ -82,7 +230,7 @@ When storing responses in a database with internal metadata:
 
 ```python
 from datetime import datetime
-from adcp import CreateMediaBuySuccess
+from adcp import CreateMediaBuySuccessResponse
 
 class MediaBuyRecord(BaseModel):
     """Database record combining ADCP response with internal metadata."""
@@ -94,12 +242,12 @@ class MediaBuyRecord(BaseModel):
     workflow_step_id: str
 
     # ADCP response (stored as JSON)
-    response_data: CreateMediaBuySuccess
+    response_data: CreateMediaBuySuccessResponse
 
     @classmethod
     def from_response(
         cls,
-        response: CreateMediaBuySuccess,
+        response: CreateMediaBuySuccessResponse,
         user_id: str,
         workflow_step_id: str
     ) -> "MediaBuyRecord":
@@ -113,13 +261,13 @@ class MediaBuyRecord(BaseModel):
             response_data=response
         )
 
-    def to_adcp_response(self) -> CreateMediaBuySuccess:
+    def to_adcp_response(self) -> CreateMediaBuySuccessResponse:
         """Extract ADCP response for wire protocol."""
         return self.response_data
 
 # Usage
 response = await client.create_media_buy(request)
-if isinstance(response, CreateMediaBuySuccess):
+if isinstance(response, CreateMediaBuySuccessResponse):
     record = MediaBuyRecord.from_response(
         response,
         user_id="user_123",
@@ -136,10 +284,10 @@ adcp_response = record.to_adcp_response()
 When processing webhook payloads with internal routing metadata:
 
 ```python
-from adcp import WebhookPayload
+from adcp import McpMcpWebhookPayload
 from pydantic import ConfigDict
 
-class InternalWebhookPayload(WebhookPayload):
+class InternalWebhookPayload(McpWebhookPayload):
     """Extended webhook payload with internal routing."""
     internal_destination: str | None = None
     retry_count: int = 0
@@ -150,7 +298,7 @@ class InternalWebhookPayload(WebhookPayload):
 async def process_webhook(payload: dict) -> None:
     """Process webhook with internal tracking."""
     # Parse with extensions
-    internal_payload = InternalWebhookPayload.model_validate(payload)
+    internal_payload = InternalMcpWebhookPayload.model_validate(payload)
 
     # Add internal routing
     internal_payload.internal_destination = determine_destination(internal_payload)
@@ -160,7 +308,7 @@ async def process_webhook(payload: dict) -> None:
     await route_to_handler(internal_payload)
 
     # When forwarding to another service, use base type
-    external_payload = WebhookPayload.model_validate(
+    external_payload = McpWebhookPayload.model_validate(
         internal_payload.model_dump(exclude={'internal_destination', 'retry_count', 'routing_key'})
     )
 ```
@@ -211,29 +359,38 @@ response = await client.create_media_buy(internal_request.to_adcp_request())
 
 ### 1. Always Use Field Exclusion for Wire Protocol
 
-**Don't** rely on serialization settings to exclude internal fields automatically:
+**Prefer `Field(exclude=True)` over call-site `model_dump(exclude={...})`.** `Field(exclude=True)` is declared once on the field, works at every nesting depth automatically, and cannot be forgotten at a call site.
 
 ```python
+from adcp import CreateMediaBuySuccessResponse
+from pydantic import Field
+
 # ❌ BAD: Relying on field name conventions
-class Extended(CreateMediaBuySuccess):
-    _internal_id: str  # Private field - might not serialize correctly
+class Extended(CreateMediaBuySuccessResponse):
+    _internal_id: str  # Private field — may or may not serialize correctly
 
-# ✅ GOOD: Explicit exclusion
-class Extended(CreateMediaBuySuccess):
+# ⚠ OK but fragile: call-site exclusion must be repeated every time model_dump() is called
+class Extended(CreateMediaBuySuccessResponse):
     internal_id: str
 
-adcp_response = CreateMediaBuySuccess.model_validate(
-    extended.model_dump(exclude={'internal_id'})
+adcp_response = CreateMediaBuySuccessResponse.model_validate(
+    extended.model_dump(exclude={"internal_id"})  # Easy to forget, silent if omitted
 )
+
+# ✅ BEST: Field-level exclusion fires automatically at all nesting depths
+class Extended(CreateMediaBuySuccessResponse):
+    internal_id: str = Field(exclude=True)
+
+adcp_response = extended.model_dump()  # internal_id is absent — no extra plumbing
 ```
 
 ### 2. Document Internal Fields
 
 Make it clear which fields are internal:
 
 ```python
-class Extended(CreateMediaBuySuccess):
-    """Extended CreateMediaBuySuccess with internal tracking.
+class Extended(CreateMediaBuySuccessResponse):
+    """Extended CreateMediaBuySuccessResponse with internal tracking.
 
     Internal fields (not part of ADCP spec):
         workflow_step_id: Internal workflow tracking
@@ -257,7 +414,7 @@ def test_internal_fields_excluded():
     )
 
     # Convert to wire protocol
-    adcp_response = CreateMediaBuySuccess.model_validate(
+    adcp_response = CreateMediaBuySuccessResponse.model_validate(
         extended.model_dump(exclude={'workflow_step_id'})
     )
 
@@ -273,7 +430,7 @@ def test_internal_fields_excluded():
 from typing import TypeGuard
 
 def is_extended_response(
-    response: CreateMediaBuySuccess
+    response: CreateMediaBuySuccessResponse
 ) -> TypeGuard[CreateMediaBuySuccessExtended]:
     """Check if response has extended internal fields."""
     return isinstance(response, CreateMediaBuySuccessExtended)
@@ -291,16 +448,16 @@ Define reusable field sets for exclusion:
 ```python
 from typing import ClassVar
 
-class CreateMediaBuySuccessExtended(CreateMediaBuySuccess):
+class CreateMediaBuySuccessExtended(CreateMediaBuySuccessResponse):
     workflow_step_id: str | None = None
     created_at: str | None = None
 
     # Define internal fields as class variable
     INTERNAL_FIELDS: ClassVar[set[str]] = {'workflow_step_id', 'created_at'}
 
-    def to_adcp_response(self) -> CreateMediaBuySuccess:
+    def to_adcp_response(self) -> CreateMediaBuySuccessResponse:
         """Convert to wire protocol, excluding internal fields."""
-        return CreateMediaBuySuccess.model_validate(
+        return CreateMediaBuySuccessResponse.model_validate(
             self.model_dump(exclude=self.INTERNAL_FIELDS)
         )
 ```
@@ -346,7 +503,7 @@ def test_roundtrip():
     )
 
     # Convert to base type
-    base = CreateMediaBuySuccess.model_validate(
+    base = CreateMediaBuySuccessResponse.model_validate(
         extended.model_dump(exclude={'workflow_step_id'})
     )
 
@@ -356,7 +513,7 @@ def test_roundtrip():
 
     # Verify can parse from wire format
     wire_format = base.model_dump_json()
-    parsed = CreateMediaBuySuccess.model_validate_json(wire_format)
+    parsed = CreateMediaBuySuccessResponse.model_validate_json(wire_format)
     assert parsed.media_buy_id == "mb_123"
 ```
 
diff --git a/src/adcp/types/base.py b/src/adcp/types/base.py
@@ -232,6 +232,12 @@ class AdCPBaseModel(BaseModel):
     model_config = ConfigDict(extra=_EXTRA_POLICY)
 
     def model_dump(self, **kwargs: Any) -> dict[str, Any]:
+        # NOTE: Pydantic v2 uses a Rust-backed serializer that does NOT call Python-level
+        # model_dump() overrides on nested child instances. If a child class overrides
+        # model_dump() for custom serialization logic, that override will not fire when
+        # the child is serialized as part of a parent model_dump() call. Use
+        # Field(exclude=True) for field-level exclusion (works at all nesting depths) or
+        # @model_serializer for custom output logic. See docs/extending-types.md.
         if "exclude_none" not in kwargs:
             kwargs["exclude_none"] = True
         return super().model_dump(**kwargs)
