GFQL typed schema: Python dataclass/Pydantic adapters for GraphSchema

[lmeyerov]

Summary

Add Python-native typed model adapters for the Arrow-canonical public GraphSchema layer.

The core schema contract remains Arrow-native: GraphSchema.to_arrow() / future graph-schema wire envelopes are the source of truth for internals, wire transport, backend storage, and cross-language interoperability. Dataclasses and Pydantic models are client-side adapters that compile to/from Arrow-backed NodeType, EdgeType, and GraphSchema declarations.

Parent / Related

• Umbrella: [FEA] GFQL native type system: schemas, inference, validation, and Arrow representation #1046
• Local schema declarations: GFQL type system follow-on A: public declarative schema model + stable exports #1337 / feat(gfql): add public declarative schema model #1457
• Schema inference/bootstrap: GFQL type system follow-on B: schema inference API + typed topology extraction #1338
• Arrow boundary APIs: GFQL type system follow-on C: public schema-Arrow APIs + plottable boundary enforcement #1339
• Remote transport: GFQL remote: send bound typed GraphSchema with gfql_remote requests #1465
• Accessor/design receipt: feat(gfql): add public Plottable.schema accessor for typed-schema introspection #1632 / feat(gfql): add Plottable schema accessor #1637

Motivation

Python users often already have typed domain models, or want generated typed models after inspecting a dataset schema:

• Existing typed app: @dataclass class Cat: ... or class Cat(BaseModel): ... should produce a NodeType / GraphSchema.
• Existing dataset/remote schema: an Arrow-backed GraphSchema should generate dataclass/Pydantic wrappers for row/entity work.
• Heavy mypy/pyright/Pydantic users should not duplicate schema definitions by hand.

Design constraints

• Arrow remains canonical; Python models never replace the wire/backend contract.
• Model adapters must round-trip through NodeType / EdgeType / GraphSchema.
• Runtime-generated models are useful for notebooks and services; static typing needs optional emitted .py / .pyi artifacts.
• Pydantic validators are Python-side validation only unless they can be expressed in Arrow/Graphistry schema metadata.
• Ambiguous Python annotations should require explicit Arrow overrides rather than guessing.

Possible API shape

from dataclasses import dataclass
from graphistry.schema import GraphSchema, NodeType, EdgeType

@dataclass
class Cat:
    id: int
    name: str
    lives: int | None = None

CatType = NodeType.from_dataclass(Cat, labels=("Animal", "Cat"))
CatModel = CatType.to_dataclass()

schema = GraphSchema.from_models(node_models=[Cat], edge_models=[...])
models = schema.to_dataclasses()

Pydantic equivalents:

CatType = NodeType.from_pydantic(CatModel, labels=("Animal", "Cat"))
CatModel = CatType.to_pydantic()

In scope

• Dataclass -> NodeType / EdgeType adapters.
• Pydantic -> NodeType / EdgeType adapters.
• GraphSchema.from_models(...) convenience layer.
• NodeType / EdgeType / GraphSchema -> runtime dataclass generation.
• NodeType / EdgeType / GraphSchema -> runtime Pydantic model generation.
• Optional codegen plan for .py / .pyi artifacts for mypy/pyright users.
• Type mapping table and explicit loss/override policy.

Out of scope

• Making dataclasses/Pydantic the wire protocol.
• Runtime per-row conversion inside GFQL execution.
• Full Pydantic validator enforcement remotely.
• CSV inference itself; bootstrap/proposal flow remains GFQL type system follow-on B: schema inference API + typed topology extraction #1338.
• Remote gfql_remote() schema transport; tracked in GFQL remote: send bound typed GraphSchema with gfql_remote requests #1465.

Acceptance

1. Dataclass required/optional primitive fields map deterministically to Arrow field type/nullability.
2. Pydantic required/optional primitive fields map to the same Arrow schema as equivalent dataclasses.
3. Field aliases/metadata can override Arrow field name, dtype, nullability, and Graphistry metadata.
4. Ambiguous annotations fail with clear override guidance.
5. Arrow schema -> model -> Arrow round trip preserves field names, Arrow types, nullability, and Graphistry metadata where representable.
6. GraphSchema with Cat/Dog/Car node models and shared Animal label preserves NodeType.name as type identity and labels as GFQL predicates.
7. Docs state that Python models are adapters around the Arrow-canonical graph contract.

Notes

Relevant current docs/APIs:

• Python stdlib supports runtime dataclass generation via dataclasses.make_dataclass.
• Pydantic v2 supports runtime model generation via create_model, validated construction via model_validate, and trusted construction via model_construct.
• Static typing users need emitted modules/stubs for full mypy/pyright visibility; runtime-generated classes alone are not enough for CI type checking.

[lmeyerov]

Closing this as superseded by #1643.

Reason: the dataclass/Pydantic adapter design should be bundled with the typed dataframe/entity façade work, not handled as a standalone lane. The adapters are useful because they feed the ORM-like Python user experience: typed table views, explicit type narrowing, and typed extracted entities. Splitting them too early risks over-designing adapters without the dataframe/entity workflow they need to serve.

Carry-forward issue:

• GFQL typed schema: Python adapters + typed dataframe/entity façades #1643 — Python adapters + typed dataframe/entity façades over Arrow-canonical GraphSchema