Skip to content

Commit c953930

Browse files
timsaucerclaude
andcommitted
Move FFI Types section alongside other areas to check
Section 7 (FFI Types) was incorrectly placed after the Output Format and Implementation Pattern sections. Move it to sit after Section 6 (SessionContext Methods), consistent with the other checkable areas. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 31fa6d7 commit c953930

File tree

1 file changed

+51
-51
lines changed

1 file changed

+51
-51
lines changed

.claude/skills/check-upstream/SKILL.md

Lines changed: 51 additions & 51 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,57 @@ The user may specify an area via `$ARGUMENTS`. If no area is specified or "all"
103103
3. Also check `crates/core/src/context.rs` for what's implemented
104104
4. Report SessionContext methods that exist upstream but are missing
105105

106+
### 7. FFI Types (datafusion-ffi)
107+
108+
**Upstream source of truth:**
109+
- Crate source: https://github.com/apache/datafusion/tree/main/datafusion/ffi/src
110+
- Rust docs: https://docs.rs/datafusion-ffi/latest/datafusion_ffi/
111+
112+
**Where they are exposed in this project:**
113+
- Rust bindings: various files under `crates/core/src/` and `crates/util/src/`
114+
- FFI example: `examples/datafusion-ffi-example/src/`
115+
- Dependency declared in root `Cargo.toml` and `crates/core/Cargo.toml`
116+
117+
**Currently supported FFI types:**
118+
- `FFI_ScalarUDF``crates/core/src/udf.rs`
119+
- `FFI_AggregateUDF``crates/core/src/udaf.rs`
120+
- `FFI_WindowUDF``crates/core/src/udwf.rs`
121+
- `FFI_TableFunction``crates/core/src/udtf.rs`
122+
- `FFI_TableProvider``crates/core/src/table.rs`, `crates/util/src/lib.rs`
123+
- `FFI_TableProviderFactory``crates/core/src/context.rs`
124+
- `FFI_CatalogProvider``crates/core/src/catalog.rs`, `crates/core/src/context.rs`
125+
- `FFI_CatalogProviderList``crates/core/src/context.rs`
126+
- `FFI_SchemaProvider``crates/core/src/catalog.rs`
127+
- `FFI_LogicalExtensionCodec` — multiple files
128+
- `FFI_ExtensionOptions``crates/core/src/context.rs`
129+
- `FFI_TaskContextProvider``crates/core/src/context.rs`
130+
131+
**Evaluated and not requiring direct Python exposure:**
132+
These upstream FFI types have been reviewed and do not need to be independently exposed to end users:
133+
- `FFI_ExecutionPlan` — already used indirectly through table providers; no need for direct exposure
134+
- `FFI_PhysicalExpr` / `FFI_PhysicalSortExpr` — internal physical planning types not expected to be needed by end users
135+
- `FFI_RecordBatchStream` — one level deeper than FFI_ExecutionPlan, used internally when execution plans stream results
136+
- `FFI_SessionRef` / `ForeignSession` — session sharing across FFI; Python manages sessions natively via SessionContext
137+
- `FFI_SessionConfig` — Python can configure sessions natively without FFI
138+
- `FFI_ConfigOptions` / `FFI_TableOptions` — internal configuration plumbing
139+
- `FFI_PlanProperties` / `FFI_Boundedness` / `FFI_EmissionType` — read from existing plans, not user-facing
140+
- `FFI_Partitioning` — supporting type for physical planning
141+
- Supporting/utility types (`FFI_Option`, `FFI_Result`, `WrappedSchema`, `WrappedArray`, `FFI_ColumnarValue`, `FFI_Volatility`, `FFI_InsertOp`, `FFI_AccumulatorArgs`, `FFI_Accumulator`, `FFI_GroupsAccumulator`, `FFI_EmitTo`, `FFI_AggregateOrderSensitivity`, `FFI_PartitionEvaluator`, `FFI_PartitionEvaluatorArgs`, `FFI_Range`, `FFI_SortOptions`, `FFI_Distribution`, `FFI_ExprProperties`, `FFI_SortProperties`, `FFI_Interval`, `FFI_TableProviderFilterPushDown`, `FFI_TableType`) — used as building blocks within the types above, not independently exposed
142+
143+
**How to check:**
144+
1. Compare the upstream `datafusion-ffi` crate's `lib.rs` exports against the lists above
145+
2. If new FFI types appear upstream, evaluate whether they represent a user-facing capability
146+
3. Check against the "evaluated and not requiring exposure" list before flagging as a gap
147+
4. Report any genuinely new types that enable user-facing functionality
148+
5. For each currently supported FFI type, verify the full pipeline is present using the checklist from "Adding a New FFI Type":
149+
- Rust PyO3 wrapper with `from_pycapsule()` method
150+
- Python Protocol type (e.g., `ScalarUDFExportable`) for FFI objects
151+
- Python wrapper class with full type hints on all public methods
152+
- ABC base class (if the type can be user-implemented)
153+
- Registered in Rust `init_module()` and Python `__init__.py`
154+
- FFI example in `examples/datafusion-ffi-example/`
155+
- Type appears in union type hints where accepted
156+
106157
## Output Format
107158

108159
For each area checked, produce a report like:
@@ -297,57 +348,6 @@ use datafusion_ffi::new_type::FFI_NewType;
297348
- [ ] FFI example in `examples/datafusion-ffi-example/`
298349
- [ ] Type appears in union type hints where accepted (e.g., `Table | TableProviderExportable`)
299350

300-
### 7. FFI Types (datafusion-ffi)
301-
302-
**Upstream source of truth:**
303-
- Crate source: https://github.com/apache/datafusion/tree/main/datafusion/ffi/src
304-
- Rust docs: https://docs.rs/datafusion-ffi/latest/datafusion_ffi/
305-
306-
**Where they are exposed in this project:**
307-
- Rust bindings: various files under `crates/core/src/` and `crates/util/src/`
308-
- FFI example: `examples/datafusion-ffi-example/src/`
309-
- Dependency declared in root `Cargo.toml` and `crates/core/Cargo.toml`
310-
311-
**Currently supported FFI types:**
312-
- `FFI_ScalarUDF``crates/core/src/udf.rs`
313-
- `FFI_AggregateUDF``crates/core/src/udaf.rs`
314-
- `FFI_WindowUDF``crates/core/src/udwf.rs`
315-
- `FFI_TableFunction``crates/core/src/udtf.rs`
316-
- `FFI_TableProvider``crates/core/src/table.rs`, `crates/util/src/lib.rs`
317-
- `FFI_TableProviderFactory``crates/core/src/context.rs`
318-
- `FFI_CatalogProvider``crates/core/src/catalog.rs`, `crates/core/src/context.rs`
319-
- `FFI_CatalogProviderList``crates/core/src/context.rs`
320-
- `FFI_SchemaProvider``crates/core/src/catalog.rs`
321-
- `FFI_LogicalExtensionCodec` — multiple files
322-
- `FFI_ExtensionOptions``crates/core/src/context.rs`
323-
- `FFI_TaskContextProvider``crates/core/src/context.rs`
324-
325-
**Evaluated and not requiring direct Python exposure:**
326-
These upstream FFI types have been reviewed and do not need to be independently exposed to end users:
327-
- `FFI_ExecutionPlan` — already used indirectly through table providers; no need for direct exposure
328-
- `FFI_PhysicalExpr` / `FFI_PhysicalSortExpr` — internal physical planning types not expected to be needed by end users
329-
- `FFI_RecordBatchStream` — one level deeper than FFI_ExecutionPlan, used internally when execution plans stream results
330-
- `FFI_SessionRef` / `ForeignSession` — session sharing across FFI; Python manages sessions natively via SessionContext
331-
- `FFI_SessionConfig` — Python can configure sessions natively without FFI
332-
- `FFI_ConfigOptions` / `FFI_TableOptions` — internal configuration plumbing
333-
- `FFI_PlanProperties` / `FFI_Boundedness` / `FFI_EmissionType` — read from existing plans, not user-facing
334-
- `FFI_Partitioning` — supporting type for physical planning
335-
- Supporting/utility types (`FFI_Option`, `FFI_Result`, `WrappedSchema`, `WrappedArray`, `FFI_ColumnarValue`, `FFI_Volatility`, `FFI_InsertOp`, `FFI_AccumulatorArgs`, `FFI_Accumulator`, `FFI_GroupsAccumulator`, `FFI_EmitTo`, `FFI_AggregateOrderSensitivity`, `FFI_PartitionEvaluator`, `FFI_PartitionEvaluatorArgs`, `FFI_Range`, `FFI_SortOptions`, `FFI_Distribution`, `FFI_ExprProperties`, `FFI_SortProperties`, `FFI_Interval`, `FFI_TableProviderFilterPushDown`, `FFI_TableType`) — used as building blocks within the types above, not independently exposed
336-
337-
**How to check:**
338-
1. Compare the upstream `datafusion-ffi` crate's `lib.rs` exports against the lists above
339-
2. If new FFI types appear upstream, evaluate whether they represent a user-facing capability
340-
3. Check against the "evaluated and not requiring exposure" list before flagging as a gap
341-
4. Report any genuinely new types that enable user-facing functionality
342-
5. For each currently supported FFI type, verify the full pipeline is present using the checklist from "Adding a New FFI Type":
343-
- Rust PyO3 wrapper with `from_pycapsule()` method
344-
- Python Protocol type (e.g., `ScalarUDFExportable`) for FFI objects
345-
- Python wrapper class with full type hints on all public methods
346-
- ABC base class (if the type can be user-implemented)
347-
- Registered in Rust `init_module()` and Python `__init__.py`
348-
- FFI example in `examples/datafusion-ffi-example/`
349-
- Type appears in union type hints where accepted
350-
351351
## Important Notes
352352

353353
- The upstream DataFusion version used by this project is specified in `crates/core/Cargo.toml` — check the `datafusion` dependency version to ensure you're comparing against the right upstream version.

0 commit comments

Comments
 (0)