Skip to content

Commit 88ed86c

Browse files
timsaucerclaude
andcommitted
Make Python API the source of truth for upstream coverage checks
Functions exposed in Python (e.g., as aliases of other Rust bindings) were being falsely reported as missing because they lacked a dedicated #[pyfunction] in Rust. The user-facing API is the Python layer, so coverage should be measured there. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 2d7ca47 commit 88ed86c

File tree

1 file changed

+20
-15
lines changed

1 file changed

+20
-15
lines changed

.claude/skills/check-upstream/SKILL.md

Lines changed: 20 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,8 @@ argument-hint: [area] (e.g., "scalar functions", "aggregate functions", "window
88

99
You are auditing the datafusion-python project to find features from the upstream Apache DataFusion Rust library that are **not yet exposed** in this Python binding project. Your goal is to identify gaps and, if asked, implement the missing bindings.
1010

11+
**IMPORTANT: The Python API is the source of truth for coverage.** A function or method is considered "exposed" if it exists in the Python API (e.g., `python/datafusion/functions.py`), even if there is no corresponding entry in the Rust bindings. Many upstream functions are aliases of other functions — the Python layer can expose these aliases by calling a different underlying Rust binding. Do NOT report a function as missing if it appears in the Python `__all__` list and has a working implementation, regardless of whether a matching `#[pyfunction]` exists in Rust.
12+
1113
## Areas to Check
1214

1315
The user may specify an area via `$ARGUMENTS`. If no area is specified or "all" is given, check all areas.
@@ -24,9 +26,9 @@ The user may specify an area via `$ARGUMENTS`. If no area is specified or "all"
2426

2527
**How to check:**
2628
1. Fetch the upstream scalar function documentation page
27-
2. Compare against functions listed in `python/datafusion/functions.py` (check the `__all__` list)
28-
3. Also check `crates/core/src/functions.rs` for what's registered in `init_module()`
29-
4. Report functions that exist upstream but are missing from this project
29+
2. Compare against functions listed in `python/datafusion/functions.py` (check the `__all__` list and function definitions)
30+
3. A function is covered if it exists in the Python API — it does NOT need a dedicated Rust `#[pyfunction]`. Many functions are aliases that reuse another function's Rust binding.
31+
4. Only report functions that are missing from the Python `__all__` list / function definitions
3032

3133
### 2. Aggregate Functions
3234

@@ -40,8 +42,9 @@ The user may specify an area via `$ARGUMENTS`. If no area is specified or "all"
4042

4143
**How to check:**
4244
1. Fetch the upstream aggregate function documentation page
43-
2. Compare against aggregate functions in `python/datafusion/functions.py`
44-
3. Report missing aggregate functions
45+
2. Compare against aggregate functions in `python/datafusion/functions.py` (check `__all__` list and function definitions)
46+
3. A function is covered if it exists in the Python API, even if it aliases another function's Rust binding
47+
4. Report only functions missing from the Python API
4548

4649
### 3. Window Functions
4750

@@ -55,8 +58,9 @@ The user may specify an area via `$ARGUMENTS`. If no area is specified or "all"
5558

5659
**How to check:**
5760
1. Fetch the upstream window function documentation page
58-
2. Compare against window functions in `python/datafusion/functions.py`
59-
3. Report missing window functions
61+
2. Compare against window functions in `python/datafusion/functions.py` (check `__all__` list and function definitions)
62+
3. A function is covered if it exists in the Python API, even if it aliases another function's Rust binding
63+
4. Report only functions missing from the Python API
6064

6165
### 4. Table Functions
6266

@@ -70,8 +74,9 @@ The user may specify an area via `$ARGUMENTS`. If no area is specified or "all"
7074

7175
**How to check:**
7276
1. Fetch the upstream table function documentation
73-
2. Compare against what's available in this project
74-
3. Report missing table functions
77+
2. Compare against what's available in the Python API
78+
3. A function is covered if it exists in the Python API, even if it aliases another function's Rust binding
79+
4. Report only functions missing from the Python API
7580

7681
### 5. DataFrame Operations
7782

@@ -84,9 +89,9 @@ The user may specify an area via `$ARGUMENTS`. If no area is specified or "all"
8489

8590
**How to check:**
8691
1. Fetch the upstream DataFrame documentation page listing all methods
87-
2. Compare against methods in `python/datafusion/dataframe.py`
88-
3. Also check `crates/core/src/dataframe.rs` for what's implemented
89-
4. Report DataFrame methods that exist upstream but are missing
92+
2. Compare against methods in `python/datafusion/dataframe.py` — this is the source of truth for coverage
93+
3. The Rust bindings (`crates/core/src/dataframe.rs`) may be consulted for context, but a method is covered if it exists in the Python API
94+
4. Report only methods missing from the Python API
9095

9196
### 6. SessionContext Methods
9297

@@ -99,9 +104,9 @@ The user may specify an area via `$ARGUMENTS`. If no area is specified or "all"
99104

100105
**How to check:**
101106
1. Fetch the upstream SessionContext documentation page listing all methods
102-
2. Compare against methods in `python/datafusion/context.py`
103-
3. Also check `crates/core/src/context.rs` for what's implemented
104-
4. Report SessionContext methods that exist upstream but are missing
107+
2. Compare against methods in `python/datafusion/context.py` — this is the source of truth for coverage
108+
3. The Rust bindings (`crates/core/src/context.rs`) may be consulted for context, but a method is covered if it exists in the Python API
109+
4. Report only methods missing from the Python API
105110

106111
### 7. FFI Types (datafusion-ffi)
107112

0 commit comments

Comments
 (0)