Merge origin/feature/querybuilder-clean and add expand examples to walkthrough

Author: Abel Milash

.claude/skills/dataverse-sdk-dev/SKILL.md

@@ -20,6 +20,32 @@ This skill provides guidance for developers working on the PowerPlatform Dataver
 5. **Consider backwards compatibility** - Avoid breaking changes
 6. **Internal vs public naming** - Modules, files, and functions not meant to be part of the public API must use a `_` prefix (e.g., `_odata.py`, `_relationships.py`). Files without the prefix (e.g., `constants.py`, `metadata.py`) are public and importable by SDK consumers
 
+### Dataverse Property Naming Rules
+
+Dataverse uses two different naming conventions for properties. Getting this wrong causes 400 errors that are hard to debug.
+
+| Property type | Name convention | Example | When used |
+|---|---|---|---|
+| **Structural** (columns) | LogicalName (always lowercase) | `new_name`, `new_priority` | `$select`, `$filter`, `$orderby`, record payload keys |
+| **Navigation** (relationships / lookups) | Navigation Property Name (usually SchemaName, PascalCase, case-sensitive) | `new_CustomerId`, `new_AgentId` | `$expand`, `@odata.bind` annotation keys |
+
+Navigation property names are case-sensitive and must match the entity's `$metadata`. Using the logical name instead of the navigation property name results in 400 Bad Request errors.
+
+**Critical rule:** The OData parser validates `@odata.bind` property names **case-sensitively** against declared navigation properties. Lowercasing `new_CustomerId@odata.bind` to `new_customerid@odata.bind` causes: `ODataException: An undeclared property 'new_customerid' which only has property annotations...`
+
+**SDK implementation:**
+
+- `_lowercase_keys()` lowercases all keys EXCEPT those containing `@odata.` (preserves navigation property casing in `@odata.bind` keys)
+- `_lowercase_list()` lowercases `$select` and `$orderby` params (structural properties)
+- `$expand` params are passed as-is (navigation properties, PascalCase)
+- `_convert_labels_to_ints()` skips `@odata.` keys entirely (they are annotations, not attributes)
+
+**When adding new code that processes record dicts or builds query parameters:**
+
+- Always use `_lowercase_keys()` for record payloads. Never manually call `.lower()` on all keys
+- Never lowercase `$expand` values or `@odata.bind` key prefixes
+- If iterating record keys, skip keys containing `@odata.` when doing attribute-level operations
+
 ### Code Style
 
 6. **No emojis** - Do not use emoji in code, comments, or output

.claude/skills/dataverse-sdk-use/SKILL.md

@@ -30,6 +30,9 @@ The SDK supports Dataverse's native bulk operations: Pass lists to `create()`, `
 - Control page size with `page_size` parameter
 - Use `top` parameter to limit total records returned
 
+### DataFrame Support
+- DataFrame operations are accessed via the `client.dataframe` namespace: `client.dataframe.get()`, `client.dataframe.create()`, `client.dataframe.update()`, `client.dataframe.delete()`
+
 ## Common Operations
 
 ### Import
@@ -105,6 +108,20 @@ for page in client.records.get(
         print(f"{account['name']} - {contact.get('fullname', 'N/A')}")
 ```
 
+#### Create Records with Lookup Bindings (@odata.bind)
+```python
+# Set lookup fields using @odata.bind with PascalCase navigation property names
+# CORRECT: use the navigation property name (case-sensitive, must match $metadata)
+guid = client.records.create("new_ticket", {
+    "new_name": "TKT-001",
+    "new_CustomerId@odata.bind": f"/new_customers({customer_id})",
+    "new_AgentId@odata.bind": f"/new_agents({agent_id})",
+})
+
+# WRONG: lowercase navigation property causes 400 error
+# "new_customerid@odata.bind" -> ODataException: undeclared property 'new_customerid'
+```
+
 #### Update Records
 ```python
 # Single update
@@ -115,7 +132,7 @@ client.records.update("account", [id1, id2, id3], {"industry": "Technology"})
 ```
 
 #### Upsert Records
-Creates or updates records identified by alternate keys. Single item → PATCH; multiple items → `UpsertMultiple` bulk action.
+Creates or updates records identified by alternate keys. Single item -> PATCH; multiple items -> `UpsertMultiple` bulk action.
 > **Prerequisite**: The table must have an alternate key configured in Dataverse for the columns used in `alternate_key`. Without it, Dataverse will reject the request with a 400 error.
 ```python
 from PowerPlatform.Dataverse.models.upsert import UpsertItem
@@ -157,6 +174,42 @@ client.records.delete("account", account_id)
 client.records.delete("account", [id1, id2, id3], use_bulk_delete=True)
 ```
 
+### DataFrame Operations
+
+The SDK provides DataFrame wrappers for all CRUD operations via the `client.dataframe` namespace, using pandas DataFrames and Series as input/output.
+
+```python
+import pandas as pd
+
+# Query records -- returns a single DataFrame
+df = client.dataframe.get("account", filter="statecode eq 0", select=["name"])
+print(f"Got {len(df)} rows")
+
+# Limit results with top for large tables
+df = client.dataframe.get("account", select=["name"], top=100)
+
+# Fetch single record as one-row DataFrame
+df = client.dataframe.get("account", record_id=account_id, select=["name"])
+
+# Create records from a DataFrame (returns a Series of GUIDs)
+new_accounts = pd.DataFrame([
+    {"name": "Contoso", "telephone1": "555-0100"},
+    {"name": "Fabrikam", "telephone1": "555-0200"},
+])
+new_accounts["accountid"] = client.dataframe.create("account", new_accounts)
+
+# Update records from a DataFrame (id_column identifies the GUID column)
+new_accounts["telephone1"] = ["555-0199", "555-0299"]
+client.dataframe.update("account", new_accounts, id_column="accountid")
+
+# Clear a field by setting clear_nulls=True (by default, NaN/None fields are skipped)
+df = pd.DataFrame([{"accountid": "guid-1", "websiteurl": None}])
+client.dataframe.update("account", df, id_column="accountid", clear_nulls=True)
+
+# Delete records by passing a Series of GUIDs
+client.dataframe.delete("account", new_accounts["accountid"])
+```
+
 ### SQL Queries
 
 SQL queries are **read-only** and support limited SQL syntax. A single SELECT statement with optional WHERE, TOP (integer literal), ORDER BY (column names only), and a simple table alias after FROM is supported. But JOIN and subqueries may not be. Refer to the Dataverse documentation for the current feature set.
@@ -359,6 +412,7 @@ except ValidationError as e:
 - Check filter/expand parameters use correct case
 - Verify column names exist and are spelled correctly
 - Ensure custom columns include customization prefix
+- For `@odata.bind` errors ("undeclared property"): the navigation property name before `@odata.bind` is case-sensitive and must match the entity's `$metadata` exactly (e.g., `new_CustomerId@odata.bind` for custom lookups, `parentaccountid@odata.bind` for system lookups). The SDK preserves `@odata.bind` key casing.
 
 ## Best Practices
 
@@ -371,7 +425,7 @@ except ValidationError as e:
 5. **Use production credentials** - ClientSecretCredential or CertificateCredential for unattended operations
 6. **Error handling** - Implement retry logic for transient errors (`e.is_transient`)
 7. **Always include customization prefix** for custom tables/columns
-8. **Use lowercase** - Generally using lowercase input won't go wrong, except for custom table/column naming
+8. **Use lowercase for column names, match `$metadata` for navigation properties** - Column names in `$select`/`$filter`/record payloads use lowercase LogicalNames. Navigation properties in `$expand` and `@odata.bind` keys are case-sensitive and must match the entity's `$metadata` (PascalCase for custom lookups like `new_CustomerId`, lowercase for system lookups like `parentaccountid`)
 9. **Test in non-production environments** first
 10. **Use named constants** - Import cascade behavior constants from `PowerPlatform.Dataverse.common.constants`
 

.gitignore

@@ -25,3 +25,4 @@ Thumbs.db
 
 # Claude local settings
 .claude/*.local.json
+.claude/*.local.md

CHANGELOG.md

@@ -5,6 +5,18 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.0b6] - 2026-03-12
+
+### Added
+- Context manager support: `with DataverseClient(...) as client:` for automatic resource cleanup, HTTP connection pooling, and `close()` for explicit lifecycle management (#117)
+- Typed return models `Record`, `TableInfo`, and `ColumnInfo` for record and table metadata operations, replacing raw `Dict[str, Any]` returns with full backward compatibility (`result["key"]` still works) (#115)
+- Alternate key management: `client.tables.create_alternate_key()`, `client.tables.get_alternate_keys()`, `client.tables.delete_alternate_key()` with typed `AlternateKeyInfo` model (#126)
+
+### Fixed
+- `@odata.bind` lookup bindings now preserve navigation property casing (e.g., `new_CustomerId@odata.bind`), fixing `400 Bad Request` errors on create/update/upsert with lookup fields (#137)
+- Reduced unnecessary HTTP round-trips on create/update/upsert when records contain `@odata.bind` keys (#137)
+- Single-record `get()` now lowercases `$select` column names consistently with multi-record queries (#137)
+
 ## [0.1.0b5] - 2026-02-27
 
 ### Fixed
@@ -70,6 +82,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Comprehensive error handling with specific exception types (`DataverseError`, `AuthenticationError`, etc.) (#22, #24)
 - HTTP retry logic with exponential backoff for resilient operations (#72)
 
+[0.1.0b6]: https://github.com/microsoft/PowerPlatform-DataverseClient-Python/compare/v0.1.0b5...v0.1.0b6
 [0.1.0b5]: https://github.com/microsoft/PowerPlatform-DataverseClient-Python/compare/v0.1.0b4...v0.1.0b5
 [0.1.0b4]: https://github.com/microsoft/PowerPlatform-DataverseClient-Python/compare/v0.1.0b3...v0.1.0b4
 [0.1.0b3]: https://github.com/microsoft/PowerPlatform-DataverseClient-Python/compare/v0.1.0b2...v0.1.0b3

README.md

@@ -24,6 +24,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
   - [Basic CRUD operations](#basic-crud-operations)
   - [Bulk operations](#bulk-operations)
   - [Upsert operations](#upsert-operations)
+  - [DataFrame operations](#dataframe-operations)
   - [Query data](#query-data) *(QueryBuilder, SQL, raw OData)*
   - [Table management](#table-management)
   - [Relationship management](#relationship-management)
@@ -40,6 +41,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
 - **📊 SQL Queries**: Execute read-only SQL queries via the Dataverse Web API `?sql=` parameter
 - **🏗️ Table Management**: Create, inspect, and delete custom tables and columns programmatically
 - **🔗 Relationship Management**: Create one-to-many and many-to-many relationships between tables with full metadata control
+- **🐼 DataFrame Support**: Pandas wrappers for all CRUD operations, returning DataFrames and Series
 - **📎 File Operations**: Upload files to Dataverse file columns with automatic chunking for large files
 - **🔐 Azure Identity**: Built-in authentication using Azure Identity credential providers with comprehensive support
 - **🛡️ Error Handling**: Structured exception hierarchy with detailed error context and retry guidance
@@ -233,6 +235,42 @@ client.records.upsert("account", [
 ])
 ```
 
+### DataFrame operations
+
+The SDK provides pandas wrappers for all CRUD operations via the `client.dataframe` namespace, using DataFrames and Series for input and output.
+
+```python
+import pandas as pd
+
+# Query records as a single DataFrame
+df = client.dataframe.get("account", filter="statecode eq 0", select=["name", "telephone1"])
+print(f"Found {len(df)} accounts")
+
+# Limit results with top for large tables
+df = client.dataframe.get("account", select=["name"], top=100)
+
+# Fetch a single record as a one-row DataFrame
+df = client.dataframe.get("account", record_id=account_id, select=["name"])
+
+# Create records from a DataFrame (returns a Series of GUIDs)
+new_accounts = pd.DataFrame([
+    {"name": "Contoso", "telephone1": "555-0100"},
+    {"name": "Fabrikam", "telephone1": "555-0200"},
+])
+new_accounts["accountid"] = client.dataframe.create("account", new_accounts)
+
+# Update records from a DataFrame (id_column identifies the GUID column)
+new_accounts["telephone1"] = ["555-0199", "555-0299"]
+client.dataframe.update("account", new_accounts, id_column="accountid")
+
+# Clear a field by setting clear_nulls=True (by default, NaN/None fields are skipped)
+df = pd.DataFrame([{"accountid": new_accounts["accountid"].iloc[0], "websiteurl": None}])
+client.dataframe.update("account", df, id_column="accountid", clear_nulls=True)
+
+# Delete records by passing a Series of GUIDs
+client.dataframe.delete("account", new_accounts["accountid"])
+```
+
 ### Query data
 
 The **QueryBuilder** is the recommended way to query records. It provides a fluent, type-safe interface that generates correct OData queries automatically — no need to remember OData filter syntax.
@@ -252,6 +290,107 @@ for record in (client.query.builder("account")
 
 The QueryBuilder handles value formatting, column name casing, and OData syntax automatically. All filter methods are discoverable via IDE autocomplete:
 
+```python
+# Get results as a pandas DataFrame (consolidates all pages)
+df = (client.query.builder("account")
+      .select("name", "telephone1")
+      .filter_eq("statecode", 0)
+      .top(100)
+      .to_dataframe())
+print(f"Got {len(df)} accounts")
+```
+
+```python
+# Comparison filters
+query = (client.query.builder("contact")
+         .filter_eq("statecode", 0)          # statecode eq 0
+         .filter_gt("revenue", 1000000)      # revenue gt 1000000
+         .filter_contains("name", "Corp")    # contains(name, 'Corp')
+         .filter_in("statecode", [0, 1])     # Microsoft.Dynamics.CRM.In(...)
+         .filter_between("revenue", 100000, 500000)  # (revenue ge 100000 and revenue le 500000)
+         .filter_null("telephone1")          # telephone1 eq null
+         )
+```
+
+For complex logic (OR, NOT, grouping), use the composable expression tree with `where()`:
+
+```python
+from PowerPlatform.Dataverse.models.filters import eq, gt, filter_in, between
+
+# OR conditions: (statecode = 0 OR statecode = 1) AND revenue > 100k
+for record in (client.query.builder("account")
+               .select("name", "revenue")
+               .where((eq("statecode", 0) | eq("statecode", 1))
+                      & gt("revenue", 100000))
+               .execute()):
+    print(record["name"])
+
+# NOT, between, and in operators
+for record in (client.query.builder("account")
+               .where(~eq("statecode", 2))                  # NOT inactive
+               .where(between("revenue", 100000, 500000))    # revenue in range
+               .execute()):
+    print(record["name"])
+```
+
+**Formatted values and annotations** -- request localized labels, currency symbols, and display names:
+
+```python
+# Get formatted values (choice labels, currency, lookup names)
+for record in (client.query.builder("account")
+               .select("name", "statecode", "revenue")
+               .include_formatted_values()
+               .execute()):
+    status = record["statecode@OData.Community.Display.V1.FormattedValue"]
+    print(f"{record['name']}: {status}")
+```
+
+**Nested expand with options** -- expand navigation properties with `$select`, `$filter`, `$orderby`, and `$top`:
+
+```python
+from PowerPlatform.Dataverse.models.query_builder import ExpandOption
+
+# Expand related tasks with filtering and sorting
+for record in (client.query.builder("account")
+               .select("name")
+               .expand(ExpandOption("Account_Tasks")
+                       .select("subject", "createdon")
+                       .filter("contains(subject,'Task')")
+                       .order_by("createdon", descending=True)
+                       .top(5))
+               .execute()):
+    print(record["name"], record.get("Account_Tasks"))
+```
+
+**Record count** -- include `$count=true` in the request:
+
+```python
+# Request count alongside results
+results = (client.query.builder("account")
+           .filter_eq("statecode", 0)
+           .count()
+           .execute())
+```
+
+**SQL queries** provide an alternative read-only query syntax:
+
+The **QueryBuilder** is the recommended way to query records. It provides a fluent, type-safe interface that generates correct OData queries automatically — no need to remember OData filter syntax.
+
+```python
+# Fluent query builder (recommended)
+for record in (client.query.builder("account")
+               .select("name", "revenue")
+               .filter_eq("statecode", 0)
+               .filter_gt("revenue", 1000000)
+               .order_by("revenue", descending=True)
+               .top(100)
+               .page_size(50)
+               .execute()):
+    print(f"{record['name']}: {record['revenue']}")
+```
+
+The QueryBuilder handles value formatting, column name casing, and OData syntax automatically. All filter methods are discoverable via IDE autocomplete:
+
 ```python
 # Comparison filters
 query = (client.query.builder("contact")