Skip to content

Commit 5f6f962

Browse files
author
Max Wang
committed
add dataframe methods
1 parent bec919c commit 5f6f962

8 files changed

Lines changed: 754 additions & 0 deletions

File tree

‎.claude/skills/dataverse-sdk-use/SKILL.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ The SDK supports Dataverse's native bulk operations: Pass lists to `create()`, `
2424
- Control page size with `page_size` parameter
2525
- Use `top` parameter to limit total records returned
2626

27+
### DataFrame Support
28+
- All CRUD operations have `_dataframe` variants: `get_dataframe`, `create_dataframe`, `update_dataframe`, `delete_dataframe`
29+
2730
## Common Operations
2831

2932
### Import
@@ -114,6 +117,38 @@ client.delete("account", account_id)
114117
client.delete("account", [id1, id2, id3], use_bulk_delete=True)
115118
```
116119

120+
### DataFrame Operations
121+
122+
The SDK provides DataFrame wrappers for all CRUD operations using pandas DataFrames and Series as input/output.
123+
124+
```python
125+
import pandas as pd
126+
127+
# Query records as paged DataFrames (one DataFrame per page)
128+
for df_page in client.get_dataframe("account", filter="statecode eq 0", select=["name"]):
129+
print(f"Page has {len(df_page)} rows")
130+
131+
# Collect all pages into one DataFrame
132+
df = pd.concat(client.get_dataframe("account", select=["name"], top=100), ignore_index=True)
133+
134+
# Fetch single record as one-row DataFrame
135+
df = client.get_dataframe("account", record_id=account_id, select=["name"])
136+
137+
# Create records from a DataFrame (returns a Series of GUIDs)
138+
new_accounts = pd.DataFrame([
139+
{"name": "Contoso", "telephone1": "555-0100"},
140+
{"name": "Fabrikam", "telephone1": "555-0200"},
141+
])
142+
new_accounts["accountid"] = client.create_dataframe("account", new_accounts)
143+
144+
# Update records from a DataFrame (id_column identifies the GUID column)
145+
new_accounts["telephone1"] = ["555-0199", "555-0299"]
146+
client.update_dataframe("account", new_accounts, id_column="accountid")
147+
148+
# Delete records by passing a Series of GUIDs
149+
client.delete_dataframe("account", new_accounts["accountid"])
150+
```
151+
117152
### SQL Queries
118153

119154
SQL queries are **read-only** and support limited SQL syntax. A single SELECT statement with optional WHERE, TOP (integer literal), ORDER BY (column names only), and a simple table alias after FROM is supported. But JOIN and subqueries may not be. Refer to the Dataverse documentation for the current feature set.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,4 @@ Thumbs.db
2525

2626
# Claude local settings
2727
.claude/*.local.json
28+
.claude/*.local.md

‎README.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
2323
- [Quick start](#quick-start)
2424
- [Basic CRUD operations](#basic-crud-operations)
2525
- [Bulk operations](#bulk-operations)
26+
- [DataFrame operations](#dataframe-operations)
2627
- [Query data](#query-data)
2728
- [Table management](#table-management)
2829
- [File operations](#file-operations)
@@ -36,6 +37,7 @@ A Python client library for Microsoft Dataverse that provides a unified interfac
3637
- **⚡ True Bulk Operations**: Automatically uses Dataverse's native `CreateMultiple`, `UpdateMultiple`, and `BulkDelete` Web API operations for maximum performance and transactional integrity
3738
- **📊 SQL Queries**: Execute read-only SQL queries via the Dataverse Web API `?sql=` parameter
3839
- **🏗️ Table Management**: Create, inspect, and delete custom tables and columns programmatically
40+
- **🐼 DataFrame Support**: Pandas wrappers for all CRUD operations, returning DataFrames and Series
3941
- **📎 File Operations**: Upload files to Dataverse file columns with automatic chunking for large files
4042
- **🔐 Azure Identity**: Built-in authentication using Azure Identity credential providers with comprehensive support
4143
- **🛡️ Error Handling**: Structured exception hierarchy with detailed error context and retry guidance
@@ -176,6 +178,39 @@ client.update("account", ids, {"industry": "Technology"})
176178
client.delete("account", ids, use_bulk_delete=True)
177179
```
178180

181+
### DataFrame operations
182+
183+
The SDK provides pandas wrappers for all CRUD operations, using DataFrames and Series for input and output.
184+
185+
```python
186+
import pandas as pd
187+
188+
# Query records as paged DataFrames (one DataFrame per page)
189+
for df_page in client.get_dataframe("account", filter="statecode eq 0", select=["name", "telephone1"]):
190+
print(f"Page has {len(df_page)} rows")
191+
192+
# Collect all pages into one DataFrame
193+
df = pd.concat(client.get_dataframe("account", select=["name"], top=100), ignore_index=True)
194+
print(f"Found {len(df)} accounts")
195+
196+
# Fetch a single record as a one-row DataFrame
197+
df = client.get_dataframe("account", record_id=account_id, select=["name"])
198+
199+
# Create records from a DataFrame (returns a Series of GUIDs)
200+
new_accounts = pd.DataFrame([
201+
{"name": "Contoso", "telephone1": "555-0100"},
202+
{"name": "Fabrikam", "telephone1": "555-0200"},
203+
])
204+
new_accounts["accountid"] = client.create_dataframe("account", new_accounts)
205+
206+
# Update records from a DataFrame (id_column identifies the GUID column)
207+
new_accounts["telephone1"] = ["555-0199", "555-0299"]
208+
client.update_dataframe("account", new_accounts, id_column="accountid")
209+
210+
# Delete records by passing a Series of GUIDs
211+
client.delete_dataframe("account", new_accounts["accountid"])
212+
```
213+
179214
### Query data
180215

181216
```python

‎examples/advanced/dataframe_operations.py‎

Lines changed: 146 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,146 @@
1+
# Copyright (c) Microsoft Corporation.
2+
# Licensed under the MIT license.
3+
4+
"""
5+
PowerPlatform Dataverse Client - DataFrame Operations Walkthrough
6+
7+
This example demonstrates how to use the pandas DataFrame extension methods
8+
for CRUD operations with Microsoft Dataverse.
9+
10+
Prerequisites:
11+
pip install PowerPlatform-Dataverse-Client
12+
pip install azure-identity
13+
"""
14+
15+
import sys
16+
import uuid
17+
from pathlib import Path
18+
19+
# Uncomment to run from local source instead of installed package
20+
# sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "src"))
21+
22+
import pandas as pd
23+
from azure.identity import InteractiveBrowserCredential
24+
from PowerPlatform.Dataverse.client import DataverseClient
25+
26+
27+
def main():
28+
# ── Setup & Authentication ────────────────────────────────────
29+
base_url = input("Enter Dataverse org URL (e.g. https://yourorg.crm.dynamics.com): ").strip()
30+
if not base_url:
31+
print("[ERR] No URL entered; exiting.")
32+
sys.exit(1)
33+
base_url = base_url.rstrip("/")
34+
35+
print("[INFO] Authenticating via browser...")
36+
credential = InteractiveBrowserCredential()
37+
client = DataverseClient(base_url, credential)
38+
39+
table = input("Enter table schema name to use [default: account]: ").strip() or "account"
40+
print(f"[INFO] Using table: {table}")
41+
42+
# Unique tag to isolate test records from existing data
43+
tag = uuid.uuid4().hex[:8]
44+
test_filter = f"contains(name,'{tag}')"
45+
print(f"[INFO] Using tag '{tag}' to identify test records")
46+
47+
# ── 1. Create records from a DataFrame ────────────────────────
48+
print("\n" + "-" * 60)
49+
print("1. Create records from a DataFrame")
50+
print("-" * 60)
51+
52+
new_accounts = pd.DataFrame([
53+
{"name": f"Contoso_{tag}", "telephone1": "555-0100", "websiteurl": "https://contoso.com"},
54+
{"name": f"Fabrikam_{tag}", "telephone1": "555-0200", "websiteurl": "https://fabrikam.com"},
55+
{"name": f"Northwind_{tag}", "telephone1": "555-0300", "websiteurl": "https://northwind.com"},
56+
])
57+
print(f" Input DataFrame:\n{new_accounts.to_string(index=False)}\n")
58+
59+
# create_dataframe returns a Series of GUIDs aligned with the input rows
60+
new_accounts["accountid"] = client.create_dataframe(table, new_accounts)
61+
print(f"[OK] Created {len(new_accounts)} records")
62+
print(f" IDs: {new_accounts['accountid'].tolist()}")
63+
64+
# ── 2. Query records as paged DataFrames ──────────────────────
65+
print("\n" + "-" * 60)
66+
print("2. Query records as paged DataFrames (lazy generator)")
67+
print("-" * 60)
68+
69+
page_count = 0
70+
for df_page in client.get_dataframe(table, select=["name", "telephone1"], filter=test_filter, page_size=2):
71+
page_count += 1
72+
print(f" Page {page_count} ({len(df_page)} records):\n{df_page.to_string(index=False)}")
73+
74+
# ── 3. Collect all pages into one DataFrame ───────────────────
75+
print("\n" + "-" * 60)
76+
print("3. Collect all pages into one DataFrame with pd.concat")
77+
print("-" * 60)
78+
79+
all_records = pd.concat(
80+
client.get_dataframe(table, select=["name", "telephone1"], filter=test_filter, page_size=2),
81+
ignore_index=True,
82+
)
83+
print(f"[OK] Got {len(all_records)} total records in one DataFrame")
84+
print(f" Columns: {list(all_records.columns)}")
85+
print(f"{all_records.to_string(index=False)}")
86+
87+
# ── 4. Fetch a single record by ID ────────────────────────────
88+
print("\n" + "-" * 60)
89+
print("4. Fetch a single record by ID")
90+
print("-" * 60)
91+
92+
first_id = new_accounts["accountid"].iloc[0]
93+
print(f" Fetching record {first_id}...")
94+
single = client.get_dataframe(table, record_id=first_id, select=["name", "telephone1"])
95+
print(f"[OK] Single record DataFrame:\n{single.to_string(index=False)}")
96+
97+
# ── 5. Update records from a DataFrame ────────────────────────
98+
print("\n" + "-" * 60)
99+
print("5. Update records with different values per row")
100+
print("-" * 60)
101+
102+
new_accounts["telephone1"] = ["555-1100", "555-1200", "555-1300"]
103+
print(f" New telephone numbers: {new_accounts['telephone1'].tolist()}")
104+
client.update_dataframe(table, new_accounts[["accountid", "telephone1"]], id_column="accountid")
105+
print("[OK] Updated 3 records")
106+
107+
# Verify the updates with a bulk get
108+
verified = next(client.get_dataframe(table, select=["name", "telephone1"], filter=test_filter))
109+
print(f" Verified:\n{verified.to_string(index=False)}")
110+
111+
# ── 6. Broadcast update (same value to all records) ───────────
112+
print("\n" + "-" * 60)
113+
print("6. Broadcast update (same value to all records)")
114+
print("-" * 60)
115+
116+
broadcast_df = new_accounts[["accountid"]].copy()
117+
broadcast_df["websiteurl"] = "https://updated.example.com"
118+
print(f" Setting websiteurl to 'https://updated.example.com' for all {len(broadcast_df)} records")
119+
client.update_dataframe(table, broadcast_df, id_column="accountid")
120+
print("[OK] Broadcast update complete")
121+
122+
# Verify all records have the same websiteurl
123+
verified = next(client.get_dataframe(table, select=["name", "websiteurl"], filter=test_filter))
124+
print(f" Verified:\n{verified.to_string(index=False)}")
125+
126+
# ── 7. Delete records by passing a Series of GUIDs ────────────
127+
print("\n" + "-" * 60)
128+
print("7. Delete records by passing a Series of GUIDs")
129+
print("-" * 60)
130+
131+
print(f" Deleting {len(new_accounts)} records...")
132+
client.delete_dataframe(table, new_accounts["accountid"], use_bulk_delete=False)
133+
print(f"[OK] Deleted {len(new_accounts)} records")
134+
135+
# Verify deletions - filter for our tagged records should return 0
136+
remaining = list(client.get_dataframe(table, select=["name"], filter=test_filter))
137+
count = sum(len(page) for page in remaining)
138+
print(f" Verified: {count} test records remaining (expected 0)")
139+
140+
print("\n" + "=" * 60)
141+
print("[OK] DataFrame operations walkthrough complete!")
142+
print("=" * 60)
143+
144+
145+
if __name__ == "__main__":
146+
main()

‎pyproject.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ dependencies = [
2929
"azure-identity>=1.17.0",
3030
"azure-core>=1.30.2",
3131
"requests>=2.32.0",
32+
"pandas>=2.0.0",
3233
]
3334

3435
[project.urls]

‎src/PowerPlatform/Dataverse/claude_skill/dataverse-sdk-use/SKILL.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ The SDK supports Dataverse's native bulk operations: Pass lists to `create()`, `
2424
- Control page size with `page_size` parameter
2525
- Use `top` parameter to limit total records returned
2626

27+
### DataFrame Support
28+
- All CRUD operations have `_dataframe` variants: `get_dataframe`, `create_dataframe`, `update_dataframe`, `delete_dataframe`
29+
2730
## Common Operations
2831

2932
### Import
@@ -114,6 +117,38 @@ client.delete("account", account_id)
114117
client.delete("account", [id1, id2, id3], use_bulk_delete=True)
115118
```
116119

120+
### DataFrame Operations
121+
122+
The SDK provides DataFrame wrappers for all CRUD operations using pandas DataFrames and Series as input/output.
123+
124+
```python
125+
import pandas as pd
126+
127+
# Query records as paged DataFrames (one DataFrame per page)
128+
for df_page in client.get_dataframe("account", filter="statecode eq 0", select=["name"]):
129+
print(f"Page has {len(df_page)} rows")
130+
131+
# Collect all pages into one DataFrame
132+
df = pd.concat(client.get_dataframe("account", select=["name"], top=100), ignore_index=True)
133+
134+
# Fetch single record as one-row DataFrame
135+
df = client.get_dataframe("account", record_id=account_id, select=["name"])
136+
137+
# Create records from a DataFrame (returns a Series of GUIDs)
138+
new_accounts = pd.DataFrame([
139+
{"name": "Contoso", "telephone1": "555-0100"},
140+
{"name": "Fabrikam", "telephone1": "555-0200"},
141+
])
142+
new_accounts["accountid"] = client.create_dataframe("account", new_accounts)
143+
144+
# Update records from a DataFrame (id_column identifies the GUID column)
145+
new_accounts["telephone1"] = ["555-0199", "555-0299"]
146+
client.update_dataframe("account", new_accounts, id_column="accountid")
147+
148+
# Delete records by passing a Series of GUIDs
149+
client.delete_dataframe("account", new_accounts["accountid"])
150+
```
151+
117152
### SQL Queries
118153

119154
SQL queries are **read-only** and support limited SQL syntax. A single SELECT statement with optional WHERE, TOP (integer literal), ORDER BY (column names only), and a simple table alias after FROM is supported. But JOIN and subqueries may not be. Refer to the Dataverse documentation for the current feature set.

0 commit comments

Comments
 (0)