-
Notifications
You must be signed in to change notification settings - Fork 371
Passthrough UserId for open ai requests #1177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: kevin-shelaga <kevin.shelaga@solo.io>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR implements user ID passthrough functionality for OpenAI API requests to enable better usage tracking and rate limiting. The implementation uses Python's contextvars to pass the user_id through async call chains without modifying function signatures.
Key Changes:
- Introduced a context variable system to store and retrieve user_id across async boundaries
- Modified OpenAI/Azure OpenAI integration to include user_id parameter in API requests when available
- Updated agent executor to set user_id in context from incoming request parameters
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
python/packages/kagent-adk/src/kagent/adk/_context.py |
New module providing context variable functions (set_user_id, get_user_id, clear_user_id) for passing user_id through async call chains |
python/packages/kagent-adk/src/kagent/adk/models/_openai.py |
Modified generate_content_async to retrieve user_id from context and pass it to OpenAI API (applies to both OpenAI and Azure OpenAI) |
python/packages/kagent-adk/src/kagent/adk/_agent_executor.py |
Updated _handle_request to set user_id in context from run_args and ensure cleanup after request processing |
python/packages/kagent-adk/tests/unittests/test_context.py |
New test file with comprehensive coverage of context variable functionality including async isolation tests |
python/packages/kagent-adk/tests/unittests/models/test_openai.py |
Added 6 new test cases covering user passthrough for OpenAI, Azure OpenAI, streaming mode, and edge cases (empty/whitespace user_id) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Shelaga <65769106+kevin-shelaga@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kevin Shelaga <65769106+kevin-shelaga@users.noreply.github.com>
| } | ||
|
|
||
| # Add user parameter for usage tracking if user_id is available in context | ||
| user_id = get_user_id() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems like the chat completions API is deprecating the user's field https://platform.openai.com/docs/api-reference/chat
user: This field is being replaced by `safety_identifier` and `prompt_cache_key`. Use
`prompt_cache_key` instead to maintain caching optimizations. A stable
identifier for your end-users. Used to boost cache hit rates by better bucketing
similar requests and to help OpenAI detect and prevent abuse.
[Learn more](https://platform.openai.com/docs/guides/safety-best-practices#safety-identifiers).
No description provided.