fix: add timeout to requests.post()/get() calls in oauth.py#1339
Open
cgrierson-smartsheet wants to merge 2 commits intodatabricks:mainfrom
Open
fix: add timeout to requests.post()/get() calls in oauth.py#1339cgrierson-smartsheet wants to merge 2 commits intodatabricks:mainfrom
cgrierson-smartsheet wants to merge 2 commits intodatabricks:mainfrom
Conversation
aa939a6 to
cd9626a
Compare
requests.post() and requests.get() calls in oauth.py's retrieve_token(), get_azure_entra_id_workspace_endpoints(), and PATOAuthTokenExchange.refresh() do not pass a timeout= parameter. When the OAuth endpoint is unreachable or slow, these calls block indefinitely. The SDK's per-request timeout (session.request(timeout=60)) does not protect against this because the token refresh runs inside session.auth, before the timeout takes effect. Extract _DEFAULT_HTTP_TIMEOUT_SECONDS constant from the existing hardcoded value in _BaseClient.__init__ and use it consistently across all OAuth HTTP calls. Add http_timeout_seconds fields to ClientCredentials and PATOAuthTokenExchange dataclasses, and timeout parameters to retrieve_token() and get_azure_entra_id_workspace_endpoints(). All call sites in credentials_provider.py and config.py pass cfg.http_timeout_seconds so the timeout is user-configurable via Config. Fixes databricks#1338 Signed-off-by: Chris Grierson <christopher.grierson@smartsheet.com>
cd9626a to
d4c5563
Compare
|
If integration tests don't run automatically, an authorized user can run them manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
timeoutto allrequests.post()/requests.get()calls inoauth.pythat currently have no timeout parameter. Without a timeout, these calls block indefinitely when the OAuth endpoint is unreachable or slow.Extract
_DEFAULT_HTTP_TIMEOUT_SECONDSconstant from the existing hardcodedor 60in_BaseClient.__init__and use it consistently. The timeout is configurable viaConfig.http_timeout_seconds, threaded through from all call sites.Changes
_base_client.py:_DEFAULT_HTTP_TIMEOUT_SECONDS = 60constant (replaces the existing inlineor 60)_BaseClient.__init__now uses the constantoauth.py:retrieve_token(): addtimeoutparameter (default_DEFAULT_HTTP_TIMEOUT_SECONDS), pass torequests.post()get_azure_entra_id_workspace_endpoints(): addtimeoutparameter (default_DEFAULT_HTTP_TIMEOUT_SECONDS), pass torequests.get()ClientCredentialsdataclass: addhttp_timeout_secondsfield (default_DEFAULT_HTTP_TIMEOUT_SECONDS), pass toretrieve_token()PATOAuthTokenExchangedataclass: addhttp_timeout_secondsfield (default_DEFAULT_HTTP_TIMEOUT_SECONDS), pass torequests.post()credentials_provider.py:ClientCredentials(...)andPATOAuthTokenExchange(...)construction sites: passhttp_timeout_seconds=cfg.http_timeout_seconds or _DEFAULT_HTTP_TIMEOUT_SECONDSget_azure_entra_id_workspace_endpoints(...)calls: passtimeout=cfg.http_timeout_seconds or _DEFAULT_HTTP_TIMEOUT_SECONDSconfig.py:oidc_endpointsproperty: passtimeout=self.http_timeout_seconds or _DEFAULT_HTTP_TIMEOUT_SECONDStoget_azure_entra_id_workspace_endpoints()Problem
These calls bypass
_BaseCliententirely and use therequestslibrary directly. The SDK's per-request timeout (session.request(timeout=60)) does not protect against hangs in these calls because they execute insidesession.auth(theheader_factorycallback), which runs before the request timeout takes effect.In long-running processes (>60 min), the M2M OAuth token expires (TTL = 3600s) and the next API call triggers a synchronous token refresh through
retrieve_token(). If the OAuth endpoint is slow at that moment,requests.post()blocks indefinitely, hanging the entire process.See #1338 for full analysis, failure mode tables, and a self-contained reproduction script.
Related: #1046 (non-configurable timeout for
_BaseClient.do()in OIDC endpoint discovery — same family of issue, different code path).Test plan
retrieve_token()raisesrequests.exceptions.ConnectTimeoutorReadTimeoutinstead of hanging when endpoint is unreachableConfig(http_timeout_seconds=N)Signed-off-by: Chris Grierson christopher.grierson@smartsheet.com