chore(sec): bump deps + drop Python 3.9 to clear all OSV findings (5.0.0)#818
Open
vikrantpuppala wants to merge 4 commits into
Open
chore(sec): bump deps + drop Python 3.9 to clear all OSV findings (5.0.0)#818vikrantpuppala wants to merge 4 commits into
vikrantpuppala wants to merge 4 commits into
Conversation
…0.0) Clears 25 OSV-Scanner findings (1 CRITICAL cryptography, 13 HIGH across cryptography/urllib3/pyjwt/pyarrow/black, 10 MED, 1 UNKNOWN) by bumping direct dependency floors and regenerating the lockfile. Forces a major version bump because the new dependency constraints will conflict with customers' transitive-dep pins on the older majors of urllib3 (1.x → 2.x), pyarrow (14/18/22 → 23), pytest (7 → 9), and black (22 → 26). Drops Python 3.8/3.9 from supported floors. Python 3.8 is 20 months past upstream EOL (2024-10), Python 3.9 is 7 months past (2025-10). The patched versions of urllib3 / pyarrow / requests / pytest / black all declare requires_python>=3.10, so keeping 3.9 in the matrix would silently downgrade these to CVE-vulnerable versions for 3.9 customers — defeating the entire purpose of the PR. Direct runtime bumps: - urllib3: >=1.26 -> >=2.7.0,<3.0.0 (urllib3 1.x -> 2.x major) - requests: ^2.18.1 -> ^2.33.0 - pyjwt: ^2.0.0 -> ^2.12.0 - pyarrow: 14/18/22+ -> >=23.0.1 (all three Python-version-gated blocks) Direct dev bumps: - pytest: ^7.1.2 -> ^9.0.3 - black: ^22.3.0 -> ^26.3.1 (reformats 13 src files in a follow-up commit) Transitive cleanups via lockfile (no pyproject change): - cryptography 43.0.3/45.0.6 -> 48.0.0 (clears CRITICAL PYSEC-2026-36 9.8) - idna 3.10 -> 3.16 - python-dotenv 1.0.1 -> 1.2.2 Verification: - OSV-Scanner v2.3.8 against new poetry.lock: 25 findings -> 0 - Unit tests (Python 3.10.18, default deps): 765 passed, 4 skipped - Unit tests (Python 3.10.18, min deps): 765 passed, 4 skipped - mypy: same 11 pre-existing errors on main; unchanged - black: applied separately in next commit Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Mechanical commit, no semantic changes. black 26.x changed its line-wrapping rules for several common patterns (trailing-comma handling in long function calls, kwarg layout in multi-line calls). Applied across 13 src files. Run locally with `poetry run black src` against black 26.5.1. Isolated from the dep-bump commit so future bisect/blame can skip past the reformat. Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
vikrantpuppala
had a problem deploying
to
azure-prod
GitHub Actions
Failure
Three independent fixes surfaced by the dep bump's pytest 7 → 9 jump:
1) pytest 9 + xdist 3.8 serializes subTest parametrization across worker
process boundaries via execnet. Class objects (thrift response types)
and thrift instances are not pickleable through execnet's restricted
serializer. Replace them with string labels:
- tests/unit/test_thrift_backend.py:
* resp_type / exec_resp_type → resp_type.__name__
* op_state_resp thrift instance → ("error_resp"/"closed_resp", inst) tuple
* error_resp thrift instance → f"resp_{i}" index label
- tests/unit/test_endpoint.py:
* CloudType enum → .value
Net effect: every subTest now passes only str/int/None to the subTest
keyword args. Test coverage is unchanged — the labels still uniquely
identify which subtest failed.
2) urllib3 2.x adds a `version_string` attribute that retry handlers
read off the response object. tests/e2e/common/retry_test_mixins.py's
SimpleHttpResponse mock missed this attribute; add it.
3) pyjwt 2.13.0's stricter type signatures expose a pre-existing mypy
`Incompatible return value type` warning in oauth.py — `exp_time and
(exp_time - buffer_time) <= current_time` returns Any|None, not bool.
Wrap exp_time in bool() to satisfy mypy without changing semantics
(None and False are both falsy; result is identical for all inputs).
Also updates poetry.lock for the pytest line (`^8.4.0` -> `^9.0.3`
restoration; intermediate downgrade was an investigation step, now
reverted).
Local verification on Python 3.10.18:
poetry run pytest tests/unit/ -n auto 765 passed, 4 skipped
~/osv-scanner scan source --lockfile poetry.lock No issues found
Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
Previous fix wrapped exp_time in bool() to satisfy mypy's [return-value]
rule. That didn't narrow the type, so mypy then flagged the
`(exp_time - buffer_time)` expression with `[operator]` "Unsupported
operand types for - ('None' and 'int')" — the short-circuit at runtime
is correct, but mypy can't track narrowing through a bool() call.
Replace with explicit `if exp_time is None: return False`. Same runtime
semantics (None and falsy exp_time both produce False), cleaner reading,
and mypy narrows the rest of the function to int after the guard.
Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
poetry.lock.^3.8.0→^3.10and the version to5.0.0(major, breaking — see below).This is the dep-bump PR the description of #798 (OSV-Scanner workflow) flagged as a follow-up. Once this merges and #798 wires the gate into branch protection, the gate becomes enforced.
What changed
Direct runtime dep bumps (in
pyproject.toml)urllib3>=1.26>=2.7.0,<3.0.0requests^2.18.1^2.33.0pyjwt^2.0.0^2.12.0pyarrow>=23.0.1(all three Python-version blocks)Direct dev dep bumps
pytest^7.1.2^9.0.3black^22.3.0^26.3.1Transitive cleanups (lockfile-only, no pyproject change)
cryptographyidnapython-dotenv(dev)Python support
python = "^3.8.0"→python = "^3.10"(drops 3.8, 3.9)["3.9","3.10","3.11","3.12","3.13","3.14"]→["3.10","3.11","3.12","3.13","3.14"](all four jobs)Why drop 3.9? The patched versions of
urllib3,pyarrow,requests,pytest, andblackall declarerequires_python>=3.10. Keeping 3.9 in the matrix would cause the resolver to silently install CVE-vulnerable older versions for 3.9 customers — defeating the purpose of the PR. Python 3.9 EOL was 2025-10-31, so 3.9 is also 7 months past upstream support.Source changes
black 26.5.1(isolated in a separate commit for clean blame). No logic changes.Why a major version bump (5.0.0)?
urllib3<2cannot installThe Python ecosystem strongly conventions transitive pins; signalling these breaks via SemVer-major is the only way customers know to re-evaluate their dep pins. A minor bump would let them silently stay on 4.x with CVEs unresolved.
Verification
poetry.lock: 25 findings → 0defaultdeps): 765 passed, 4 skippedmindeps): 765 passed, 4 skippedblack --check src: cleanmypy: same 11 pre-existing errors asmain. None introduced by this PR.Test plan
black --checkandmypypass on all Python versions (same baseline asmain)pip install databricks-sql-connector==5.0.0on Python 3.9 fails with a clear error messageMigration notes for the CHANGELOG / users
For most users,
pip install -U databricks-sql-connectoris the only required change.urllib3<2,pyarrow<23,pyjwt<2.12, orrequests<2.33must lift the pin first.This pull request and its description were written by Isaac.