chore: pin deps to SHA256 hashes via pip-compile#4268
Open
luarss wants to merge 7 commits into
Open
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces lockfiles for managing Python dependencies across various configurations, including GitHub workflows, ReadtheDocs, and the dependency installer script, while also upgrading the Python build version to 3.10 on ReadtheDocs. Feedback on the changes highlights a portability issue in DependencyInstaller.sh, where the use of realpath may fail on macOS, and suggests a more robust alternative using cd and pwd.
| fi | ||
| local pkgs="pandas numpy firebase_admin click pyyaml yamlfix" | ||
| local lockfile | ||
| lockfile="$(dirname "$(realpath "$0")")/requirements-common_lock.txt" |
There was a problem hiding this comment.
Using realpath can fail on macOS and older BSD systems where it is not installed by default. Since this script explicitly supports macOS, it is safer and more portable to use cd and pwd to resolve the absolute path of the script's directory.
Suggested change
| lockfile="$(dirname "$(realpath "$0")")/requirements-common_lock.txt" | |
| lockfile="$(cd "$(dirname "$0")" && pwd)/requirements-common_lock.txt" |
luarss
force-pushed
the
topic/pin-deps-orfs
branch
2 times, most recently
from
d565232 to
5b6cab8
Compare
Replace unpinned docs/requirements.txt with a requirements.in source spec and a pip-compile-generated requirements_lock.txt with SHA256 hashes for all packages. Switch sphinx-external-toc from a git URL to its PyPI release so hashes can be verified. Update .readthedocs.yaml to use the lock file and bump Python from 3.7 (EOL) to 3.10. Signed-off-by: Jack Luar <jluar@precisioninno.com>
Add .github/requirements/tclint.in and a pip-compile-generated lock file with SHA256 hashes. Update the lint-tcl workflow to install from the lock file and pin Python to 3.10 via actions/setup-python. Signed-off-by: Jack Luar <jluar@precisioninno.com>
Add .github/requirements/yaml-test.in and a pip-compile-generated lock file with SHA256 hashes. Update the yaml-test workflow to install from the lock file, add the lock file to sparse-checkout, and pin Python to 3.10 via actions/setup-python. Also pins the previously unpinned pyyaml. Signed-off-by: Jack Luar <jluar@precisioninno.com>
Add .github/requirements/update-rules.in and a pip-compile-generated lock file with SHA256 hashes. Update the update-rules workflow to install from the lock file instead of unpinned firebase-admin. Signed-off-by: Jack Luar <jluar@precisioninno.com>
Add etc/requirements-common.in and a pip-compile-generated lock file with SHA256 hashes for all packages previously installed unpinned (pandas, numpy, firebase_admin, click, pyyaml, yamlfix). Update _installPipCommon() to install from the lock file. Signed-off-by: Jack Luar <jluar@precisioninno.com>
Merge the three per-workflow requirements files (tclint, yaml-test, update-rules) into a single .github/requirements/requirements.in and a shared requirements_lock.txt. All CI workflows now install from one place, reducing lock-file sprawl and making dependency updates a single-file change. Signed-off-by: Jack Luar <jluar@precisioninno.com>
DependencyInstaller.sh resolves the pip lockfile relative to its own directory, so requirements-common_lock.txt must be copied alongside it into /tmp/installer/etc/ inside the container. Signed-off-by: Jack Luar <jluar@precisioninno.com>
luarss
force-pushed
the
topic/pin-deps-orfs
branch
from
2406b27 to
078c489
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #4263
Summary
Pin all Python dependencies to SHA256 hashes using
pip-compile --generate-hashesfrom pip-tools. This mirrors the pattern already in place forflow/util/requirements_lock.txtand extends it to every other pip install location in the repo.Changes
docs/requirements.txt(all unpinned)requirements.in+requirements_lock.txt(SHA256-pinned)pip install(partially/fully unpinned).github/requirements/requirements_lock.txtshared by alletc/DependencyInstaller.shetc/requirements-common.in+requirements-common_lock.txtDetails
*.insource file (high-level constraints) and a generated*_lock.txtwith full SHA256 hashes for every package including transitive deps.docs/requirements.txt→docs/requirements.in: thegit+https://URL forsphinx-external-tocis replaced withsphinx-external-toc==0.3.1(same release, available on PyPI, hash-verifiable)..readthedocs.yaml: Python bumped from 3.7 (EOL) to 3.10; install target updated todocs/requirements_lock.txt.lint-tcl,yaml-test,update-rules) now share.github/requirements/requirements.inand a single lock file. Workflows that had no explicit Python version now pin to 3.10 viaactions/setup-python, consistent withupdate-rules.pip-compile --generate-hashes <file>.in -o <file>_lock.txt