-
Notifications
You must be signed in to change notification settings - Fork 2.6k
ci: refactor/optimize Docker builds #10387
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
mpangrazzi
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
I was also thinking -as a future improvement- about using GH cache when it comes to build the image.
- name: Build base images
# ...
with:
# ...
set: |
*.cache-from=type=gha
*.cache-to=type=gha,mode=maxThis will store Docker build layers GA cache (we should have 10GB of free cache). So on subsequent builds, unchanged layers are pulled from cache instead of rebuilt.
Note: no need to do it now, it's just an idea and definitely not tested. It may speed up the build phase quite a bit though.
Great idea! I'll see if feasible |
Pull Request Test Coverage Report for Build 21062222416Details
💛 - Coveralls |
|
I did some attempts with caching, but I ended up not rebuilding the image when important files are changed, which is not desired. As you said, caching probably needs a better investigation. Skipping for now. |
Related Issues
I investigated Haystack Docker builds in relation to deepset-ai/hayhooks#199
I found out several optimization opportunities.
Proposed Changes:
stabletag for v2.Y.Z versions (needed for Use latest stable Haystack in Hayhooks Docker image hayhooks#199)xpdfand related libraries (only needed in Haystack 1.x - see fix: provide a fallback for PyMuPDF #4564) -> image size reduced from 303 MB to 248 MBHow did you test it?
I tried to use this PR for testing with some workarounds. See for example https://github.com/deepset-ai/haystack/actions/runs/21027111342/job/60454125178
The only aspect I could not fully test is the
stabletag: it requires the next Haystack release to be published.I also tested the built image locally, in isolation and with Hayhooks. Seems to work as always.
Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:and added!in case the PR includes breaking changes.