Skip to content

Conversation

@shahdyousefak
Copy link
Contributor

@shahdyousefak shahdyousefak commented Jun 19, 2025

This PR addresses a few issues, #1091 + beyond.

issue with multilang isnt just the torch v..
25-06-19 19:12 UTC [WARNING]: Cannot find /app/models/opus-mt-en-fr, is it pre-downloaded?
25-06-19 19:12 UTC [INFO]: Downloading model from HuggingFace model hub...
25-06-19 19:12 UTC [ERROR]: Could not find MarianMTModel neither in <module 'transformers.models.marian' from '/opt/conda/lib/python3.10/site-packages/transformers/models/marian/init.py'> nor in <module 'transformers' from '/opt/conda/lib/python3.10/site-packages/transformers/init.py'>

the Hugging Face transformers lib was attempting to import MarianMTModel, which triggers loading modules from torchvision, but the version of torchvision bundled in pytorch/pytorch:latest (which is the base image we’re using) is incompatible with the latest transformers.

  • removed git clone logic inside Dockerfile, switching to runtime model loading using Hugging Face’s transformers API -> also reduces Docker image size.
  • Updated requirements.txt and upgraded torch.
  • Pinned base image. now a stable, up-to-date version.
  • added a basic warmup endpoint (and the warmup flag to the docker compose), which didnt go into Warmup script to run warmup across LLM-based preprocessors after they are healthy #1075 due to multilang-support service being broken.
[2025-06-19 20:03:14 +0000] [1] [INFO] Starting gunicorn 23.0.0
[2025-06-19 20:03:14 +0000] [1] [INFO] Listening at: http://0.0.0.0:80 (1)
[2025-06-19 20:03:14 +0000] [1] [INFO] Using worker: sync
[2025-06-19 20:03:14 +0000] [7] [INFO] Booting worker with pid: 7
25-06-19 20:03 UTC [INFO]: Loading model from Hugging Face: Helsinki-NLP/opus-mt-en-fr
25-06-19 20:03 UTC [INFO]: Translator(en, fr) instantiated!
25-06-19 20:03 UTC [INFO]: Translator(en, fr) running on NVIDIA GeForce GTX 1660 Ti
25-06-19 20:03 UTC [INFO]: Translation service is instantiated and ready!
25-06-19 20:03 UTC [DEBUG]: (1) Tokenizing input.
25-06-19 20:03 UTC [DEBUG]: (2) Generating tensor.
The following generation flags are not valid and may be ignored: ['temperature']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
25-06-19 20:03 UTC [DEBUG]: (3) Decoding tensor.
25-06-19 20:03 UTC [INFO]: Le service de traduction est instantané et prêt!
25-06-19 20:05 UTC [INFO]: [WARMUP] Warmup endpoint triggered.
25-06-19 20:05 UTC [INFO]: Loading model from Hugging Face: Helsinki-NLP/opus-mt-en-fr
25-06-19 20:05 UTC [INFO]: Translator(en, fr) instantiated!
25-06-19 20:05 UTC [INFO]: Translator(en, fr) running on NVIDIA GeForce GTX 1660 Ti
25-06-19 20:05 UTC [DEBUG]: (1) Tokenizing input.
25-06-19 20:05 UTC [DEBUG]: (2) Generating tensor.

Please ensure you've followed the checklist and provide all the required information before requesting a review.
If you do not have everything applicable to your PR, it will not be reviewed!
If you don't know what something is or if it applies to you, ask!

Don't delete below this line.


Required Information

  • I referenced the issue addressed in this PR.
  • I described the changes made and how these address the issue.
  • I described how I tested these changes.

Coding/Commit Requirements

  • I followed applicable coding standards where appropriate (e.g., PEP8)
  • I have not committed any models or other large files.

New Component Checklist (mandatory for new microservices)

  • [] I added an entry to docker-compose.yml and build.yml.
  • [] I created A CI workflow under .github/workflows.
  • [] I have created a README.md file that describes what the component does and what it depends on (other microservices, ML models, etc.).

OR

  • I have not added a new component in this PR.

@shahdyousefak shahdyousefak changed the title Multilang fix multilang-support Dockerfile: upgrade PyTorch, CUDA, cleanup, warmup endpoint Jun 19, 2025
@shahdyousefak shahdyousefak changed the title multilang-support Dockerfile: upgrade PyTorch, CUDA, cleanup, warmup endpoint multilang-support Dockerfile (upgrade PyTorch, CUDA,), cleanup, warmup endpoint Jun 19, 2025
@shahdyousefak shahdyousefak changed the title multilang-support Dockerfile (upgrade PyTorch, CUDA,), cleanup, warmup endpoint multilang-support Dockerfile (upgrade PyTorch, CUDA), cleanup, warmup endpoint Jun 19, 2025
@shahdyousefak shahdyousefak linked an issue Jun 19, 2025 that may be closed by this pull request
@shahdyousefak shahdyousefak requested a review from jeffbl June 19, 2025 20:26
Copy link
Member

@jeffbl jeffbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to limit the number of model downloads, other comments just tweaks.

@jeffbl jeffbl assigned shahdyousefak and unassigned jeffbl Jun 19, 2025
@shahdyousefak shahdyousefak requested a review from jeffbl June 19, 2025 21:55
Copy link
Member

@jeffbl jeffbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still surprised python3 needs to be installed, but no harm, should be a noop if not.

@jeffbl jeffbl merged commit fa760a5 into main Jun 19, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

multilang service fails due to bad torch version

3 participants