bugfix/gracefully-handle-large-thumbnails #787
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: User report: mapillary_tools crashes when trying to upload the attached set of images. Also, there is no mention on which image is causing the crash, meaning the user has to hunt through potentially thousands of images to find the misbehaving one. Traceback (most recent call last): File "main.py", line 8, in <module> File "mapillary_tools\commands\__main__.py", line 164, in main File "mapillary_tools\commands\process_and_upload.py", line 33, in run File "mapillary_tools\commands\upload.py", line 75, in run File "mapillary_tools\upload.py", line 117, in upload File "mapillary_tools\upload.py", line 108, in upload File "mapillary_tools\upload.py", line 631, in _continue_or_fail File "mapillary_tools\uploader.py", line 517, in upload_images File "mapillary_tools\uploader.py", line 543, in _upload_sequence File "concurrent\futures\_base.py", line 619, in result_iterator File "concurrent\futures\_base.py", line 317, in _result_or_cancel File "concurrent\futures\_base.py", line 449, in result File "concurrent\futures\_base.py", line 401, in __get_result File "concurrent\futures\thread.py", line 59, in run File "mapillary_tools\uploader.py", line 602, in upload File "mapillary_tools\uploader.py", line 643, in dump_image_bytes File "mapillary_tools\exif_write.py", line 201, in dump_image_bytes File "mapillary_tools\exif_write.py", line 186, in _safe_dump File "mapillary_tools\exif_write.py", line 153, in _safe_dump File "piexif\_dump.py", line 94, in dump ValueError: Given thumbnail is too large. max 64kB [PYI-9472:ERROR] Failed to execute script 'main' due to unhandled exception! "But I guess the actual problems are quite clear: (1) ValueError: Given thumbnail is too large. max 64kB, Mapillary needs to gracefully handle that situation and (2) on file errors, Mapillary needs to report the failing image name." Test Plan: Added unit tests, ran the existing ones as described in README. It looks like the tests get stuck on tests/unit/test_persistent_cache.py::test_multithread_shared_cache_comprehensive but I believe this is unrelated to the change. To reproduce test getting stuck run: ``` pytest -s -vv tests ``` and observe it getting stuck after 181 tests.
caglarpir
requested changes
Nov 27, 2025
Author
|
As part of the validation, ran formatter, which was not ran before. Since the formatting changes are trivial, I kept it in the same commit as the PR follow up comments. |
caglarpir
reviewed
Nov 27, 2025
bal-a
force-pushed
the
bal/bugfix-thumbnail
branch
from
c0d8465 to
6b2a36d
Compare
bal-a
commented
Dec 9, 2025
| test_exif.copy(test_exif2) | ||
| test_exif3 = setup_unittest_data.join("test_exif_3.jpg") | ||
| test_exif.copy(test_exif3) | ||
| fixed_exif = setup_unittest_data.join("fixed_exif.jpg") |
Author
There was a problem hiding this comment.
Linter complaining, not used, deleted
bal-a
commented
Dec 9, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
User report:
mapillary_tools crashes when trying to upload the attached set of images.
Also, there is no mention on which image is causing the crash, meaning the user has to hunt through potentially thousands of images to find the misbehaving one.
Traceback (most recent call last):
File "main.py", line 8, in
File "mapillary_tools\commands_main_.py", line 164, in main File "mapillary_tools\commands\process_and_upload.py", line 33, in run File "mapillary_tools\commands\upload.py", line 75, in run File "mapillary_tools\upload.py", line 117, in upload File "mapillary_tools\upload.py", line 108, in upload File "mapillary_tools\upload.py", line 631, in _continue_or_fail File "mapillary_tools\uploader.py", line 517, in upload_images File "mapillary_tools\uploader.py", line 543, in _upload_sequence File "concurrent\futures_base.py", line 619, in result_iterator File "concurrent\futures_base.py", line 317, in _result_or_cancel File "concurrent\futures_base.py", line 449, in result File "concurrent\futures_base.py", line 401, in __get_result File "concurrent\futures\thread.py", line 59, in run File "mapillary_tools\uploader.py", line 602, in upload File "mapillary_tools\uploader.py", line 643, in dump_image_bytes File "mapillary_tools\exif_write.py", line 201, in dump_image_bytes File "mapillary_tools\exif_write.py", line 186, in _safe_dump File "mapillary_tools\exif_write.py", line 153, in _safe_dump File "piexif_dump.py", line 94, in dump
ValueError: Given thumbnail is too large. max 64kB [PYI-9472:ERROR] Failed to execute script 'main' due to unhandled exception!
"But I guess the actual problems are quite clear: (1) ValueError: Given thumbnail is too large. max 64kB, Mapillary needs to gracefully handle that situation and (2) on file errors, Mapillary needs to report the failing image name."
Test Plan:
Added unit tests, ran the existing ones as described in README.
It looks like the tests get stuck on
tests/unit/test_persistent_cache.py::test_multithread_shared_cache_comprehensive
but I believe this is unrelated to the change.
To reproduce test getting stuck run:
and observe it getting stuck after 181 tests.