Regression: "Could not calculate [Algorithm] signature" error during file upload in Docker local environments (v6.10+)

[Koalax3]

What steps does it take to reproduce the issue?
Run Dataverse in a local Docker environment using the standard docker-compose configuration.

Use any version from 6.10, 6.10.1 up to 7.x.

Create a dataset and attempt to upload a file (e.g., dorastar.png) as a dataset constituent.

Note: The issue does not occur when adding the same image as a Dataverse collection thumbnail (vignette).

When does this issue occur?
Every time a file is added to a dataset in a local Docker setup. It seems to fail during the checksum calculation (MD5/SHA) immediately after the file is written to the temporary or final storage location.

Which page(s) does it occur on?
Dataset Page (Upload)

File API

What happens?
The upload fails with: [SEVERE] Could not calculate [Algorithm] signature for the new file [filename].
The file is not attached to the dataset. Curiously, this only happens on local Docker installs; deployed environments (production/staging) do not exhibit this behavior.

To whom does it occur?
All users on local Docker development environments.

What did you expect to happen?
The checksum should be calculated successfully and the file attached, as it was in version 6.9, where the issue is non-existent.

Which version of Dataverse are you using?
Broken on: 6.10, 6.10.1, 7.x (Payara 7)

Working on: 6.9

Any related open or closed issues to this bug report?
To be determined. It may be related to changes in edu.harvard.iq.dataverse.util.FileUtil or updates in the Jakarta EE / Payara underlying libraries handling I/O streams.

Screenshots / Logs:
Plaintext
2026-04-09T15:28:28.822+0000|WARNING|Payara...|edu.harvard.iq.dataverse.util.FileUtil|...|Could not calculate MD5 signature for the new file dorastar.png
Are you thinking about creating a pull request for this issue?
No.

Contact Information:
This issue was identified by the INIST (CNRS, France) team. For further technical details or reproduction environment specifics, please contact: lucas.depret@inist.fr

[pdurbin]

@Koalax3 hi! Thanks for the bug report.

I'm on 5dc8b47 (develop branch, post 6.10.1) and I tried to reproduce this with mvn test -Dtest=FilesIT#test_001_AddFileGood which uploads a png via API. I didn't see the error you're reporting.

Then I uploaded another png through the web interface. Again, no error. Here's how the UI looks:

In the Payara logs, I see normal message like this:

dev_dataverse>   Incrementing recorded storage use by 392605 bytes for dataset 5|#]

I'm using Docker, but I'm using the developer setup described at https://guides.dataverse.org/en/6.10.1/developers/dev-environment.html#quickstart

It that what you're using? Or are you using the other Docker compose file described at https://guides.dataverse.org/en/6.10.1/container/running/demo.html#quickstart ?

If it's easier, please feel free to start a topic in #containers at https://chat.dataverse.org about this! Also, we have a container meeting next week, if you'd like to join: https://www.gdcc.io/working-groups/containerization.html

[Koalax3]

I have yet problem in v6.9 **Detailed Reproduction Steps** Clean Start: Launch a Dataverse instance in a local Docker environment (standard docker-compose).

Initial Upload: Create a new dataset and upload an initial file (e.g., datastar.png).

Observed behavior: The file is uploaded successfully, and the MD5 is calculated without issue.

Publication: Publish the dataset (Major or Minor version).

Post-Publication Upload: Immediately after the dataset is published, attempt to upload a second, different file (e.g., maskgroup.png) to create a new Draft version.

Observed behavior: The upload fails with the error: [SEVERE] Could not calculate [Algorithm] signature for the new file maskgroup.png.

System Contamination: From this point forward, the error persists and "contaminates" the entire instance. Any attempt to upload a file to any other dataset (new or existing) will result in the same MD5/Signature calculation error.

Technical Observations
This issue was initially thought to be a regression in v6.10/v7.x, but further testing confirmed it also occurs on v6.9 under these specific conditions.

The error only manifests in local Docker environments. Deployed/Production instances do not seem to exhibit this behavior.

Adding a "Vignette" (Collection thumbnail) still works perfectly even when the Dataset file upload is broken, suggesting the issue is specific to the DataFile storage logic/driver (file1) and not the entire Java I/O subsystem.

The fact that it spreads to all datasets suggests a shared service or a Singleton (like FileUtil or the Storage IO stream) is getting stuck in an error state after a publication event.

[Koalax3]

update :
Steps to reproduce:

Clean Start: Run a Dataverse instance in a local Docker environment (standard docker-compose).

Initial Upload: Create a new dataset and upload an initial file (e.g., datastar.png).

Status: Success.

Publication: Publish the dataset (Major or Minor version).

Post-Publication Upload: Attempt to upload a second file (e.g., maskgroup.png) to the same dataset or any other dataset.

Status: Failure. Error: [SEVERE] Could not calculate [Algorithm] signature.

The "Restart" Test: Restart the Dataverse container (docker-compose restart dataverse).

Retry Upload: Attempt the upload again.

Status: Success (until the next publication, which triggers the bug again).

Conclusion: The publication process leaves the storage service in a corrupted/locked state. Only a restart of the application server clears this state.

docker-compose.yml

...
dataverse:
    container_name: "dataverse"
    hostname: dataverse
    image: gdcc/dataverse:6.9-noble
    restart: on-failure
    user: payara
    environment:
      DATAVERSE_SITEURL: "http://${MACHINE_IP:-localhost}:8080"
      DATAVERSE_DB_HOST: postgres
      DATAVERSE_DB_PASSWORD: **
      DATAVERSE_DB_USER: **
      DATAVERSE_FEATURE_API_BEARER_AUTH: "1"
      DATAVERSE_MAIL_SYSTEM_EMAIL: "Demo Dataverse <dataverse@example.org>"
      DATAVERSE_MAIL_MTA_HOST: "***"
      JVM_ARGS:
        -Ddataverse.files.storage-driver-id=filedv
        -Ddataverse.files.filedv.type=file
        -Ddataverse.files.filedv.label=Filesystem
        -Ddataverse.files.filedv.directory=${STORAGE_DIR}/store
        -Ddataverse.pid.providers=fake
        -Ddataverse.pid.default-provider=fake
        -Ddataverse.pid.fake.type=FAKE
        -Ddataverse.pid.fake.label=FakeDOIProvider
        -Ddataverse.pid.fake.authority=${PREFIX}
        -Ddataverse.lang.directory=${STORAGE_DIR}/lang
        -Ddataverse.api.blocked.policy=unblock-key
        -Ddataverse.api.blocked.key=${BLOCKED_API_KEY}
        -Ddataverse.api.blocked.endpoints=admin,builtin-users
    ports:
      - "8080:8080" # HTTP (Dataverse Application)
      - "4848:4848" # HTTP (Payara Admin Console)
      - "9009:9009" # JDWP
      - "8686:8686" # JMX
    networks:
      - dataverse
    depends_on:
      - postgres
      - solr
      - dv_initializer
    volumes:
      - ./data/app/data:${STORAGE_DIR}
      - ./data/app/secrets:/secrets
      - ./styles:/var/www/dataverse/branding
      - ./langs/fr_FR:${STORAGE_DIR}/lang
    tmpfs:
      - /dumps:mode=770,size=2052M,uid=1000,gid=1000
      - /tmp:mode=770,size=2052M,uid=1000,gid=1000
    mem_limit: 2147483648 # 2 GiB
    mem_reservation: 1024m
    privileged: false
...

[pdurbin]

Post-Publication Upload: Immediately after the dataset is published, attempt to upload a second, different file (e.g., maskgroup.png) to create a new Draft version.

I just tried this (same dev env and commit as above) but I'm not seeing the error.

Done uploading file 1 and publishing

Uploading file 2

No error uploading file 2

"View differences" shows file 2 was uploaded

Payara log

dev_dataverse>   Incrementing recorded storage use by 16221 bytes for dataset 9|#]

@Koalax3 if you want get on a Zoom call, please let me know! I'm sure there's a bug in here somewhere! 🐞 😅