WebDAV returns empty body for open-ended range request (bytes=0-)

[mzur]

Hi everyone, I may have discovered a bug and would like to know if you can confirm this or if I'm still doing something wrong.

Background

I'm working on an integration between a web application and the DESY instance of dCache. The web application accesses dCache via WebDAV and displays images or videos. For faster loading, it requests macaroons, so the browser can directly load the files from dCache without knowing the original access credentials. When used as the src of a HTML video element, the browser handles the request and usually uses range requests to fetch parts of the video.

The bug

The initial request to fetch the start of the video is usually done with the Range: bytes=0- header because the browser does not know the length of the video yet. However, when sent open-ended like this, dCache seems to return an empty body. This is also the case with open-ended ranges from the middle of the file (e.g. Range: bytes=12345-). When I manually add any end to the range (e.g. Range: bytes=0-1023) the same request returns data in the body.

I also see this with public files, so I don't expect this to be an issue with the macaroon.

How to reproduce

Here is an example cURL request with output:

$ curl -v 'https://dcache.host:2880/path/to/video.mp4?authz=MACAROON' \
   -H 'Range: bytes=0-' \
   -o -

> GET /path/to/video.mp4?authz=MACAROON HTTP/1.1
> Host: dcache.host:2880
> Range: bytes=0-
> Connection: keep-alive
> 
< HTTP/1.1 206 Partial Content
< Date: Thu, 11 Dec 2025 08:45:46 GMT
< Server: dCache/9.2.35
< Content-Range: bytes 0-172889459/172889460
< ETag: "..."
< Content-Disposition: attachment
< Content-Length: 172889460
< 
* transfer closed with 172889460 bytes remaining to read
* Closing connection
curl: (18) transfer closed with 172889460 bytes remaining to read

The same request with an end in the range responds with data:

$ curl -v 'https://dcache.host:2880/path/to/video.mp4?authz=MACAROON' \
   -H 'Range: bytes=0-1023' \
   -o -

> GET /path/to/video.mp4?authz=MACAROON HTTP/1.1
> Host: dcache.host:2880
> Range: bytes=0-1023
> Connection: keep-alive
> 
< HTTP/1.1 206 Partial Content
< Date: Thu, 11 Dec 2025 08:44:37 GMT
< Server: dCache/9.2.35
< Content-Range: bytes 0-1023/172889460
< ETag: "..."
< Content-Disposition: attachment
< Content-Length: 1024
< 
 ftypisomisomiso2avc1mp41H�moovlmvhd�IP@G�trak\tkhdIP@�8$edtselstIPGsmdia mdhd2D�-hdlrvideVideoHandlerGminfvmhd$dinfdref
                                                                                                                        url F�stbl�stsd�avc1�8HH��4avcCd(��d(��@x'��D�<`�Xh���"�paspbtrt��3��3stts�Tstss:3,%  �	�
�


�cttsa%* Connection #0 to host localhost left intact

[paulmillar]

Hi @mzur,

Thanks for reporting this problem.

DESY actually runs multiple dCache instances, each tailored for specific user community.

From the context, I believe you're using the dCache InfiniteSpace service. DESY makes this storage available as part of the Helmholtz Cloud portfolio of services.

Just to mention it, the preferred way to provide feedback for HIFIS services is through the HIFIS ticketing system: support@hifis.net.

However, back to the problem at hand. I can confirm that this problem exists. A slightly easier way to reproduce the problem is with

paul@monkeywrench:~$ curl -H "Range: bytes=0-" https://hifis-storage.desy.de/ATTENTION-READ.txt
curl: (18) transfer closed with 62 bytes remaining to read
paul@monkeywrench:~$ 

This example validates your statement that it works with public files, so the macaroon isn't the cause of the problem.

Here's the same request without a byte-range:

paul@monkeywrench:~$ curl https://hifis-storage.desy.de/ATTENTION-READ.txt
Web Frontend has moved to:
https://hifis-storage-web.desy.de/
paul@monkeywrench:~$ 

The problem appears to be present in all supported releases of dCache. Verified with "system-test" on 9.2.35 and the tip of master branch.

The problem also seems to be limited to proxied transfers.

The redirecting door doesn't suffer from this problem:

paul@monkeywrench:~$ curl -L -H "Range: bytes=0-" https://dcache-doma-frontend01.desy.de:2880/ATTENTION-READ.txt
Web Frontend has moved to:
https://hifis-storage-web.desy.de/
paul@monkeywrench:~$ 

(also verified with system-test)

Using a byte range with generated output (e.g., HTML directory rendered output) also works without a problem:

paul@monkeywrench:~$ curl -sH "Range: bytes=0-" https://hifis-storage.desy.de/ | head -10
    <!DOCTYPE html>
    <html lang="en">
      <head>
          <meta charset="utf-8">
          <meta http-equiv="X-UA-Compatible" content="IE=edge">
          <meta name="viewport" content="width=device-width, initial-scale=1">

          <title>dCache File System</title>

          <link rel="stylesheet" href="/.webdav/css/font-awesome.min.css" rel="stylesheet">
paul@monkeywrench:~$ 

[mzur]

Just to mention it, the preferred way to provide feedback for HIFIS services is through the HIFIS ticketing system: support@hifis.net.

Sorry, this wasn't clear to me from the contributing guide.

I submitted a fix in #7973

[kofemann]

@paulmillar the fix is merged. Do we need to backport to 10.2?

[paulmillar]

Hi @kofemann,

HIFIS storage is currently on dCache 9.2:

paul@monkeywrench:~$ curl -sI https://hifis-storage.desy.de/|grep Server
Server: dCache/9.2.35
paul@monkeywrench:~$ 

So, backporting to 9.2 would be helpful.

Cheers,
Paul.