Sysbox fails to apply soft file limit if the hard limit is capped

[ulfjack]

This is with docker-ce=5:28.2.2-1debian.11bullseye, sysbox=0.6.7.linux on a Ubuntu 24.04 box, although I also confirmed it with sysbox 0.7.0.linux. This works:

$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=12345:12345 ubuntu:latest cat /proc/1/limits
Max open files            12345                12345                files     
$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=200000:524280 ubuntu:latest cat /proc/1/limits
Max open files            200000               524280               files     

This doesn't work:

$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=524288:524288 ubuntu:latest cat /proc/1/limits
Max open files            1024                 524288               files     
$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=524289:524289 ubuntu:latest cat /proc/1/limits
Max open files            1024                 524288               files     
$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=12345:524289 ubuntu:latest cat /proc/1/limits
Max open files            1024                 524288               files     
$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=12346:524288 ubuntu:latest cat /proc/1/limits
Max open files            1024                 524288               files     
$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=524280:524288 ubuntu:latest cat /proc/1/limits
Max open files            1024                 524288               files     
$ docker run --runtime=sysbox-runc --rm -it --ulimit nofile=200000:524288 ubuntu:latest cat /proc/1/limits
Max open files            1024                 524288               files     

By comparison, running without sysbox applies the limit as expected:

$ docker run --rm -it --ulimit nofile=524280:524280 ubuntu:latest cat /proc/1/limits
Max open files               524280               524280               files

I can't find any reason why this would happen, and I looked through the sysbox source code, but that's the behavior I see on this machine.

[ulfjack]

We believe this is due to Golang doing bad things with ulimits. As a workaround, set the soft limit to a smaller value. Since this works in upstream Docker, I suspect that they had the same issue and implemented a fix that sysbox should also use.