Dear dCache developers,
I'd like to ask you for a feature request on how log rotation is handled currently for dCache logs, in particular for access and billing logs.
According to my understanding, the rotation happens for access logs on a fixed, daily basis, while billing logs are generated as daily files. Both settings are not configurable. Among other configuration settings, the number of archives kept is configurable (only for access logs), allowing us as site admins only limited ability to tune the rotation.
I'd like you to ask for more flexibility for these kind of logs mainly in 2 ways:
- Introduce a flexible time period to rotate the logs with
- Allow as an alternative and/or in addition to the time-based rotation also the size-based rotation to cover cases when the log file exceeds certain size threshold.
The latter is of particular importance for our dCache operations at KIT, since we see occasionally, that there are bursts of accesses from various users (in particular for CMS experiment ...), such that our log directory runs full much faster than the daily log rotation.
Increasing the scratch space for the logs is not an elegant solution for this problem, and we would like to avoid assigning more disk space just for logs, when it can be used in a better way.
Also decreasing the log level is not an appropriate solution for us, since we would like to keep the access entries in those logs to be able to push them to our monitoring infrastructure (based on opensearch).
Attached, you can find a dCache codebase analysis (with LLM-support) for possible, minimal changes in that context.
Thank you very much in advance for consideration,
Best regards,
Artur for the KIT dCache admin team.
log-rotation-flexibility.md
Dear dCache developers,
I'd like to ask you for a feature request on how log rotation is handled currently for dCache logs, in particular for access and billing logs.
According to my understanding, the rotation happens for access logs on a fixed, daily basis, while billing logs are generated as daily files. Both settings are not configurable. Among other configuration settings, the number of archives kept is configurable (only for access logs), allowing us as site admins only limited ability to tune the rotation.
I'd like you to ask for more flexibility for these kind of logs mainly in 2 ways:
The latter is of particular importance for our dCache operations at KIT, since we see occasionally, that there are bursts of accesses from various users (in particular for CMS experiment ...), such that our log directory runs full much faster than the daily log rotation.
Increasing the scratch space for the logs is not an elegant solution for this problem, and we would like to avoid assigning more disk space just for logs, when it can be used in a better way.
Also decreasing the log level is not an appropriate solution for us, since we would like to keep the access entries in those logs to be able to push them to our monitoring infrastructure (based on opensearch).
Attached, you can find a dCache codebase analysis (with LLM-support) for possible, minimal changes in that context.
Thank you very much in advance for consideration,
Best regards,
Artur for the KIT dCache admin team.
log-rotation-flexibility.md