Skip to content

Commit b005762

Browse files
committed
Set default --max-requests for API workers to prevent unbounded RSS growth
API workers currently run with max_requests=0 (unlimited lifetime), which means glibc heap fragmentation accumulates indefinitely and RSS grows without bound (~1 kB/request from normal Django ORM alloc/dealloc churn). Set max_requests=10000 and max_requests_jitter=500 as defaults for PulpApiWorker when gunicorn's effective max_requests is still 0 and the user did not pass --max-requests on the pulpcore-api CLI. An explicit --max-requests 0 disables recycling (gunicorn semantics) and is not overridden. Workers are gracefully recycled after ~10000 requests, resetting fragmented heap memory. Jitter prevents all workers from restarting simultaneously. Documented in docs/admin/learn/architecture.md. closes #7482 Assisted-by: Claude (Anthropic) Made-with: Cursor
1 parent 3cd99e7 commit b005762

3 files changed

Lines changed: 17 additions & 0 deletions

File tree

CHANGES/7482.bugfix

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Set default ``--max-requests 10000`` and ``--max-requests-jitter 500`` for API workers
2+
to prevent unbounded RSS growth from glibc heap fragmentation over long-lived worker processes.

docs/admin/learn/architecture.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,14 @@ Pulp's REST API is a Django application that runs standalone using the `gunicorn
2323
A simple way to run the REST API as a standalone service is using the provided `pulpcore-api`
2424
entrypoint. It is `gunicorn` based and provides many of its options.
2525

26+
!!! note "API worker recycling"
27+
By default, `pulpcore-api` enables gunicorn ``--max-requests`` and ``--max-requests-jitter`` so
28+
worker processes are periodically replaced. That limits memory growth from allocator
29+
fragmentation in long-lived workers. Override via gunicorn's usual mechanisms (CLI flags,
30+
``GUNICORN_CMD_ARGS``, or a config file). To **disable** recycling and keep unlimited worker
31+
lifetime, pass ``--max-requests 0`` on the ``pulpcore-api`` command line (gunicorn treats
32+
``0`` as unlimited; Pulp only applies its own defaults when ``--max-requests`` was not passed
33+
there). Disabling recycling is not recommended for production.
2634

2735
The REST API should only be deployed via the `pulpcore-api` entrypoint.
2836

pulpcore/app/entrypoint.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,6 +124,13 @@ def load_app_specific_config(self):
124124
PulpApiWorker.__module__ + "." + PulpApiWorker.__qualname__,
125125
enforced=True,
126126
)
127+
# Gunicorn's default for max_requests is 0 (unlimited worker lifetime). Apply Pulp defaults
128+
# only when the user did not pass --max-requests on the pulpcore-api CLI. An explicit
129+
# --max-requests 0 means "disable recycling" and must not be replaced.
130+
if self.cfg.max_requests == 0 and self.options.get("max_requests") is None:
131+
self.cfg.set("max_requests", 10000)
132+
if self.options.get("max_requests_jitter") is None:
133+
self.cfg.set("max_requests_jitter", 500)
127134

128135
def load(self):
129136
using_pulp_api_worker.set(True)

0 commit comments

Comments
 (0)