feat(doccano-django): real Python line coverage via coverage.py overlay

Replaces the prior API-route-surface "coverage" (which counted
fired routes / known routes — a proxy that read like real coverage
but didn't measure code execution) with actual line coverage via
coverage.py 7.6.1.

Architecture:
  - `Dockerfile.coverage` extends `doccano/doccano:backend` to
    install coverage[toml] and drop a `coverage_subprocess.pth`
    file into site-packages, so every gunicorn worker that forks
    auto-starts `coverage.process_startup()`.
  - `.coveragerc` runs in parallel mode (one .coverage.<pid> per
    worker) with sigterm = true so flushing happens on graceful
    shutdown.
  - `docker-compose.coverage.yml` is an OVERLAY: the GH Actions
    coverage workflow applies it via `-f docker-compose.yml -f
    docker-compose.coverage.yml`. The base `Dockerfile` and
    `docker-compose.yml` are untouched, so keploy/integrations and
    keploy/enterprise CI lanes consume the base compose and pay
    zero coverage-instrumentation cost.
  - `flow.sh::doccano_report_coverage` shells into the running
    backend, runs `coverage combine` + `coverage report
    --format=total`, emits `Covered N/M (XX.X%)` matching the
    helper script's regex. When called against the base image
    (no overlay) it prints "INFO: ... uninstrumented" and exits 0
    so enterprise lanes' `flow.sh coverage || true` informational
    calls keep working.

Removed:
  - `doccano_list_routes` (the Django URL-resolver walk).
  - `doccano_list_recorded_routes` (the keploy-tests / fired-routes
    reader).
  - The legacy route-surface `doccano_report_coverage` body.
  - `list-routes` subcommand (was diagnostic only for the surface
    metric).

Validated locally: e2e run produced `coverage=59.0` to
GITHUB_OUTPUT against a clean stack (gunicorn 4 workers, traffic
loop fired, SIGTERM flush, combine+report inside container).
59% reflects bootstrap + the sample's small traffic surface;
adding curls to flow.sh::doccano_record_traffic moves the
number up.

Signed-off-by: Akash Kumar <meakash7902@gmail.com>

Author: AkashKumar7902

.github/workflows/doccano-django.yml

@@ -175,8 +175,8 @@ jobs:
           if python3 -c "import sys; sys.exit(0 if (${RELEASE} - ${BUILD}) > ${THRESHOLD} else 1)"; then
             echo "::error::doccano-django coverage dropped from ${RELEASE}% → ${BUILD}% (-${drop}pp), exceeding the ${THRESHOLD}pp threshold."
             echo "Suggested actions:"
-            echo "  * Add curl(s) to flow.sh::doccano_record_traffic that exercise the routes you changed/touched."
-            echo "  * If the route(s) was intentionally retired, drop it from doccano-django/flow.sh::doccano_list_routes' SCOPE_PREFIXES too so it's removed from the denominator."
+            echo "  * Add curl(s) to flow.sh::doccano_record_traffic that exercise the new code paths."
+            echo "  * Or extend the .coveragerc 'omit' list if the new module is not part of the runtime backend (migrations, management commands, tests)."
             exit 1
           fi
           echo "OK — coverage delta within ${THRESHOLD}pp threshold."
@@ -196,4 +196,4 @@ jobs:
 
             Threshold: PR may not drop coverage by more than **${{ env.COVERAGE_THRESHOLD }}pp**. Override per-repo via the `DOCCANO_COVERAGE_THRESHOLD` actions variable.
 
-            Coverage measures the API surface (`/v1/projects/*` + `/v1/me` + `/v1/users` + `/v1/health` + `/v1/auth`) that `flow.sh::doccano_record_traffic` actually exercises against the running backend's URL resolver. Reports are attached as artifacts on each job ("coverage-build" / "coverage-release").
+            Coverage is **Python line coverage** (coverage.py 7.6.1) of the doccano backend code — the bytes that `flow.sh::doccano_record_traffic` actually executes. Instrumentation lives in a separate `Dockerfile.coverage` + `docker-compose.coverage.yml` overlay; the base `docker-compose.yml` consumed by keploy/integrations and keploy/enterprise CI lanes runs uninstrumented and pays zero coverage cost. Per-file reports are attached as artifacts on each job (`coverage-build` / `coverage-release`).

.github/workflows/scripts/run-and-measure.sh

@@ -1,54 +1,51 @@
 #!/usr/bin/env bash
 #
-# run-and-measure.sh — bring doccano up via the sample's compose,
-# run flow.sh bootstrap + record-traffic with the per-call audit
-# log enabled, run flow.sh coverage, and emit `coverage=PCT`
-# onto $GITHUB_OUTPUT for the downstream coverage-gate job.
+# run-and-measure.sh — bring doccano up under the coverage overlay,
+# run flow.sh bootstrap + record-traffic, flush coverage from each
+# gunicorn worker, run flow.sh coverage to combine + report, and
+# emit `coverage=PCT` onto $GITHUB_OUTPUT for the downstream
+# coverage-gate job.
 #
-# Called from .github/workflows/doccano-django.yml's
-# build-coverage and release-coverage jobs (one per ref under
-# comparison). Both jobs source the same script so the
-# measurement is identical across refs — any drift in the
-# numerator definition would otherwise produce a misleading
-# delta.
+# Called from .github/workflows/doccano-django.yml's build-coverage
+# and release-coverage jobs (one per ref under comparison). Both
+# jobs source the same script so the measurement is identical
+# across refs — any drift in the numerator definition would
+# otherwise produce a misleading delta.
 #
-# Inputs (all from the workflow env):
-#   DOCCANO_FIRED_ROUTES_FILE   — per-call audit log path; passed
-#                                 through to flow.sh so its
-#                                 record-traffic loop logs each
-#                                 (METHOD, URL) pair, and so its
-#                                 coverage subcommand uses that
-#                                 file as the standalone
-#                                 numerator.
-#   DOCCANO_PHASE               — label spliced into the project
-#                                 name so build vs. release runs
-#                                 don't collide on volume names
-#                                 (compose project naming inside
-#                                 the GH runner is per-job
-#                                 anyway, but DOCCANO_PHASE shows
-#                                 up in the test fixtures and
-#                                 is useful for diffing logs).
-#   GITHUB_OUTPUT               — standard GH Actions sink for
-#                                 step outputs.
+# Coverage isolation contract:
+#   * Base `Dockerfile` and `docker-compose.yml` are untouched.
+#   * The overlay `Dockerfile.coverage` + `docker-compose.coverage.yml`
+#     adds coverage.py + the auto-start .pth file. ONLY this script
+#     applies the overlay; the keploy/integrations and
+#     keploy/enterprise CI lanes consume the base compose and pay
+#     zero coverage-instrumentation cost.
+#
+# Inputs (from the workflow env):
+#   DOCCANO_PHASE     — label spliced into the project name so
+#                       build vs release runs don't collide.
+#   GITHUB_OUTPUT     — standard GH Actions sink for step outputs.
 set -Eeuo pipefail
 
 export DOCCANO_BACKEND_CONTAINER="${DOCCANO_BACKEND_CONTAINER:-doccano_backend}"
 export DOCCANO_DB_CONTAINER="${DOCCANO_DB_CONTAINER:-doccano_db}"
 export DOCCANO_APP_PORT="${DOCCANO_APP_PORT:-18080}"
 export DOCCANO_FIXED_TOKEN="${DOCCANO_FIXED_TOKEN:-ac38262065f0ae1476b6a707d9d697a101764a6b}"
-: "${DOCCANO_FIRED_ROUTES_FILE:?DOCCANO_FIRED_ROUTES_FILE must be set by the workflow}"
 
-# Reset audit log for this run; otherwise a prior run's entries
-# would inflate the numerator on a re-trigger.
-: >"$DOCCANO_FIRED_ROUTES_FILE"
+mkdir -p coverage
+chmod 777 coverage    # worker UID inside container differs from runner UID
+sudo rm -rf coverage/.coverage* 2>/dev/null || rm -rf coverage/.coverage* 2>/dev/null || true
+
+COMPOSE=(docker compose -f docker-compose.yml -f docker-compose.coverage.yml)
 
-# Stage 1: bring up doccano with bootstrap so the admin user +
-# fixed token persist into the named volume.
-DOCCANO_SKIP_BOOTSTRAP=0 docker compose up -d
+# Stage 1: bring up doccano with bootstrap so the schema migrations
+# and the admin user persist into the named DB volume. The overlay
+# image runs gunicorn with coverage.process_startup() auto-armed in
+# every forked worker.
+DOCCANO_SKIP_BOOTSTRAP=0 "${COMPOSE[@]}" up -d --build
 
-# Wait for the backend to start serving (not just port-open).
-# Cold doccano boot runs Django migrations + admin user create,
-# which on a GH runner can hit 90-120s.
+# Wait for the backend to start serving (cold doccano boot runs
+# Django migrations + admin user create — on a GH runner this can
+# hit 90-120s).
 for i in $(seq 1 120); do
     code=$(curl -sS -o /dev/null -w '%{http_code}' \
         "http://127.0.0.1:${DOCCANO_APP_PORT}/v1/health/" 2>/dev/null || echo "")
@@ -57,31 +54,49 @@ for i in $(seq 1 120); do
 done
 
 bash flow.sh bootstrap 240
-docker compose down --remove-orphans
+"${COMPOSE[@]}" down --remove-orphans
 
 # Stage 2: re-launch in skip-bootstrap mode against the populated
-# volume — same shape the keploy lanes use.
-DOCCANO_SKIP_BOOTSTRAP=1 docker compose up -d
+# volume; same shape the keploy lanes use. The overlay layer is
+# preserved across compose-down (only `down -v` would wipe the
+# named volume), so coverage tooling is still wired in.
+DOCCANO_SKIP_BOOTSTRAP=1 "${COMPOSE[@]}" up -d
 
-# Drive traffic. flow.sh::doccano_record_traffic gates on
-# doccano_wait_for_fixed_token internally, so this won't fire
-# curls at a half-booted backend.
+# flow.sh::doccano_record_traffic gates on doccano_wait_for_fixed_token
+# internally, so this won't fire curls at a half-booted backend.
 bash flow.sh record-traffic
 
-# Coverage report — uses DOCCANO_FIRED_ROUTES_FILE as numerator
-# since no keploy/test-set-* tree exists in the standalone case.
+# Flush coverage from each gunicorn worker. coverage.py with
+# sigterm = true writes the in-flight per-worker .coverage.<pid>
+# data file to /coverage on SIGTERM; `compose kill -s SIGTERM`
+# delivers it to the container's main process which propagates to
+# its workers via gunicorn's graceful shutdown.
+"${COMPOSE[@]}" kill -s SIGTERM backend
+# coverage.py's sigterm hook is synchronous but the OS-level
+# write+fsync needs a moment.
+sleep 3
+
+# Bring backend back up so `flow.sh coverage` can docker-exec
+# `coverage combine` + `coverage report` inside.
+"${COMPOSE[@]}" up -d backend
+for i in $(seq 1 60); do
+    if docker exec "$DOCCANO_BACKEND_CONTAINER" sh -c 'ls /coverage/.coverage.* >/dev/null 2>&1'; then
+        break
+    fi
+    sleep 1
+done
+
 COVERAGE_REPORT_FILE="$PWD/coverage_report.txt" bash flow.sh coverage
 
-# Pull the percentage out of the report's `Covered N/M (XX.X%)`
-# line. Anchored on the parenthesised form so a future change to
-# the report's prose doesn't break the parse.
+# Parse `Covered N/M (XX.X%)` — anchored on the parenthesised form
+# so a future report-prose change doesn't break the parse.
 pct=$(grep -oE '\([0-9]+\.[0-9]+%\)' coverage_report.txt | head -1 | tr -d '()%')
 if [ -z "$pct" ]; then
     echo "::error::Could not parse coverage percentage from coverage_report.txt"
     cat coverage_report.txt || true
     exit 1
 fi
 echo "coverage=${pct}" >>"$GITHUB_OUTPUT"
-echo "coverage: ${pct}% (audit log: $DOCCANO_FIRED_ROUTES_FILE)"
+echo "coverage: ${pct}% (Python line coverage via coverage.py)"
 
-docker compose down -v --remove-orphans
+"${COMPOSE[@]}" down -v --remove-orphans

doccano-django/.coveragerc

@@ -0,0 +1,21 @@
+[run]
+# Per-process line coverage of the backend Django code.
+#
+# parallel + sigterm: gunicorn forks WORKERS subprocesses; each
+# writes its own .coverage.<host>.<pid> file under /coverage.
+# `combine` merges them at report time. `sigterm = true` flushes
+# the in-flight data on SIGTERM so the reaper from the workflow
+# captures it.
+parallel = true
+sigterm = true
+branch = false
+data_file = /coverage/.coverage
+source = /backend
+
+omit =
+    */tests/*
+    */migrations/*
+    */__pycache__/*
+    /backend/manage.py
+    /backend/config/wsgi.py
+    /backend/config/asgi.py

doccano-django/.gitignore

@@ -0,0 +1,2 @@
+coverage/
+coverage_report.txt

doccano-django/Dockerfile.coverage

@@ -0,0 +1,37 @@
+# Coverage-instrumented variant of the doccano backend image.
+#
+# Base `Dockerfile` (and `docker-compose.yml`) are deliberately
+# untouched so the keploy enterprise / integrations lanes — which
+# consume them as-is — pay zero coverage-instrumentation cost. This
+# overlay image is built and run ONLY by the standalone GitHub
+# Actions workflow under `.github/workflows/doccano-django.yml`,
+# wired in via `docker-compose.coverage.yml`.
+#
+# What the overlay adds:
+#   * `coverage` (Python coverage.py) installed into the same
+#     site-packages as gunicorn / Django.
+#   * `.coveragerc` placed at /backend/.coveragerc — the working
+#     directory the upstream image starts gunicorn from. With
+#     `COVERAGE_PROCESS_START=/backend/.coveragerc` exported into
+#     the container env (set in the compose overlay), every
+#     gunicorn worker that imports `coverage.process_startup` via
+#     site-packages will pick the rcfile up; combined with `parallel
+#     = true` and `sigterm = true` in the rcfile, this gives us
+#     real per-worker line coverage that flushes on SIGTERM.
+FROM doccano/doccano:backend
+
+USER root
+RUN pip install --no-cache-dir 'coverage[toml]==7.6.1'
+
+# Subprocess auto-start: a .pth file in site-packages is processed
+# at every Python startup, so each gunicorn worker that forks calls
+# coverage.process_startup() before any Django code runs. This is
+# the canonical way coverage.py instruments forked subprocesses
+# (see "Measuring sub-processes" in the coverage.py docs).
+RUN echo 'import coverage; coverage.process_startup()' \
+    > /usr/local/lib/python3.10/site-packages/coverage_subprocess.pth
+
+COPY .coveragerc /backend/.coveragerc
+RUN mkdir -p /coverage \
+    && chown -R doccano:doccano /coverage /backend/.coveragerc
+USER doccano

doccano-django/docker-compose.coverage.yml

@@ -0,0 +1,19 @@
+# Coverage overlay — applied with:
+#
+#   docker compose -f docker-compose.yml -f docker-compose.coverage.yml up -d --build
+#
+# Used ONLY by the standalone .github/workflows/doccano-django.yml
+# CI workflow. Keploy CI lanes (enterprise, integrations) ignore
+# this file and run the base compose unchanged, so they pay zero
+# coverage-instrumentation cost.
+services:
+  backend:
+    build:
+      context: .
+      dockerfile: Dockerfile.coverage
+    environment:
+      # Tells the .pth-file-installed coverage.process_startup() to
+      # actually start tracking; see Dockerfile.coverage.
+      COVERAGE_PROCESS_START: /backend/.coveragerc
+    volumes:
+      - ./coverage:/coverage