Skip to content

Commit 09232a5

Browse files
pirateclaude
andcommitted
CI: per-user mine timeout + lower max-api-fetches
At ~30-90 min cold-mine per user, an 80-user queue would take 40-100 hours. Bound each user to 25 min wallclock via 'timeout --kill-after =30s 1500 python3 ...' so a single repo-heavy account can't block the queue. Partial state is already deployed via the watcher's interim deploys, so a timed-out user still shows up with whatever phase it reached. Also drop max-api-fetches from 1500 to 800. Most of the marginal fetches are PR-detail / commit-stats that only matter for the rarer "PR-only repo" lines counts — losing some completeness here is fine for a first-pass prepopulate. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent b7d2c59 commit 09232a5

1 file changed

Lines changed: 10 additions & 3 deletions

File tree

.github/workflows/mine-and-deploy.yml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -201,20 +201,27 @@ jobs:
201201
done
202202
}
203203
204+
# Per-user wallclock cap. Some users have hundreds of repos and
205+
# cold-mining them takes forever; bounding to 25min/user keeps
206+
# the queue moving (partial data already deployed via the
207+
# watcher's interim deploys).
208+
USER_TIMEOUT=1500
204209
while IFS= read -r user || [ -n "$user" ]; do
205210
user="${user%%#*}"
206211
user="${user//[[:space:]]/}"
207212
[ -z "$user" ] && continue
208213
[ "$user" = "pirate" ] && continue
209214
echo "::group::Mining @$user"
210215
# Run mining in the background; watch loop deploys partials.
211-
python3 generate_stats.py --user "$user" \
216+
timeout --kill-after=30s "$USER_TIMEOUT" python3 \
217+
generate_stats.py --user "$user" \
212218
--no-search-commits \
213-
--max-api-fetches 1500 &
219+
--max-api-fetches 800 &
214220
MINE_PID=$!
215221
watch_and_deploy "$MINE_PID" "$user" &
216222
WATCH_PID=$!
217-
wait "$MINE_PID" || echo "::warning::mining @$user exited non-zero"
223+
wait "$MINE_PID" \
224+
|| echo "::warning::mining @$user exited non-zero (timeout or error)"
218225
# Stop the watcher and do a final deploy with the final HTML.
219226
kill "$WATCH_PID" 2>/dev/null || true
220227
wait "$WATCH_PID" 2>/dev/null || true

0 commit comments

Comments
 (0)