Commit fe4a526
fix: switch TAC-based sg_only Dockerfiles from ubuntu base to TAC base image
Both bustub-hyperloglog-impl-001 and openhands-search-file-test-001 used
ubuntu:22.04 as their sg_only base image. The verifier for these tasks
runs /utils/eval.py via the python_default binary, both of which are only
present in the TAC base image — so the MCP verifier always crashed with
"Evaluator failed" and scored 0.
Root cause of MCP scoring 0.00 (while baseline scored >0):
bustub: BL=0.17 -> MCP=0.00 (sde-implement-hyperloglog-image missing)
openhands: BL=0.40 -> MCP=0.00 (sde-write-a-unit-test-*-image missing)
Fix: change FROM to the same TAC base image used in the baseline Dockerfile,
then add sg_only-specific layers:
- Truncate source files (agent must use Sourcegraph MCP)
- Recommit truncated state (prevents git history leaking code)
- Add clone manifest (clone-at-verify strategy for verifier)
- Add sg_only mode markers (/tmp/.sg_only_mode, /tmp/.sg_only_workdir)
- Keep TAC env vars (TAC_SERVER_HOSTNAME, DECRYPTION_KEY)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>1 parent 30c785c commit fe4a526
File tree
2 files changed
+45
-40
lines changed- benchmarks
- ccb_feature/bustub-hyperloglog-impl-001/environment
- ccb_test/openhands-search-file-test-001/environment
2 files changed
+45
-40
lines changedLines changed: 22 additions & 21 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
| 1 | + | |
| 2 | + | |
| 3 | + | |
3 | 4 | | |
4 | | - | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
8 | | - | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
9 | 12 | | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 13 | + | |
16 | 14 | | |
17 | 15 | | |
18 | 16 | | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
30 | 28 | | |
31 | 29 | | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
| |||
Lines changed: 23 additions & 19 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
2 | | - | |
| 1 | + | |
| 2 | + | |
| 3 | + | |
3 | 4 | | |
4 | | - | |
| 5 | + | |
5 | 6 | | |
6 | 7 | | |
7 | 8 | | |
8 | | - | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
9 | 12 | | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
| 13 | + | |
16 | 14 | | |
17 | 15 | | |
18 | 16 | | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
28 | 32 | | |
29 | 33 | | |
30 | 34 | | |
| |||
0 commit comments