Skip to content

Commit bde7973

Browse files
committed
add prompting for more full completion of the spec
1 parent ecdeb5a commit bde7973

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

evals/git-evals/judge-git-eval.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,7 @@ ${codebuffChanges}
6767
${evalRun.error ? evalRun.error : 'None'}
6868
[/ERROR]
6969
70-
Please analyze the trace of the implementation attempt and provide:
70+
Please analyze the implementation attempt and provide:
7171
1. A detailed analysis of the implementation trace and the final changes. Include how the changes compare to the ground truth change. Does it have similar behavior at least?
7272
2. Key strengths and weaknesses of the implementation
7373
3. Numerical scores (0-10):

evals/git-evals/run-git-evals.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,7 @@ Note that files can only be changed with tools. If no tools are called, no files
134134
135135
You must decide whether to:
136136
1. 'continue' - Generate a follow-up prompt for Codebuff
137-
2. 'complete' - The implementation is done and satisfies the spec
137+
2. 'complete' - The implementation is done and fully satisfies the spec, including tests, documentation, and any other relevant artifacts
138138
3. 'halt' - The implementation is off track and unlikely to be completed within ${MAX_ATTEMPTS - attempts} more attempts
139139
140140
If deciding to continue, include a clear, focused prompt for Codebuff in next_prompt. Note that Codebuff does not have access to the spec, so you must describe the changes you want Codebuff to make in a way that is clear and concise.

0 commit comments

Comments
 (0)