For rebuttal, add UIE eval results#103
Conversation
|
| GitGuardian id | GitGuardian status | Secret | Commit | Filename | |
|---|---|---|---|---|---|
| 30182866 | Triggered | OpenRouter API Key | e03502f | results/260411-kdd-rebuttal-cv-uie-model/eval.sh | View secret |
🛠 Guidelines to remediate hardcoded secrets
- Understand the implications of revoking this secret by investigating where it is used in your code.
- Replace and store your secret safely. Learn here the best practices.
- Revoke and rotate this secret.
- If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.
To avoid such incidents in the future consider
- following these best practices for managing and storing secrets including API keys and other credentials
- install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.
🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.
There was a problem hiding this comment.
Pull request overview
Adds CV-parse evaluation artifacts for the “PP-UIE-7B” UIE model run, intended to support KDD rebuttal reporting.
Changes:
- Added a full JSON dump of CV parsing evaluation outputs and summary metrics.
- Added a helper
eval.shscript to reproduce the run against OpenRouter.
Reviewed changes
Copilot reviewed 1 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
results/260411-kdd-rebuttal-cv-uie-model/PP-UIE-7B_Sculpt-AI_GIMBench-cv-parse_260411-020045.json |
Stores per-item extraction results and aggregate accuracy for the UIE CV-parse evaluation. |
results/260411-kdd-rebuttal-cv-uie-model/eval.sh |
Script to run the same evaluation configuration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @@ -0,0 +1,6 @@ | |||
| export API_KEY=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 | |||
There was a problem hiding this comment.
The script commits a live OpenRouter API key (sk-or-v1-…) into the repository. Remove the key from the script, rotate/revoke it immediately, and read it from an environment variable or secret manager at runtime (with a clear error if unset).
| python -m gimbench.cv.cv_parse --use_uie --model_name "PP-UIE-7B" --api_key $API_KEY --base_url $API_BASE | ||
|
|
||
| shutdown -h +3 |
There was a problem hiding this comment.
Including shutdown -h +3 in a repo script is risky because running this file will power off the host machine unexpectedly (including in CI or by other developers). Consider removing it, or guarding it behind an explicit opt-in flag/environment variable so evaluation can be run safely by default.
| "phone_number": { | ||
| "prediction": "+ 4 9$ (0) 621 181 2098", | ||
| "expected": "+49 (0) 621 181 2098", | ||
| "verbatim_correct": false, | ||
| "judge_model_correct": true, | ||
| "correct": true | ||
| }, | ||
| "email": { | ||
| "prediction": "b n.zhang@uni-mannheim.de", | ||
| "expected": "n.zhang@uni-mannheim.de", | ||
| "verbatim_correct": false, | ||
| "judge_model_correct": false, | ||
| "correct": false |
There was a problem hiding this comment.
This results JSON appears to contain personal data from CVs (e.g., emails and phone numbers under extraction_details). If this repository is shared publicly, committing per-example PII is a privacy/security risk; consider redacting these fields or only committing aggregated metrics (e.g., totals/accuracy) rather than raw predictions/expected values.
No description provided.