feat: prompt pre fetch plugin call#81
Conversation
Signed-off-by: julianstephen <julian.stephen@gmail.com>
40c59bd to
63e847f
Compare
rubambiza
left a comment
There was a problem hiding this comment.
Thanks for adding prompt pre-fetch support — the overall structure looks good. A few things to address before this can merge:
CI is red — build fails on both Python 3.11 and 3.12. Please investigate and fix.
assert payload.args is not None (plugin.py ~line 90) — assert statements are stripped in optimized mode (python -O) and will crash the process otherwise. This should be a proper check that returns a PromptPrehookResult with a violation, similar to the error handling below it.
Unreachable code — the original return PromptPrehookResult(continue_processing=True) at the end of prompt_pre_fetch is now dead code after the new implementation. Can be removed.
Minor:
- f-string
f"... payload {payload}"in the debug log could dump large objects. Consider logging justpayload.prompt_idor similar. - Missing newline at end of
tests/single-server-filter.yaml.
|
@julianstephen Since this is still in draft, feel free to ignore my comments for now 😅 I should have specified to Claude to surface PRs that are ready for reviews. |
Summary
Fixes for #56
Related issue(s)
(Optional) Testing Instructions
Create a new envoy filter that will route to ext proc on prompt gets