Conversation
2ed87bb to
57d9058
Compare
|
Judging by the logs it seems CodeGate finds the package and adds it to the context, but for some reason the response does not take it into account. Will debug further. |
Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
57d9058 to
3c3422f
Compare
|
I had to update the malicious package test case for this and the rest of the local providers because of the capabilities of the 0.5B model we are using for these tests. It looks like it cannot handle the system prompts as precisely as we expect which results in non-deterministic responses, i.e. sometimes it includes example code snippets even though it says the package is malicious, sometimes it doesn't include the link to insights. To handle all of this decided to simplify the test and just check if CodeGate was able to flag it. |
jhrozek
left a comment
There was a problem hiding this comment.
good enough test for a small model
The following adds malicious package test for llamacpp.
Related to: #904