Conversation
|
Judging by the logs it seems CodeGate finds the package and adds it to the context, but for some reason the response does not take it into account. Will debug further. |
Signed-off-by: Radoslav Dimitrov <radoslav@stacklok.com>
5cca228 to
4c2eb9e
Compare
|
I had to update the malicious package test case for this and the rest of the local providers because of the capabilities of the 0.5B model we are using for these tests. It looks like it cannot handle the system prompts as precisely as we expect which results in non-deterministic responses, i.e. sometimes it includes example code snippets even though it says the package is malicious, sometimes it doesn't include the link to insights. To handle all of this decided to simplify the test and just check if CodeGate was able to flag it. |
jhrozek
left a comment
There was a problem hiding this comment.
small models are hard to predict
The following PR adds malicious package test for Ollama.
Related to: #904