ISSUE-690: Create AI model whitelist configuration#693
ISSUE-690: Create AI model whitelist configuration#693
Conversation
94bd5bf to
96d131b
Compare
donny-wong
left a comment
There was a problem hiding this comment.
Hi @Naragod , I think you misunderstood the requirements for this Issue. We specifically would like to whitelist what is allowed to put in the remote_url argument for the RemoteModel class.
1c5c352 to
a908525
Compare
Hi @donny-wong I have made some changes. Please take a look. |
Will mention here
Will document here what we spoke about: We will not use a whitelist file but instead put the list of allowed endpoint urls in the settings.yml file. |
34b2621 to
831c2b4
Compare
0cbcffe to
362ccd9
Compare
05f7be1 to
362ccd9
Compare
for more information, see https://pre-commit.ci
|
|
||
| from ..tester import Test, Tester | ||
| from ..specs import TestSpecs | ||
| from ...config import config as server_config |
There was a problem hiding this comment.
I'm getting an import error from here when running the AI test. Did you encounter this?
Extra Info
[AI Feedback]
Messages on stderr:
Traceback (most recent call last):
File "<string>", line 1, in <module>
import sys, json; sys.path.append("/app/autotest_server"); from testers.ai.ai_tester import AiTester as Tester; from testers.specs import TestSpecs; Tester(resource_settings=[], specs=TestSpecs.from_json(sys.stdin.read())).run()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/autotest_server/testers/ai/ai_tester.py", line 7, in <module>
from ...config import config as server_config
ImportError: attempted relative import beyond top-level package
Description:
We wish to dynamically whitelist AI models available for use. To prevent the potential exposure of sensitive data when autograding/autotesting assignments, we wish to restrict usage to remote models.
Implementation
Allow users when creating an assignment the ability to submit a whitelist of available models. When the autotest is ran, any models used missing from this list will fail to execute.