Testcases on Sync Agent API's #17
Conversation
|
Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA). To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application. When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated. If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public. |
oracle-contributor-agreement
bot
added
the
OCA Required
BhavaniKondra
changed the title
Pushpendra-Garg
requested review from
AKSHAY-058,
Pushpendra-Garg and
prateeks99
| region="us-chicago-1", | ||
| oci_apiformat="GENERIC", | ||
| oci_compartment_id=oci_compartment_id, | ||
| model="meta.llama-4-maverick-17b-128e-instruct-fp8", |
There was a problem hiding this comment.
Can we pass region, model using environment variables ?
| END; | ||
| """ | ||
| with select_ai.cursor() as cur: | ||
| cur.execute(ddl) |
There was a problem hiding this comment.
A log can be added to confirm creation of PL SQL function.
|
|
||
| def test_3000_sql_tool_created(sql_tool): | ||
| logger.info("Validating SQL tool creation") | ||
| assert sql_tool.tool_name == SQL_TOOL_NAME |
There was a problem hiding this comment.
Similar log can be added to log the tool name and profile name.
|
|
||
| def test_3001_rag_tool_created(rag_tool): | ||
| logger.info("Validating RAG tool creation") | ||
| assert rag_tool.tool_name == RAG_TOOL_NAME |
There was a problem hiding this comment.
Similar log can be added to log the tool name and profile name.
|
|
||
| def test_3002_plsql_tool_created(plsql_tool): | ||
| logger.info("Validating PL/SQL tool creation") | ||
| assert plsql_tool.tool_name == PLSQL_TOOL_NAME |
There was a problem hiding this comment.
Similar log can be added to log the tool name and function name.
|
|
||
|
|
||
| def test_3006_enable_disable_sql_tool(sql_tool): | ||
| logger.info("Disabling SQL tool") |
There was a problem hiding this comment.
Is there a way to confirm if tool is enabled/disabled ?
AKSHAY-058
left a comment
AKSHAY-058
left a comment
There was a problem hiding this comment.
Please list all the bugs filed with this effort
Changes look good
| assert task_a.task_name == TASK_A_NAME | ||
| assert task_a.description == "Primary analysis task" | ||
| assert task_a.attributes.enable_human_tool is False | ||
| assert task_a.attributes.tools == ["MOVIE_SQL_TOOL"] |
There was a problem hiding this comment.
It would be good to log task object attributes.
|
|
||
| def test_3102_fetch_task(task_a): | ||
| logger.info("Fetching TASK_A") | ||
| fetched = Task.fetch(TASK_A_NAME) |
| def test_3103_list_tasks_with_regex(): | ||
| logger.info("Listing tasks with regex") | ||
| tasks = list(Task.list(f"{BASE}.*")) | ||
| names = [t.task_name for t in tasks] |
There was a problem hiding this comment.
List all the names. Please log in sorted order to get consistent output every time.
|
|
||
| def test_3104_disable_enable_task(task_b): | ||
| logger.info("Disabling TASK_B") | ||
| task_b.disable() |
There was a problem hiding this comment.
Is there a way to verify this ?
| logger.info("Creating and deleting disabled task") | ||
| attrs = TaskAttributes(instruction="Temporary task", tools=None) | ||
| task = Task(task_name=task_name, description="Temp task", attributes=attrs) | ||
| task.create() |
There was a problem hiding this comment.
Should we log task here for debugging purpose ?
| logger.info("Deleting initially disabled task") | ||
| attrs = TaskAttributes(instruction="Initially disabled task", tools=None) | ||
| task = Task(task_name=task_name, attributes=attrs) | ||
| task.create(enabled=False) |
There was a problem hiding this comment.
Should we log the task details ?
| logger.info("Verifying agent identity") | ||
| assert agent.agent_name == PYSAI_AGENT_NAME | ||
| assert agent.description == PYSAI_AGENT_DESC | ||
| assert agent.attributes == agent_attributes |
There was a problem hiding this comment.
Please add log for agent attributes.
|
|
||
| def test_3202_fetch(agent_attributes): | ||
| logger.info("Fetching agent: %s", PYSAI_AGENT_NAME) | ||
| a = Agent.fetch(PYSAI_AGENT_NAME) |
|
|
||
| def test_3204_disable_enable(agent): | ||
| logger.info("Disabling agent") | ||
| agent.disable() |
| a.create() | ||
|
|
||
| logger.info("Delete agent: %s", name) | ||
| a.delete(force=True) |
There was a problem hiding this comment.
Verify if agent is removed.
|
|
||
| logger.info("Recreate agent: %s", name) | ||
| a.create(replace=False) | ||
|
|
There was a problem hiding this comment.
Verify if agent is created
|
|
||
| # As per your Teams learning: force=True allows idempotent delete | ||
| logger.info("Second delete with force=True should succeed") | ||
| a.delete(force=True) |
There was a problem hiding this comment.
Second delete with force = False
It should raise an exception, please check
There was a problem hiding this comment.
It won't raise as force=True.
|
|
||
| def test_3300_create_and_identity(team, team_attributes): | ||
| log_step("Validating team identity and attributes") | ||
| assert team.team_name == PYSAI_TEAM_NAME |
There was a problem hiding this comment.
Log team object attributes ?
| t = Team(name, team_attributes, "TMP") | ||
| t.create() | ||
| t.delete(force=True) | ||
| expect_error("ORA-20053", lambda: t.delete(force=False)) |
There was a problem hiding this comment.
Additional test case for double delete with force=True
It should pass successfully.
| tool_params=ToolParams( | ||
| credential_name="EMAIL_CRED", | ||
| notification_type="EMAIL", | ||
| recipient="kondra.nagabhavani@oracle.com", |
There was a problem hiding this comment.
Please fetch recipient, sender and smtp_host from ENV variables.
3 participants
Description
Adds test coverage for Agent synchronous APIs including positive and negative scenarios, and validates behavior for error cases and edge conditions. This improves coverage of agent tool APIs.
No existing issues are fixed directly; this is a test coverage improvement.
Type of change
How Has This Been Tested?
Tests were executed locally against the existing test framework and verified by running the full test suite.
Test Configuration:
Checklist: