Skip to content

Testcases on Sync Agent API's #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BhavaniKondra wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Testcases on Sync Agent API's #17

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions log/tkex_test_3001_tools.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
INFO: Creating SQL profile: PYSAI_SQL_PROFILE_A6929915727F43B5B6D310884E982E86
INFO: Creating SQL tool: PYSAI_SQL_TOOL_A6929915727F43B5B6D310884E982E86
INFO: --- Starting test: test_3000_sql_tool_created ---
INFO: Validating SQL tool creation
INFO: --- Finished test: test_3000_sql_tool_created ---
INFO: Creating RAG profile: PYSAI_RAG_PROFILE_A6929915727F43B5B6D310884E982E86
INFO: Creating RAG tool: PYSAI_RAG_TOOL_A6929915727F43B5B6D310884E982E86
INFO: --- Starting test: test_3001_rag_tool_created ---
INFO: Validating RAG tool creation
INFO: --- Finished test: test_3001_rag_tool_created ---
INFO: Creating PL/SQL function: PYSAI_CALC_AGE_A6929915727F43B5B6D310884E982E86
INFO: Creating PL/SQL tool: PYSAI_PLSQL_TOOL_A6929915727F43B5B6D310884E982E86
INFO: --- Starting test: test_3002_plsql_tool_created ---
INFO: Validating PL/SQL tool creation
INFO: --- Finished test: test_3002_plsql_tool_created ---
INFO: Creating Web Search tool: PYSAI_WEB_TOOL_A6929915727F43B5B6D310884E982E86
INFO: --- Starting test: test_3007_web_search_tool_created ---
INFO: Validating Web Search tool creation
INFO: --- Finished test: test_3007_web_search_tool_created ---
INFO: --- Starting test: test_3003_list_tools ---
INFO: Listing all tools
INFO: Tools present: {'PYSAI_WEB_TOOL_A6929915727F43B5B6D310884E982E86', 'PYSAI_PLSQL_TOOL_A6929915727F43B5B6D310884E982E86', 'PYSAI_RAG_TOOL_A6929915727F43B5B6D310884E982E86', 'WEBSEARCH', 'EMAIL', 'HUMAN', 'PYSAI_SQL_TOOL_A6929915727F43B5B6D310884E982E86'}
INFO: --- Finished test: test_3003_list_tools ---
INFO: --- Starting test: test_3004_list_tools_regex ---
INFO: Listing tools using regex ^PYSAI_
INFO: Matched tools: {'PYSAI_RAG_TOOL_A6929915727F43B5B6D310884E982E86', 'PYSAI_PLSQL_TOOL_A6929915727F43B5B6D310884E982E86', 'PYSAI_SQL_TOOL_A6929915727F43B5B6D310884E982E86', 'PYSAI_WEB_TOOL_A6929915727F43B5B6D310884E982E86'}
INFO: --- Finished test: test_3004_list_tools_regex ---
INFO: --- Starting test: test_3005_fetch_tool ---
INFO: Fetching SQL tool
INFO: --- Finished test: test_3005_fetch_tool ---
INFO: --- Starting test: test_3006_enable_disable_sql_tool ---
INFO: Disabling SQL tool
INFO: Enabling SQL tool
INFO: --- Finished test: test_3006_enable_disable_sql_tool ---
INFO: Creating EMAIL tool: NEG_EMAIL_TOOL
INFO: Creating SLACK tool: NEG_SLACK_TOOL
INFO: Creating HTTP tool: NEG_HTTP_TOOL
INFO: Creating SQL tool with INVALID profile: NEG_SQL_TOOL
INFO: --- Starting test: test_3010_sql_tool_with_invalid_profile_created ---
INFO: Validating SQL tool with invalid profile is stored
INFO: --- Finished test: test_3010_sql_tool_with_invalid_profile_created ---
INFO: Creating RAG tool with INVALID profile: NEG_RAG_TOOL
INFO: --- Starting test: test_3011_rag_tool_with_invalid_profile_created ---
INFO: Validating RAG tool with invalid profile is stored
INFO: --- Finished test: test_3011_rag_tool_with_invalid_profile_created ---
INFO: Creating PL/SQL tool with INVALID function: NEG_PLSQL_TOOL
INFO: --- Starting test: test_3012_plsql_tool_with_invalid_function_created ---
INFO: Validating PL/SQL tool with invalid function is stored
INFO: --- Finished test: test_3012_plsql_tool_with_invalid_function_created ---
INFO: --- Starting test: test_3017_fetch_non_existent_tool ---
INFO: Fetching non-existent tool
ERROR: Agent Tool TOOL_DOES_NOT_EXIST not found
INFO: --- Finished test: test_3017_fetch_non_existent_tool ---
INFO: --- Starting test: test_3018_list_invalid_regex ---
INFO: Listing tools with invalid regex
ERROR: ORA-12726: unmatched bracket in regular expression
Help: https://docs.oracle.com/error-help/db/ora-12726/
INFO: --- Finished test: test_3018_list_invalid_regex ---
INFO: --- Starting test: test_3019_list_tools ---
INFO: Listing all tools
INFO: Tools present: {'PYSAI_WEB_TOOL_A6929915727F43B5B6D310884E982E86', 'PYSAI_PLSQL_TOOL_A6929915727F43B5B6D310884E982E86', 'NEG_RAG_TOOL', 'PYSAI_RAG_TOOL_A6929915727F43B5B6D310884E982E86', 'NEG_SQL_TOOL', 'WEBSEARCH', 'EMAIL', 'HUMAN', 'PYSAI_SQL_TOOL_A6929915727F43B5B6D310884E982E86', 'NEG_PLSQL_TOOL'}
INFO: --- Finished test: test_3019_list_tools ---
INFO: Deleting NEG_PLSQL_TOOL
INFO: Deleting NEG_RAG_TOOL
INFO: Deleting NEG_SQL_TOOL
INFO: Deleting Web Search tool
INFO: Deleting PL/SQL tool
INFO: Dropping PL/SQL function
INFO: Deleting RAG tool
INFO: Deleting RAG profile
INFO: Deleting SQL tool
INFO: Deleting SQL profile
75 changes: 75 additions & 0 deletions log/tkex_test_3101_tasks.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
INFO: Creating TASK_A: PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TASK_A
INFO: TASK_A created successfully
INFO: --- Starting test: test_3100_task_creation ---
INFO: Verifying TASK_A creation
INFO: --- Finished test: test_3100_task_creation ---
INFO: Creating TASK_B: PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TASK_B
INFO: TASK_B created successfully
INFO: --- Starting test: test_3101_task_chaining ---
INFO: Verifying TASK_B chaining
INFO: --- Finished test: test_3101_task_chaining ---
INFO: --- Starting test: test_3102_fetch_task ---
INFO: Fetching TASK_A
INFO: --- Finished test: test_3102_fetch_task ---
INFO: --- Starting test: test_3103_list_tasks_with_regex ---
INFO: Listing tasks with regex
INFO: --- Finished test: test_3103_list_tasks_with_regex ---
INFO: --- Starting test: test_3104_disable_enable_task ---
INFO: Disabling TASK_B
INFO: Enabling TASK_B
INFO: --- Finished test: test_3104_disable_enable_task ---
INFO: --- Starting test: test_3105_set_single_attribute_invalid ---
INFO: Setting invalid single attribute for TASK_B
INFO: Expected Oracle failure: ORA-20051: Invalid task attribute - description
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 12435
ORA-06512: at line 1
INFO: --- Finished test: test_3105_set_single_attribute_invalid ---
INFO: --- Starting test: test_3110_fetch_non_existent_task ---
INFO: Fetching non-existent task
INFO: Expected failure (NOT_FOUND): Agent Task PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_NO_SUCH_TASK not found
INFO: --- Finished test: test_3110_fetch_non_existent_task ---
INFO: --- Starting test: test_3111_duplicate_task_creation_fails ---
INFO: Creating duplicate TASK_A without replace
INFO: Expected Oracle failure: ORA-20051: Task PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TASK_A already exists.
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 11214
ORA-06512: at line 1
INFO: --- Finished test: test_3111_duplicate_task_creation_fails ---
INFO: --- Starting test: test_3113_set_invalid_attribute ---
INFO: Setting invalid attribute for TASK_A
INFO: Expected Oracle failure: ORA-20051: Invalid task attribute - unknown_attribute
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 12435
ORA-06512: at line 1
INFO: --- Finished test: test_3113_set_invalid_attribute ---
INFO: --- Starting test: test_3114_invalid_regex_pattern ---
INFO: Listing tasks with invalid regex
INFO: Expected Oracle failure: ORA-12726: unmatched bracket in regular expression
Help: https://docs.oracle.com/error-help/db/ora-12726/
INFO: --- Finished test: test_3114_invalid_regex_pattern ---
INFO: --- Starting test: test_3115_delete_disabled_task_without_force ---
INFO: Creating and deleting disabled task
INFO: Expected failure (NOT_FOUND): Agent Task PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TEMP_DELETE not found
INFO: --- Finished test: test_3115_delete_disabled_task_without_force ---
INFO: --- Starting test: test_3116_missing_instruction ---
INFO: Creating task with missing instruction
INFO: Expected Oracle failure: ORA-20051: Missing value for task attribute - instruction
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 11214
ORA-06512: at line 1
INFO: --- Finished test: test_3116_missing_instruction ---
INFO: --- Starting test: test_3117_delete_enabled_task_without_force_succeeds ---
INFO: Creating and deleting enabled task
INFO: Expected failure (NOT_FOUND): Agent Task PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_FORCE_DELETE_TEST not found
INFO: --- Finished test: test_3117_delete_enabled_task_without_force_succeeds ---
INFO: --- Starting test: test_3118_delete_disabled_task_with_force_succeeds ---
INFO: Deleting initially disabled task
INFO: Expected failure (NOT_FOUND): Agent Task PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_DISABLED_CREATE not found
INFO: --- Finished test: test_3118_delete_disabled_task_with_force_succeeds ---
INFO: Deleting TASK_B: PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TASK_B
INFO: Expected failure (NOT_FOUND): Agent Task PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TASK_B not found
INFO: TASK_B deleted successfully
INFO: Deleting TASK_A: PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TASK_A
INFO: Expected failure (NOT_FOUND): Agent Task PYSAI_3101_E97F2DC6CEC54B84947D3DDBF98AAB30_TASK_A not found
INFO: TASK_A deleted successfully
90 changes: 90 additions & 0 deletions log/tkex_test_3201_agents.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
INFO: Creating profile: PYSAI_3200_PROFILE_CBDDF3A609AA451A8A26119EF7056ED2
INFO: Creating agent: PYSAI_3200_AGENT_8FCFDC441DCF4A2A9D9B97B2985C6FD0
INFO: --- Starting test: test_3200_identity ---
INFO: Verifying agent identity
INFO: --- Finished test: test_3200_identity ---
INFO: --- Starting test: test_3201_list ---
INFO: Listing agents with pattern: None
INFO: Agents found: ['PYSAI_3200_AGENT_8FCFDC441DCF4A2A9D9B97B2985C6FD0', 'CUSTOMERAGENT']
INFO: --- Finished test: test_3201_list ---
INFO: --- Starting test: test_3201_list ---
INFO: Listing agents with pattern: .*
INFO: Agents found: ['PYSAI_3200_AGENT_8FCFDC441DCF4A2A9D9B97B2985C6FD0', 'CUSTOMERAGENT']
INFO: --- Finished test: test_3201_list ---
INFO: --- Starting test: test_3201_list ---
INFO: Listing agents with pattern: ^PYSAI_3200_AGENT_
INFO: Agents found: ['PYSAI_3200_AGENT_8FCFDC441DCF4A2A9D9B97B2985C6FD0']
INFO: --- Finished test: test_3201_list ---
INFO: --- Starting test: test_3202_fetch ---
INFO: Fetching agent: PYSAI_3200_AGENT_8FCFDC441DCF4A2A9D9B97B2985C6FD0
INFO: --- Finished test: test_3202_fetch ---
INFO: --- Starting test: test_3203_fetch_non_existing ---
INFO: Fetching non-existing agent: PYSAI_NO_SUCH_AGENT_caf57d1b53f34072ab6ef2f1d5a1d85e
INFO: Expected failure (NOT_FOUND): Agent PYSAI_NO_SUCH_AGENT_caf57d1b53f34072ab6ef2f1d5a1d85e not found
INFO: --- Finished test: test_3203_fetch_non_existing ---
INFO: --- Starting test: test_3204_disable_enable ---
INFO: Disabling agent
INFO: --- Finished test: test_3204_disable_enable ---
INFO: --- Starting test: test_3205_set_attribute ---
INFO: Setting role attribute
INFO: --- Finished test: test_3205_set_attribute ---
INFO: --- Starting test: test_3206_set_attributes ---
INFO: Replacing attributes
INFO: --- Finished test: test_3206_set_attributes ---
INFO: --- Starting test: test_3207_set_attribute_invalid_key ---
INFO: Setting invalid attribute key
INFO: Expected Oracle failure: ORA-20050: Invalid agent attribute - no_such_key
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 12435
ORA-06512: at line 1
INFO: --- Finished test: test_3207_set_attribute_invalid_key ---
INFO: --- Starting test: test_3208_set_attribute_none ---
INFO: Setting attribute to None
INFO: Expected Oracle failure: ORA-20050: Missing value for agent attribute - role
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 12435
ORA-06512: at line 1
INFO: --- Finished test: test_3208_set_attribute_none ---
INFO: --- Starting test: test_3209_set_attribute_empty ---
INFO: Setting attribute to empty string
INFO: Expected Oracle failure: ORA-20050: Missing value for agent attribute - role
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 12435
ORA-06512: at line 1
INFO: --- Finished test: test_3209_set_attribute_empty ---
INFO: --- Starting test: test_3210_create_existing_without_replace ---
INFO: Create existing agent without replace should fail
INFO: Expected Oracle failure: ORA-20050: Agent PYSAI_3200_AGENT_8FCFDC441DCF4A2A9D9B97B2985C6FD0 already exists.
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 10544
ORA-06512: at line 1
INFO: --- Finished test: test_3210_create_existing_without_replace ---
INFO: --- Starting test: test_3211_delete_and_recreate ---
INFO: Create agent: PYSAI_RECREATE_411b793a4cb846c1b61b52dde0ffff82
INFO: Delete agent: PYSAI_RECREATE_411b793a4cb846c1b61b52dde0ffff82
INFO: Recreate agent: PYSAI_RECREATE_411b793a4cb846c1b61b52dde0ffff82
INFO: Cleanup agent: PYSAI_RECREATE_411b793a4cb846c1b61b52dde0ffff82
INFO: --- Finished test: test_3211_delete_and_recreate ---
INFO: --- Starting test: test_3212_disable_after_delete ---
INFO: --- Finished test: test_3212_disable_after_delete ---
INFO: --- Starting test: test_3213_enable_after_delete ---
INFO: --- Finished test: test_3213_enable_after_delete ---
INFO: --- Starting test: test_3214_set_attribute_after_delete ---
INFO: Expected Oracle failure: ORA-20050: Agent PYSAI_TMP_DEL_17534F029EBC49E9971625388D64D27E does not exist.
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD$BASE", line 2251
ORA-06512: at "C##CLOUD$SERVICE.DBMS_CLOUD_AI_AGENT", line 12435
ORA-06512: at line 1
INFO: --- Finished test: test_3214_set_attribute_after_delete ---
INFO: --- Starting test: test_3215_double_delete ---
INFO: Testing double delete semantics: PYSAI_TMP_DOUBLE_DEL_5c0e1bc87f3b4240bf302a965ecf7813
INFO: Second delete with force=True should succeed
INFO: --- Finished test: test_3215_double_delete ---
INFO: --- Starting test: test_3216_fetch_after_delete ---
INFO: Expected failure (NOT_FOUND): Agent PYSAI_TMP_FETCH_DEL_07b0fae441d1458b961a520736712c79 not found
INFO: --- Finished test: test_3216_fetch_after_delete ---
INFO: --- Starting test: test_3217_list_all_non_empty ---
INFO: Listing all agents
INFO: Total agents found: 2
INFO: --- Finished test: test_3217_list_all_non_empty ---
INFO: Deleting agent: PYSAI_3200_AGENT_8FCFDC441DCF4A2A9D9B97B2985C6FD0
INFO: Deleting profile: PYSAI_3200_PROFILE_CBDDF3A609AA451A8A26119EF7056ED2
Loading