Tabular, Thoughts, PDF Export, SQL plugin updates, and bug fixes#782
Tabular, Thoughts, PDF Export, SQL plugin updates, and bug fixes#782paullizer wants to merge 27 commits intoDevelopmentfrom
Conversation
* initial * adding logging
* Improve SQL action ui and config * adding user_id and created by modified by to group and global agents/actions and activity logs for all agent/action CRUD tasks
* initial tabular data support * working with workspaces, aligned with chat * add popup modal view for xlsx * added tabular data analysis as a mini-agent within chats to process tab data without full agent * Update route_backend_chats.py * SK Mini-Agent Performance Optimization * fixed inline sk trigger bug
* Improve SQL action ui and config * adding user_id and created by modified by to group and global agents/actions and activity logs for all agent/action CRUD tasks * schema is injected directly into the agent's brain at load time, and descriptions explicitly instruct the LLM on the correct tool-calling workflow.
…o use the resilient pattern:
There was a problem hiding this comment.
Pull request overview
This PR is a broad feature+fix bundle that expands conversation export capabilities (PDF + intro summaries + persistent cached summaries), adds “Processing Thoughts” end-to-end (admin toggle → persistence → UI), improves workspace agent/action UX (list/grid + view modal), and hardens SQL plugin behavior (schema awareness + companion schema plugin + sys.* catalog view migration).
Changes:
- Add persistent conversation summaries (new API + UI surface) and enhance export workflow (PDF format + optional intro summary step).
- Add Processing Thoughts (settings, Cosmos containers, backend routes, streaming + polling UI).
- Improve SQL plugins and enhanced citations handling for tabular files (download/preview + chat-upload blob storage), plus workspace list/grid/view UX updates.
Reviewed changes
Copilot reviewed 74 out of 77 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| functional_tests/test_sql_schema_sys_catalog_views.py | Adds functional coverage for SQL Server sys.* schema queries (but currently hard-codes an older VERSION). |
| functional_tests/test_sql_query_plugin_schema_awareness.py | Adds functional coverage for SQL plugin workflow guidance (currently asserts outdated strings / version header). |
| functional_tests/test_sql_auto_schema_companion.py | Adds functional coverage for auto-creating SQLSchemaPlugin companion (not verified via tools in this environment). |
| functional_tests/test_persistent_conversation_summary.py | Adds functional coverage for cached summary intro behavior (version header currently out of sync). |
| docs/explanation/fixes/v0.239.008/CHAT_TABULAR_SK_TRIGGER_FIX.md | Fix documentation for chat-uploaded tabular SK trigger behavior. |
| docs/explanation/fixes/SQL_QUERY_PLUGIN_SCHEMA_AWARENESS_FIX.md | Fix documentation for SQL schema awareness + companion plugin + sys.* migration. |
| docs/explanation/features/v0.239.022/CONVERSATION_EXPORT.md | Feature documentation snapshot for updated export shape/workflow. |
| docs/explanation/features/v0.239.003/PROCESSING_THOUGHTS.md | Feature documentation for Processing Thoughts. |
| application/single_app/templates/workspace.html | Adds list/grid view toggle and shared “item view” modal for personal workspace. |
| application/single_app/templates/group_workspaces.html | Adds list/grid view toggle and shared “item view” modal for group workspace. |
| application/single_app/templates/chats.html | Exposes enable_thoughts flag to frontend settings. |
| application/single_app/templates/admin_settings.html | Adds tabular processing toggle + Processing Thoughts settings section. |
| application/single_app/templates/_sidebar_nav.html | Adds admin sidebar jump link for Processing Thoughts section. |
| application/single_app/templates/_plugin_modal.html | Adds SQL “Test Connection” UI and collapsible Advanced settings. |
| application/single_app/templates/_agent_examples_modal.html | Renders template instructions as sanitized markdown instead of . |
| application/single_app/static/json/schemas/sql_schema_plugin.additional_settings.schema.json | Fixes schema key names and required fields for SQL schema additional settings. |
| application/single_app/static/json/schemas/sql_schema.definition.json | Expands allowed auth types for sql_schema plugin. |
| application/single_app/static/json/schemas/sql_query_plugin.additional_settings.schema.json | Fixes schema key names and required fields for SQL query additional settings. |
| application/single_app/static/json/schemas/sql_query.definition.json | Expands allowed auth types for sql_query plugin. |
| application/single_app/static/js/workspace/workspace_plugins.js | Adds grid rendering + view modal wiring for personal actions. |
| application/single_app/static/js/workspace/workspace_agents.js | Adds list/grid rendering + view modal wiring for personal agents. |
| application/single_app/static/js/workspace/group_plugins.js | Adds list/grid rendering + view modal wiring for group actions. |
| application/single_app/static/js/workspace/group_agents.js | Adds list/grid rendering + view modal wiring + “Chat” for group agents. |
| application/single_app/static/js/plugin_common.js | Adds view button + grid rendering shared helpers for plugins/actions. |
| application/single_app/static/js/chat/chat-thoughts.js | New client module for thoughts polling, SSE thought handling, and per-message toggle rendering. |
| application/single_app/static/js/chat/chat-streaming.js | Wires SSE thought events into the streaming UI. |
| application/single_app/static/js/chat/chat-messages.js | Adds thoughts toggle + polling lifecycle to message send/receive flow. |
| application/single_app/static/js/chat/chat-loading-indicator.js | Adds dynamic loading indicator text updates (currently has an XSS issue). |
| application/single_app/static/js/chat/chat-input-actions.js | Adds “Download Original” for blob-backed chat file modal. |
| application/single_app/static/js/chat/chat-export.js | Adds PDF option + summary intro step + summary model selection to export wizard. |
| application/single_app/static/js/chat/chat-enhanced-citations.js | Adds tabular citation type + preview/download modal. |
| application/single_app/static/js/chat/chat-conversation-details.js | Adds summary card UI + generate/regenerate summary flow. |
| application/single_app/static/js/admin/admin_settings.js | Adds tabular toggle + thoughts gating to optional-features step logic. |
| application/single_app/static/css/styles.css | Adds grid card + markdown rendering styles. |
| application/single_app/static/css/chats.css | Adds Processing Thoughts UI styles (toggle, timeline, streaming badge). |
| application/single_app/semantic_kernel_plugins/sql_schema_plugin.py | Migrates SQL Server schema discovery to sys.* + improves pyodbc.Row handling + updates descriptions. |
| application/single_app/semantic_kernel_plugins/sql_query_plugin.py | Adds query_database + updates descriptions/metadata (metadata currently reintroduces “MUST” language). |
| application/single_app/semantic_kernel_plugins/plugin_invocation_logger.py | Adds callback registry for invocation events. |
| application/single_app/semantic_kernel_plugins/logged_plugin_loader.py | Enables SQL plugin creation path + auto-creates companion SQLSchemaPlugin for SQLQueryPlugin. |
| application/single_app/route_frontend_chats.py | Stores chat-uploaded tabular files in blob storage when enhanced citations enabled; Cosmos message references blob. |
| application/single_app/route_frontend_admin_settings.py | Saves enable_thoughts + defaults tabular processing toggle setting. |
| application/single_app/route_enhanced_citations.py | Adds tabular download + workspace tabular download + tabular preview endpoints. |
| application/single_app/route_backend_user_agreement.py | Allows workspace_type="chat". |
| application/single_app/route_backend_thoughts.py | New API routes for per-message thoughts + pending thoughts polling. |
| application/single_app/route_backend_documents.py | Reads blob-backed file messages for file content endpoint (no preview limits). |
| application/single_app/route_backend_conversations.py | Adds summary to metadata response + new POST summary endpoint; archives/deletes thoughts on conversation delete. |
| application/single_app/route_backend_agents.py | Adds activity logging and created_by/modified_by propagation for agents. |
| application/single_app/functions_thoughts.py | Implements ThoughtTracker + thoughts CRUD + archive/delete helpers. |
| application/single_app/functions_settings.py | Adds enable_thoughts + enable_tabular_processing_plugin defaults and dependency enforcement. |
| application/single_app/functions_personal_agents.py | Adds created_by/modified_by tracking for personal agents. |
| application/single_app/functions_personal_actions.py | Adds created_by/modified_by tracking for personal actions. |
| application/single_app/functions_group_agents.py | Adds created_by/modified_by tracking + optional user_id param for group agents. |
| application/single_app/functions_group_actions.py | Adds created_by/modified_by tracking + optional user_id param for group actions. |
| application/single_app/functions_global_agents.py | Adds created_by/modified_by tracking + optional user_id param for global agents. |
| application/single_app/functions_global_actions.py | Adds created_by/modified_by tracking for global actions (currently fails to default user_id). |
| application/single_app/functions_content.py | Adds batched embedding generation helper and reduces per-call jitter delay. |
| application/single_app/config.py | Bumps VERSION, adds chat blob containers, adds Cosmos containers for thoughts, ensures containers exist. |
| application/single_app/app.py | Registers new thoughts backend routes. |
| README.md | Documents SQL Database Agents optional feature. |
| CLAUDE.md | Updates internal guidance wording for release notes/versioning. |
application/single_app/semantic_kernel_plugins/tabular_processing_plugin.py
Outdated
Show resolved
Hide resolved
application/single_app/static/json/schemas/sql_query_plugin.additional_settings.schema.json
Show resolved
Hide resolved
application/single_app/static/json/schemas/sql_schema_plugin.additional_settings.schema.json
Show resolved
Hide resolved
Bionic711
left a comment
There was a problem hiding this comment.
I will approve after the conflicts are solved because I do not want to review again.
|
Nice update to multi-tab update. I do not have it splitting off csvs yet. its working great right now finding details across tabs and associating records across tabs. Also resolved requested items. lets merge this and I will work on the csv parts and can get more testing of dev and get your multi-gpt endpoints in, any change i make will not impact that stuff. |
New Features
Persistent Conversation Summaries
message_time_startandmessage_time_end— when a conversation has new messages beyond the cached range, a fresh summary is generated automatically.POST /api/conversations/<id>/summaryendpoint accepts an optionalmodel_deploymentand returns the generated summary.GET /api/conversations/<id>/metadataresponse now includes asummaryfield.generate_conversation_summary()as a shared helper used by both the export pipeline and the new API endpoint.route_backend_conversation_export.py,route_backend_conversations.py,chat-conversation-details.js,functions_conversation_metadata.py)PDF Conversation Export
bi-filetype-pdficon.route_backend_conversation_export.py,chat-export.js, PyMuPDF Story API, conversation export workflow)Conversation Export Intro Summaries
route_backend_conversation_export.py,chat-export.js, conversation export workflow)Agent & Action User Tracking (created_by / modified_by)
created_by,created_at,modified_by, andmodified_atfields that track which user created or last modified the entity.created_byandcreated_atvalues are preserved whilemodified_byandmodified_atare refreshed with the current user and timestamp.user_idparameter added tosave_group_agent,save_global_agent,save_group_action, andsave_global_actionfor caller-supplied user tracking (backward-compatible, defaults toNone).functions_personal_agents.py,functions_group_agents.py,functions_global_agents.py,functions_personal_actions.py,functions_group_actions.py,functions_global_actions.py)Activity Logging for Agent & Action CRUD Operations
activity_logsCosmos DB container and Application Insights.log_agent_creation,log_agent_update,log_agent_deletion,log_action_creation,log_action_update,log_action_deletion.user_id,activity_type,entity_type(agent/action),operation(create/update/delete),workspace_type(personal/group/global), andworkspace_context(group_id when applicable).functions_activity_logging.py,route_backend_agents.py,route_backend_plugins.py)Tabular Data Analysis — SK Mini-Agent for Normal Chat
TABULAR_EXTENSIONSconfiguration and routes the query through the SK mini-agent pipeline.user-documents,group-documents,public-documents,personal-chat) with automatic fallback resolution if the primary source lookup fails. A user asking about an Excel file in their personal workspace gets the same analytical treatment as one asking about a CSV uploaded directly to a chat.TabularProcessingPluginexposesdescribe_tabular_file,aggregate_column(sum, mean, count, min, max, median, std, nunique, value_counts),filter_rows(==, !=, >, <, >=, <=, contains, startswith, endswith),query_tabular_data(pandas query syntax),group_by_aggregate, andlist_tabular_files— all registered as Semantic Kernel functions that the mini-agent orchestrates autonomously.enable_enhanced_citationsadmin setting must be enabled for tabular data analysis to activate.run_tabular_sk_analysis(),TabularProcessingPlugin,collect_tabular_sk_citations(),TABULAR_EXTENSIONS)Tabular Tool Execution Citations
@plugin_function_loggerdecorator on allTabularProcessingPluginfunctions records each invocation including function name, input parameters, returned results, execution duration, and success/failure status.tool_name(e.g.,TabularProcessingPlugin.aggregate_column),function_arguments(the exact parameters passed), andfunction_result(the computed data returned).collect_tabular_sk_citations(),plugin_invocation_logger.py)SK Mini-Agent Performance Optimization
list_tabular_files) and schema inspection (describe_tabular_file), allowing the model to jump directly to analysis tool calls.@kernel_functionmethods converted toasync defusingasyncio.to_thread(). This enables Semantic Kernel's built-inasyncio.gather()to truly parallelize batched tool calls (e.g., 3 simultaneousaggregate_columncalls) instead of executing them serially on the event loop._df_cache,asyncio.to_thread, pre-dispatch schema injection inrun_tabular_sk_analysis())SQL Test Connection Button
POST /api/plugins/test-sql-connection.route_backend_plugins.py,plugin_modal_stepper.js,_plugin_modal.html)Bug Fixes
On-Demand Summary Generation — Content Normalization Fix
POST /api/conversations/<id>/summaryendpoint failing with an error when generating summaries from the conversation details modal.contentin Cosmos DB can be a list of content parts (e.g.,[{type: "text", text: "..."}]) rather than a plain string. The endpoint was passing the raw list ascontent_text, which either stringified incorrectly or produced empty transcript text._normalize_content()to properly flatten list/dict content into plain text, matching the export pipeline's behavior.route_backend_conversations.py,_normalize_content,generate_conversation_summary)Export Summary Reasoning-Model Compatibility
developerrole instead ofsystemfor instruction messages, removing allmax_tokens/max_completion_tokenscaps so the model decides output length naturally, and adding null-safe content extraction forNoneresponses.route_backend_conversation_export.py,_build_summary_intro,generate_conversation_summary)Conversation Export Schema and Markdown Refresh
route_backend_conversation_export.py,test_conversation_export.py, conversation export rendering)Export Tag/Classification Rendering Fix
{'category': 'model', 'value': 'gpt-5'}) in both Markdown and PDF exports.category: valuestrings, with smart handling for participant names, document titles, and generic category/value pairs.route_backend_conversation_export.py,_format_taghelper, Markdown/PDF metadata rendering)Export Summary Error Visibility
debug_printandlog_eventlogging to all summary generation error paths, including the empty-response path that previously failed silently.route_backend_conversation_export.py,_build_summary_intro, export error rendering)Content Safety for Streaming Chat Path
/api/chat/stream) SSE path, matching the existing non-streaming (/api/chat) implementation.AnalyzeTextOptionsanalysis, severity threshold checking (severity ≥ 4 blocks the message), blocklist matching, persistence of blocked messages tocosmos_safety_container, creation of safety-role message documents, and proper SSE event delivery of blocked status to the client.[DONE]event, then stops — preventing any further LLM invocation.route_backend_chats.py, streaming SSE generator,AnalyzeTextOptions,cosmos_safety_container)SQL Schema Plugin — Eliminate Redundant Schema Calls
get_database_schematwice per query even though the full schema was already injected into the agent's instructions at load time.@kernel_functiondescriptions insql_schema_plugin.pysaid "ALWAYS call this function FIRST," which overrode the schema context already available in the instructions.get_database_schema,get_table_schema,get_table_list,get_relationships) to use the resilient pattern: "If the database schema is already provided in your instructions, use that directly and do NOT call this function."sql_query_plugin.py.sql_schema_plugin.py,@kernel_functiondescriptions, schema injection)SQL Schema Plugin — Empty Tables from INFORMATION_SCHEMA
get_database_schemareturning'tables': {}(empty) despite the database having tables, while relationships were returned correctly.INFORMATION_SCHEMA.TABLESandINFORMATION_SCHEMA.COLUMNSviews, which returned empty results in the Azure SQL environment. Meanwhile, the relationships query usedsys.foreign_keys/sys.tables/sys.columnscatalog views which worked perfectly.sys.*catalog views consistently:sys.tables/sys.schemasfor table enumeration,sys.columnswithTYPE_NAME()for column details, andsys.indexes/sys.index_columnsfor primary key detection.pyodbc.Rowhandling throughout the plugin — removed allisinstance(table, tuple)checks that could fail with pyodbc Row objects, replaced with robust try/except indexing.sql_schema_plugin.py,sys.tables,sys.columns,sys.indexes, pyodbc.Row handling)SQL Query Plugin — Auto-Create Companion Schema Plugin
sql_queryaction never had aSQLSchemaPluginloaded in the kernel. The descriptions demanded callingget_database_schema— a function that didn't exist — creating an impossible dependency that caused the LLM to ask for clarification.LoggedPluginLoadernow automatically creates a companionSQLSchemaPluginwhenever aSQLQueryPluginis loaded, using the same connection details. This ensures schema discovery is always available.@kernel_functiondescriptions to be resilient: "If the database schema is provided in your instructions, use it directly. Otherwise, call get_database_schema." This dual-path approach works whether schema is injected via instructions or available via plugin functions._extract_sql_schema_for_instructions()to also detectSQLQueryPlugininstances and create a temporary schema extractor if noSQLSchemaPluginis found.logged_plugin_loader.py,sql_query_plugin.py,semantic_kernel_loader.py)SQL Query Plugin Schema Awareness
@kernel_functiondescriptions were generic with no workflow guidance, agent instructions had no database schema context, and the two plugins operated independently with no linkage.@kernel_functiondescriptions in both SQL plugins to be prescriptive workflow guides (modeled after the working LogAnalyticsPlugin), explicitly instructing the LLM to discover schema first before generating queries.query_database(question, query)convenience function toSQLQueryPluginfor intent-aligned tool calling.logged_plugin_loader.py(was previously commented out).sql_query_plugin.py,sql_schema_plugin.py,semantic_kernel_loader.py,logged_plugin_loader.py)Chat-Uploaded Tabular Files Now Trigger SK Mini-Agent in Model-Only Mode
filerole messages in conversation history.run_tabular_sk_analysis(source_hint="chat")to pre-compute results. The streaming path also now properly handlesfilerole messages (tabular and non-tabular) matching the non-streaming path's behavior.route_backend_chats.py,run_tabular_sk_analysis(),collect_tabular_sk_citations())Group SQL Action/Plugin Save Failure
sql://sql_query/sql://sql_schemaendpoint logic as personal action routes.sql_query.definition.json,sql_schema.definition.json) only allowingconnection_stringauth type, blockinguser,identity, andservicePrincipaltypes that the UI and runtime support.__Secretkey suffix mismatch in additional settings schemas whereconnection_string__Secretandpassword__Secretdidn't match the runtime's expectedconnection_stringandpasswordfield names. Also removed duplicateazuresqlenum value.route_backend_plugins.py,plugin_modal_stepper.js,sql_query.definition.json,sql_schema.definition.json,sql_query_plugin.additional_settings.schema.json,sql_schema_plugin.additional_settings.schema.json)User Interface Enhancements
Agent Responded Thought — Seconds & Total Duration
'gpt-5-nano' responded (16.3s from initial message)).request_start_timeis now captured at the top of both the non-streaming and streaming chat handlers, so the duration reflects the full request lifecycle — including content safety, hybrid search, and agent invocation — not just the model response time.route_backend_chats.py,request_start_time, agent responded thoughts)Enhanced Agent Execution Thoughts
gpt-5-nano).generationstep type (lightning bolt icon) — no frontend changes required.route_backend_chats.py,ThoughtTracker, agent execution pipeline)List/Grid View Toggle for Agents and Actions
myCustomAgent→My Custom Agent).view-utils.jsprovides reusable functions for all four workspace areas.view-utils.js,workspace_agents.js,workspace_plugins.js,plugin_common.js,group_agents.js,group_plugins.js,workspace.html,group_workspaces.html,styles.css)Chat with Agent Button for Group Agents
group_agents.js,group_workspaces.html)Hidden Deprecated Action Types
sql_schema,ui_test,queue_storage,blob_storage,embedding_model) are now hidden from the action creation wizard type selector. Existing actions of these types remain functional.plugin_modal_stepper.js)Advanced Settings Collapse Toggle
_plugin_modal.html,plugin_modal_stepper.js)