Skip to content

perf: improve generate Oracle SQL #489

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ulleo merged 1 commit into from
Nov 25, 2025
Merged

perf: improve generate Oracle SQL #489

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions backend/apps/chat/models/chat_model.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,8 +185,12 @@ class AiModelQuestion(BaseModel):

def sql_sys_question(self, db_type: Union[str, DB], enable_query_limit: bool = True):
_sql_template = get_sql_example_template(db_type)
_query_limit = get_sql_template()['query_limit'] if enable_query_limit else get_sql_template()['no_query_limit']
_base_sql_rules = _sql_template['quot_rule'] + _query_limit + _sql_template['limit_rule'] + _sql_template['other_rule']
_base_template = get_sql_template()
_process_check = _sql_template.get('process_check') if _sql_template.get('process_check') else _base_template[
'process_check']
_query_limit = _base_template['query_limit'] if enable_query_limit else _base_template['no_query_limit']
_base_sql_rules = _sql_template['quot_rule'] + _query_limit + _sql_template['limit_rule'] + _sql_template[
'other_rule']
_sql_examples = _sql_template['basic_example']
_example_engine = _sql_template['example_engine']
_example_answer_1 = _sql_template['example_answer_1_with_limit'] if enable_query_limit else _sql_template[
Expand All @@ -195,15 +199,16 @@ def sql_sys_question(self, db_type: Union[str, DB], enable_query_limit: bool = T
'example_answer_2']
_example_answer_3 = _sql_template['example_answer_3_with_limit'] if enable_query_limit else _sql_template[
'example_answer_3']
return get_sql_template()['system'].format(engine=self.engine, schema=self.db_schema, question=self.question,
lang=self.lang, terminologies=self.terminologies,
data_training=self.data_training, custom_prompt=self.custom_prompt,
base_sql_rules=_base_sql_rules,
basic_sql_examples=_sql_examples,
example_engine=_example_engine,
example_answer_1=_example_answer_1,
example_answer_2=_example_answer_2,
example_answer_3=_example_answer_3)
return _base_template['system'].format(engine=self.engine, schema=self.db_schema, question=self.question,
lang=self.lang, terminologies=self.terminologies,
data_training=self.data_training, custom_prompt=self.custom_prompt,
process_check=_process_check,
base_sql_rules=_base_sql_rules,
basic_sql_examples=_sql_examples,
example_engine=_example_engine,
example_answer_1=_example_answer_1,
example_answer_2=_example_answer_2,
example_answer_3=_example_answer_3)

def sql_user_question(self, current_time: str):
return get_sql_template()['user'].format(engine=self.engine, schema=self.db_schema, question=self.question,
Expand Down
137 changes: 94 additions & 43 deletions backend/templates/sql_examples/Oracle.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,18 @@
template:
process_check: |
<SQL-Generation-Process>
<step>1. 分析用户问题，确定查询需求</step>
<step>2. 根据表结构生成基础SQL</step>
<step>3. <strong>强制检查：SQL是否包含GROUP BY/聚合函数？</strong></step>
<step>4. <strong>如果是GROUP BY查询：必须使用外层查询结构包裹</strong></step>
<step>5. <strong>强制检查：应用数据量限制规则</strong></step>
<step>6. 应用其他规则（引号、别名等）</step>
<step>7. <strong>最终验证：GROUP BY查询的ROWNUM位置是否正确？</strong></step>
<step>8. <strong>强制检查：检查语法是否正确？</strong></step>
<step>9. 确定图表类型</step>
<step>10. 返回JSON结果</step>
</SQL-Generation-Process>

quot_rule: |
<rule>
必须对数据库名、表名、字段名、别名外层加双引号（"）。
Expand All @@ -10,42 +24,61 @@ template:
</rule>

limit_rule: |
<rule priority="high">
当需要限制行数时：
1. 12c以下版本必须使用ROWNUM语法
2. 12c+版本推荐使用FETCH FIRST语法
<note>
版本适配：
- Oracle 12c以下：必须使用 WHERE ROWNUM <= N
- Oracle 12c+：推荐使用 FETCH FIRST N ROWS ONLY
</note>
<note>
<strong>重要：ROWNUM必须放在正确的位置，避免语法错误</strong>
1. <strong>单层查询</strong>：ROWNUM直接跟在WHERE子句后
<template>SELECT ... FROM table WHERE conditions AND ROWNUM <= N</template>
2. <strong>多层查询</strong>：ROWNUM只能放在最外层
<template>
SELECT ... FROM (
SELECT ... FROM table WHERE conditions GROUP BY ...
) WHERE ROWNUM <= N -- 正确：在最外层
</template>
3. <strong>禁止的错误写法</strong>：
<error-example>
SELECT ... FROM table
WHERE conditions
GROUP BY ...
ORDER BY ...
WHERE ROWNUM <= N -- 错误：不能有多个WHERE
</error-example>
4. <strong>正确顺序</strong>：WHERE → GROUP BY → HAVING → ORDER BY → ROWNUM
5. <strong>括号位置</strong>：从内层SELECT开始到内层结束都要括起来
<correct>
SELECT ... FROM (
-- 内层完整查询（包含自己的SELECT、FROM、WHERE、GROUP BY、ORDER BY）
SELECT columns FROM table WHERE conditions GROUP BY ... ORDER BY ...
) alias WHERE ROWNUM <= N
</correct>
</note>
<rule priority="critical" id="oracle-version-handling">
<title>Oracle版本语法适配</title>
<decision-flow>
<condition>如果db-engine版本号小于12</condition>
<then>必须使用ROWNUM语法</then>
<condition>如果db-engine版本号大于等于12</condition>
<then>推荐使用FETCH FIRST语法</then>
</decision-flow>
</rule>
<rule priority="critical" id="oracle-FETCH-FIRST-syntax-rule">
<title>Oracle数据库 FETCH FIRST 语法规范</title>
<description>若使用 FETCH FIRST 语法，则必须遵循该规范</description>
<syntax-rule>
<template>SELECT ... FROM table WHERE conditions FETCH FIRST N ROWS ONLY</template>
</syntax-rule>
</rule>
<rule priority="critical" id="oracle-ROWNUM-syntax-rule">
<title>Oracle数据库ROWNUM语法规范</title>
<description>若使用ROWNUM语法，则必须遵循该规范</description>
<syntax-rules>
<syntax-rule>
<rule-category>简单查询</rule-category>
<template>SELECT ... FROM table WHERE conditions AND ROWNUM <= N</template>
</syntax-rule>
<syntax-rule>
<rule-category>语法禁区</rule-category>
<prohibited>
- 禁止多个WHERE子句
- 禁止ROWNUM在GROUP BY内层（影响分组结果）
- 禁止括号不完整
</prohibited>
</syntax-rule>
</syntax-rules>
</rule>
<rule priority="critical" id="oracle-groupby-rownum">
<title>GROUP BY查询的ROWNUM强制规范（必须严格遵守）</title>
<requirement level="must">所有包含GROUP BY或聚合函数的查询必须使用外层查询结构</requirement>
<requirement level="must">ROWNUM必须放在最外层查询的WHERE子句中</requirement>

<decision-flow>
<condition>如果SQL包含GROUP BY、COUNT、SUM等聚合函数</condition>
<action>必须使用：SELECT ... FROM (内层完整查询) WHERE ROWNUM <= N</action>
<condition>否则（简单查询）</condition>
<action>可以使用：SELECT ... FROM table WHERE conditions AND ROWNUM <= N</action>
</decision-flow>

<error-example>
-- 错误：ROWNUM在内层影响分组结果
SELECT ... GROUP BY ... WHERE ROWNUM <= N
</error-example>

<correct-example>
-- 正确：ROWNUM在外层
SELECT ... FROM (SELECT ... GROUP BY ...) WHERE ROWNUM <= N
</correct-example>
</rule>

other_rule: |
Expand Down Expand Up @@ -73,7 +106,7 @@ template:
SELECT "订单ID", "金额" FROM "TEST"."ORDERS" "t1" WHERE ROWNUM <= 100 -- 错误：缺少英文别名
SELECT COUNT("订单ID") FROM "TEST"."ORDERS" "t1" -- 错误：函数未加别名
</output-bad>
<output-good>
<output-good version="12c-below">
SELECT
"t1"."订单ID" AS "order_id",
"t1"."金额" AS "amount",
Expand All @@ -90,14 +123,22 @@ template:
SELECT DATE, status FROM PUBLIC.USERS -- 错误：未处理关键字和引号
SELECT "DATE", ROUND(active_ratio) FROM "PUBLIC"."USERS" -- 错误：百分比格式错误
</output-bad>
<output-good>
<output-good version="12c-below">
SELECT
"u"."DATE" AS "create_date",
TO_CHAR("u"."active_ratio" * 100, '990.99') || '%' AS "active_percent"
FROM "PUBLIC"."USERS" "u"
WHERE "u"."status" = 1
AND ROWNUM <= 1000
</output-good>
<output-good version="12c+">
SELECT
"u"."DATE" AS "create_date",
TO_CHAR("u"."active_ratio" * 100, '990.99') || '%' AS "active_percent"
FROM "PUBLIC"."USERS" "u"
WHERE "u"."status" = 1
FETCH FIRST 1000 ROWS ONLY
</output-good>
</example>

<example>
Expand All @@ -108,9 +149,9 @@ template:
count(*) AS "user_count"
FROM "PUBLIC"."USERS" "u"
WHERE "u"."status" = 1
AND ROWNUM <= 100
AND ROWNUM <= 100 -- 严重错误：影响分组结果！
GROUP BY "u"."DEPARTMENT"
ORDER BY "department_name" -- 错误：ROWNUM 应当写在最外层，这样会导致查询结果条数比实际数据的数量少
ORDER BY "department_name"
</output-bad>
<output-bad>
SELECT "department_name", "user_count" FROM
Expand All @@ -123,7 +164,7 @@ template:
ORDER BY "department_name"
WHERE ROWNUM <= 100 -- 错误：语法错误，同级内只能有一个WHERE
</output-bad>
<output-good>
<output-good version="12c-below">
SELECT "department_name", "user_count" FROM (
SELECT
"u"."DEPARTMENT" AS "department_name",
Expand All @@ -133,12 +174,22 @@ template:
GROUP BY "u"."DEPARTMENT"
ORDER BY "department_name"
)
WHERE ROWNUM <= 100 -- 外层限制（确保最终结果可控）
WHERE ROWNUM <= 100 -- 正确，在外层限制数量（确保最终结果可控）
</output-good>
<output-good version="12c+">
SELECT
"u"."DEPARTMENT" AS "department_name",
count(*) AS "user_count"
FROM "PUBLIC"."USERS" "u"
WHERE "u"."status" = 1
GROUP BY "u"."DEPARTMENT"
ORDER BY "department_name"
FETCH FIRST 100 ROWS ONLY
</output-good>
</example>
</basic-examples>

example_engine: Oracle 19c
example_engine: Oracle 11g
example_answer_1: |
{"success":true,"sql":"SELECT \"country\" AS \"country_name\", \"continent\" AS \"continent_name\", \"year\" AS \"year\", \"gdp\" AS \"gdp\" FROM \"Sample_Database\".\"sample_country_gdp\" ORDER BY \"country\", \"year\"","tables":["sample_country_gdp"],"chart-type":"line"}
example_answer_1_with_limit: |
Expand Down
40 changes: 34 additions & 6 deletions backend/templates/template.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,37 @@ template:

{data_training}
sql:
process_check: |
<SQL-Generation-Process>
<step>1. 分析用户问题，确定查询需求</step>
<step>2. 根据表结构生成基础SQL</step>
<step>3. <strong>强制检查：应用数据量限制规则</strong></step>
<step>4. 应用其他规则（引号、别名等）</step>
<step>5. <strong>强制检查：检查语法是否正确？</strong></step>
<step>6. 确定图表类型</step>
<step>7. 返回JSON结果</step>
</SQL-Generation-Process>
query_limit: |
<rule priority="high">
1. <strong>必须遵守</strong>：所有生成的SQL必须包含数据量限制
2. <strong>默认限制</strong>：1000条（除非用户明确指定其他数量）
<rule priority="critical" id="data-limit-policy">
<title>数据量限制策略（必须严格遵守 - 零容忍）</title>
<requirements>
<requirement level="must-zero-tolerance">所有生成的SQL必须包含数据量限制，这是强制要求</requirement>
<requirement level="must">默认限制：1000条（除非用户明确指定其他数量）</requirement>
<requirement level="must">忘记添加数据量限制是不可接受的错误</requirement>
</requirements>

<enforcement>
<action>如果生成的SQL没有数据量限制，必须重新生成</action>
<action>在最终返回前必须验证限制是否存在</action>
</enforcement>
</rule>
no_query_limit: |
<rule priority="high">
如果没有指定数据条数的限制，则查询的SQL默认返回全部数据
<rule priority="critical" id="data-limit-policy">
<title>数据量限制策略（必须严格遵守）</title>
<requirements>
<requirement>默认不限制数据量，返回全部数据（除非用户明确指定其他数量）</requirement>
<requirement>不要臆测场景可能需要的数据量限制，以用户明确指定的数量为准</requirement>
</requirements>
</rule>
system: |
<Instruction>
Expand All @@ -27,9 +50,10 @@ template:
<sql-examples>：提供一组SQL示例，你可以参考这些示例来生成你的回答，其中<question>内是提问，<suggestion-answer>内是对于该<question>提问的解释或者对应应该回答的SQL示例。
若有<Other-Infos>块，它会提供一组<content>，可能会是额外添加的背景信息，或者是额外的生成SQL的要求，请结合额外信息或要求后生成你的回答。
用户的提问在<user-question>内，<error-msg>内则会提供上次执行你提供的SQL时会出现的错误信息，<background-infos>内的<current-time>会告诉你用户当前提问的时间
你必须遵守<Rules>内规定的生成SQL规则
你必须遵守<SQL-Generation-Process>内规定的检查步骤生成你的回答
</Instruction>

你必须遵守以下规则:
<Rules>
<rule>
请使用语言：{lang} 回答，若有深度思考过程，则思考过程也需要使用 {lang} 输出
Expand Down Expand Up @@ -90,6 +114,8 @@ template:
</rule>
</Rules>

{process_check}

{basic_sql_examples}

<example>
Expand Down Expand Up @@ -369,6 +395,8 @@ template:

### 以往提问:
{old_questions}

/no_think
analysis:
system: |
<Instruction>
Expand Down