Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 198 additions & 0 deletions .kiro/settings/lsp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
{
"languages": {
"cpp": {
"name": "clangd",
"command": "clangd",
"args": [
"--background-index"
],
"file_extensions": [
"cpp",
"cc",
"cxx",
"c",
"h",
"hpp",
"hxx"
],
"project_patterns": [
"CMakeLists.txt",
"compile_commands.json",
"Makefile"
],
"exclude_patterns": [
"**/build/**",
"**/cmake-build-**/**"
],
"multi_workspace": false,
"initialization_options": {},
"request_timeout_secs": 60
},
"rust": {
"name": "rust-analyzer",
"command": "rust-analyzer",
"args": [],
"file_extensions": [
"rs"
],
"project_patterns": [
"Cargo.toml"
],
"exclude_patterns": [
"**/target/**"
],
"multi_workspace": false,
"initialization_options": {
"cargo": {
"buildScripts": {
"enable": true
}
},
"diagnostics": {
"enable": true,
"enableExperimental": true
},
"workspace": {
"symbol": {
"search": {
"scope": "workspace"
}
}
}
},
"request_timeout_secs": 60
},
"typescript": {
"name": "typescript-language-server",
"command": "typescript-language-server",
"args": [
"--stdio"
],
"file_extensions": [
"ts",
"js",
"tsx",
"jsx"
],
"project_patterns": [
"package.json",
"tsconfig.json"
],
"exclude_patterns": [
"**/node_modules/**",
"**/dist/**"
],
"multi_workspace": false,
"initialization_options": {
"preferences": {
"disableSuggestions": false
}
},
"request_timeout_secs": 60
},
"java": {
"name": "jdtls",
"command": "jdtls",
"args": [],
"file_extensions": [
"java"
],
"project_patterns": [
"pom.xml",
"build.gradle",
"build.gradle.kts",
".project"
],
"exclude_patterns": [
"**/target/**",
"**/build/**",
"**/.gradle/**"
],
"multi_workspace": false,
"initialization_options": {
"settings": {
"java": {
"compile": {
"nullAnalysis": {
"mode": "automatic"
}
},
"configuration": {
"annotationProcessing": {
"enabled": true
}
}
}
}
},
"request_timeout_secs": 60
},
"go": {
"name": "gopls",
"command": "gopls",
"args": [],
"file_extensions": [
"go"
],
"project_patterns": [
"go.mod",
"go.sum"
],
"exclude_patterns": [
"**/vendor/**"
],
"multi_workspace": false,
"initialization_options": {
"usePlaceholders": true,
"completeUnimported": true
},
"request_timeout_secs": 60
},
"python": {
"name": "pyright",
"command": "pyright-langserver",
"args": [
"--stdio"
],
"file_extensions": [
"py"
],
"project_patterns": [
"pyproject.toml",
"setup.py",
"requirements.txt",
"pyrightconfig.json"
],
"exclude_patterns": [
"**/__pycache__/**",
"**/venv/**",
"**/.venv/**",
"**/.pytest_cache/**"
],
"multi_workspace": false,
"initialization_options": {},
"request_timeout_secs": 60
},
"ruby": {
"name": "solargraph",
"command": "solargraph",
"args": [
"stdio"
],
"file_extensions": [
"rb"
],
"project_patterns": [
"Gemfile",
"Rakefile"
],
"exclude_patterns": [
"**/vendor/**",
"**/tmp/**"
],
"multi_workspace": false,
"initialization_options": {},
"request_timeout_secs": 60
}
}
}
51 changes: 38 additions & 13 deletions api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This module provides components organized into two main areas aligned with the [

### Unified Language Specification

- **`UnifiedQueryPlanner`**: Accepts PPL (Piped Processing Language) queries and returns Calcite `RelNode` logical plans as intermediate representation.
- **`UnifiedQueryPlanner`**: Accepts PPL (Piped Processing Language), OpenSearch SQL, and ANSI SQL queries and returns Calcite `RelNode` logical plans as intermediate representation.
- **`UnifiedQueryTranspiler`**: Converts Calcite logical plans (`RelNode`) into SQL strings for various target databases using different SQL dialects.

### Unified Execution Runtime
Expand All @@ -17,7 +17,7 @@ This module provides components organized into two main areas aligned with the [
- **`UnifiedFunction`**: Engine-agnostic function interface that enables functions to be evaluated across different execution engines without engine-specific code duplication.
- **`UnifiedFunctionRepository`**: Repository for discovering and loading functions as `UnifiedFunction` instances, providing a bridge between function definitions and external execution engines.

Together, these components enable complete workflows: parse PPL queries into logical plans, transpile those plans into target database SQL, compile and execute queries directly, or export PPL functions for use in external execution engines.
Together, these components enable complete workflows: parse PPL, SQL, or ANSI SQL queries into logical plans, transpile those plans into target database SQL, compile and execute queries directly, or export PPL functions for use in external execution engines.

### Experimental API Design

Expand All @@ -32,27 +32,53 @@ Together, these components enable complete workflows: parse PPL queries into log
Create a context with catalog configuration, query type, and optional settings:

```java
UnifiedQueryContext context = UnifiedQueryContext.builder()
// PPL
UnifiedQueryContext pplContext = UnifiedQueryContext.builder()
.language(QueryType.PPL)
.catalog("opensearch", opensearchSchema)
.catalog("spark_catalog", sparkSchema)
.defaultNamespace("opensearch")
.cacheMetadata(true)
.setting("plugins.query.size_limit", 200)
.build();

// OpenSearch SQL (default — uses ANTLR parser with OpenSearch UDFs like match())
UnifiedQueryContext sqlContext = UnifiedQueryContext.builder()
.language(QueryType.SQL)
.catalog("opensearch", opensearchSchema)
.defaultNamespace("opensearch")
.build();

// Standard SQL via Calcite's native parser (set conformance to opt in)
UnifiedQueryContext calciteSqlContext = UnifiedQueryContext.builder()
.language(QueryType.SQL)
.conformance(SqlConformanceEnum.DEFAULT)
.catalog("opensearch", opensearchSchema)
.defaultNamespace("opensearch")
.build();

// MySQL-compatible SQL
UnifiedQueryContext mysqlContext = UnifiedQueryContext.builder()
.language(QueryType.SQL)
.conformance(SqlConformanceEnum.MYSQL_5)
.catalog("opensearch", opensearchSchema)
.defaultNamespace("opensearch")
.build();
```

### UnifiedQueryPlanner

Use `UnifiedQueryPlanner` to parse and analyze PPL queries into Calcite logical plans. The planner accepts a `UnifiedQueryContext` and can be reused for multiple queries.
Use `UnifiedQueryPlanner` to parse and analyze queries into Calcite logical plans. The planner accepts a `UnifiedQueryContext` and can be reused for multiple queries within the same language.

```java
// Create planner with context
UnifiedQueryPlanner planner = new UnifiedQueryPlanner(context);
// PPL
UnifiedQueryPlanner planner = new UnifiedQueryPlanner(pplContext);
RelNode plan = planner.plan("source = logs | where status = 200");

// OpenSearch SQL (supports OpenSearch UDFs like match, match_phrase, etc.)
UnifiedQueryPlanner sqlPlanner = new UnifiedQueryPlanner(sqlContext);
RelNode plan = sqlPlanner.plan("SELECT * FROM logs WHERE match(message, 'error')");

// Plan multiple queries (context is reused)
RelNode plan1 = planner.plan("source = logs | where status = 200");
RelNode plan2 = planner.plan("source = metrics | stats avg(cpu)");
// Standard SQL via Calcite's native parser (conformance set on context)
UnifiedQueryPlanner calcitePlanner = new UnifiedQueryPlanner(calciteSqlContext);
RelNode plan = calcitePlanner.plan("SELECT \"name\", COUNT(*) FROM \"logs\" GROUP BY \"name\"");
```

### UnifiedQueryTranspiler
Expand Down Expand Up @@ -226,5 +252,4 @@ public class MySchema extends AbstractSchema {

## Future Work

- Expand support to SQL language.
- Extend planner to generate optimized physical plans using Calcite's optimization frameworks.
1 change: 1 addition & 0 deletions api/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ plugins {

dependencies {
api project(':ppl')
api project(':sql')

testImplementation testFixtures(project(':api'))
testImplementation group: 'junit', name: 'junit', version: '4.13.2'
Expand Down
Loading
Loading