Skip to content

Conversation

@yuancu
Copy link
Collaborator

@yuancu yuancu commented Dec 25, 2025

Description

Status: blocked by #4892 , diff can be viewed from 42a7386

Lambda function is used in transform function:

## TRANSFORM
### Description
Usage: `transform(array, function)` transform the element of array one by one using lambda. The lambda function can accept one single input or two input. If the lambda accepts two argument, the second one is the index of element in array.
Argument type: array:ARRAY, function:LAMBDA
Return type: ARRAY
Example
```ppl
source=people
| eval array = array(1, -2, 3), result = transform(array, x -> x + 2)
| fields result
| head 1
```

When defining a lambda function, it's parameter type needs to be explicitly specified. Otherwise, the parameters will be deemed of type ANY per the constructor of SqlLambdaScope, blocking the subsequent type inference.

public SqlLambdaScope(
    SqlValidatorScope parent, SqlLambda lambdaExpr) {
  super(parent);
  this.lambdaExpr = lambdaExpr;

  // default parameter type is ANY
  final RelDataType any =
      validator.typeFactory.createTypeWithNullability(
          validator.typeFactory.createSqlType(SqlTypeName.ANY), true);
  parameterTypes = new HashMap<>();
  lambdaExpr.getParameters().forEach(param -> parameterTypes.put(param.toString(), any));
}

For the expressiontransform(array, (x, i) -> x + i) , x should be of array's element type, while i should be of integer type (it represents index). This PR passes on this semantic via a SqlOperandTypeInference definition.

TODOs:

  • Make lambda return type inferrable
  • Yaml test, Explain IT, integration test

Related Issues

Resolves #4972

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…checking

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

# Conflicts:
#	core/src/main/java/org/opensearch/sql/executor/QueryService.java

# Conflicts:
#	core/src/main/java/org/opensearch/sql/executor/QueryService.java
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
… logics

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
- 2 more ITs passed in PPLBuiltinFunctionIT

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

# Conflicts:
#	core/src/main/java/org/opensearch/sql/expression/function/PPLFuncImpTable.java
- this fix testRand, where desrialization of sarg does not restore its type
- todo: update the toRex in ExtendedRelJson to the align with the latest version

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…estamp; (time, timestamp) -> timestamp (1240/1599)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…2/1872)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
- allow type cast
- rewrite call to sql compare to custom ip comapre

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

# Conflicts:
#	core/src/main/java/org/opensearch/sql/expression/function/PPLBuiltinOperators.java
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…1356/1599 | 1476/1915)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…d in mvindex's implementation (1580/2015)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…iting (1579/2015)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…pe hint

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…e inference (1701/2015)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…lLibrary.SPARK from lookup table to reduce confusion (1717/2018)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…y and isblank (1738/2018)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…018)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…ns are passed on

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…066)

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Update big5 explain results

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Fix clickbench explain ITs

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Update 35 explain IT plans

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

wip

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Fix pushdown calcite explain ITs

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>

Fix no pushdown ITs

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
…king is deferred to the validation phase against the finished logical plan

Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 25, 2025

Important

Review skipped

Ignore keyword(s) in the title.

⛔ Ignored keywords (3)
  • WIP
  • DO NOT MERGE
  • DRAFT

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

@yuancu yuancu added enhancement New feature or request bugFix labels Dec 25, 2025
Signed-off-by: Yuanchun Shen <yuanchu@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugFix enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Make return type inferrable for TRANSFORM and other lambda functions

1 participant