Skip to content

Conversation

@LantaoJin
Copy link
Member

@LantaoJin LantaoJin commented Dec 19, 2025

Description

Refactor #3696

Support nested aggregation in PPL only when calcite enabled.

With this PR, follow PPL query is able to execute a nested aggregation query.

source=logs | head 10000 | stats min(pages.load_time)

And it equals the DSL

GET logs/_search
{
  "aggs": {
    "pages": {
      "nested": {
        "path": "pages"
      },
      "aggs": {
        "min_load_time": { "min": { "field": "pages.load_time" } }
      }
    }
  }
}

Deprecated implementation for SQL: #2814

Limitation:

  • PPL only
  • Calcite should be enabled
  • Throw UnsupportedOperationException if pushdown cannot be applied.

Related Issues

Resolves #4949, #2813 and #2529, and #2739

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • New functionality has javadoc added.
  • New functionality has a user manual doc added.
  • New PPL command checklist all confirmed.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff or -s.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Lantao Jin <ltjin@amazon.com>
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 19, 2025

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

Signed-off-by: Lantao Jin <ltjin@amazon.com>
@LantaoJin LantaoJin added the enhancement New feature or request label Dec 20, 2025
Signed-off-by: Lantao Jin <ltjin@amazon.com>
Comment on lines -178 to -179
protected Function0<CalcitePrepare> createPrepareFactory() {
return OpenSearchPrepareImpl::new;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After upgraded to 1.41. this method was not called any more, change it to createPrepare()


@Override
protected PreparedResult implement(RelRoot root) {
Hook.PLAN_BEFORE_IMPLEMENTATION.run(root);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hook was called twice for non-full-scannable plan

HintStrategyTable.builder()
.hintStrategy(
"stats_args",
"agg_args",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename stats_args to agg_args since aggregation was not only happens in stats command.

curl -sS -H 'Content-Type: application/json' \
-X POST localhost:9200/_plugins/_ppl/_explain?format=extended \
-d '{"query" : "source=state_country | where age>30"}'
-d '{"query" : "source=state_country | head 10 | where age>30"}'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

avoid whole-plan-pushdown

Comment on lines 36 to +39
if (result.getCause() == null) {
if (result.getSuppressed().length > 0) {
return result.getSuppressed()[0];
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some exception without cause (root exception), the actual root cause may be attached in its suppressed cause.

return new OpenSearchErrorMessage(exception, exception.status().getStatus());
}
return new ErrorMessage(e, status);
return new ErrorMessage(cause, status);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer to display the root cause instead of wrapping exception.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] top and other bucket-dependent aggregations don't work on nested fields

1 participant