Separate FORMAT parameter from EXPLAIN verbosity option in explain API

[coderabbitai]

Description

The format query parameter in the explain API is currently overloaded to serve two distinct purposes:

1. Output format control (JSON, CSV, YAML) - controls how the response is serialized
2. Explain verbosity level (SIMPLE, STANDARD, EXTENDED, COST) - controls the content/detail level of the explain output

This violates the single responsibility principle and creates confusion, as FORMAT should only control the serialization format of the response, not the semantic content.

Proposal

Introduce a new query parameter (e.g., explain_level, explain_option, or verbosity) specifically for controlling the explain verbosity, while keeping format strictly for output serialization format.

Current behavior:

/_plugins/_ppl/_explain?format=extended  // controls BOTH verbosity AND output format

Proposed behavior:

/_plugins/_ppl/_explain?format=json&explain_level=extended  // separate concerns
/_plugins/_ppl/_explain?format=yaml&explain_level=standard

Context

This issue was identified during code review of PR #4914 (RexCall and RelDataType standardization for script push down).

Related discussion: #4914 (comment)

Benefits

1. Clear separation of concerns: FORMAT controls serialization, explain option controls content
2. Better API design: Each parameter has a single, well-defined responsibility
3. Improved extensibility: Easier to add new output formats or explain levels independently
4. Reduced confusion: Users understand what each parameter does

Implementation considerations

• Maintain backward compatibility by supporting the old format parameter behavior
• Add deprecation warnings for the old usage pattern
• Update documentation to reflect the new parameter
• Update integration tests

Reported by: @qianheng-aws

[LantaoJin]

Can we simplify to /_plugins/_ppl/_explain?format=yaml&mode=standard or /_plugins/_ppl/_explain?format=yaml&level=standard? @qianheng-aws