chore(audit): audit string expressions across Spark 3.4.3, 3.5.8, 4.0.1

[andygrove]

Which issue does this PR close?

Closes #.

Rationale for this change

Following the same pattern as #4436 (any), #4437 (bit_and), and the earlier date/time audit, this PR audits every supported string expression in Comet against Spark 3.4.3, 3.5.8, and 4.0.1, records the findings inline in the support doc, and applies the support-level consistency fixes the audit surfaced.

What changes are included in this PR?

Support-doc audit notes

Add per-version audit sub-bullets to all 37 supported string SQL function names in docs/source/contributor-guide/spark_expressions_support.md. The notes record:

• whether the Spark class is identical across 3.4.3 / 3.5.8 / 4.0.1
• Spark 4.0 changes (collation widening via StringTypeWithCollation; CollationSupport routing for Upper/Lower/InitCap and the startswith/endswith/contains/instr/replace/translate/trim family; NullIntolerant -> nullIntolerant: Boolean refactor; BinaryPad rewrite for lpad/rpad)
• known divergences between Comet and Spark, each linked to the tracking issue listed below

Support-level consistency fixes (in strings.scala)

• CometInitCap: Incompatible(None) -> Incompatible(Some(reason)) via a shared private val; drop the dead convert override.
• CometLength: extract the BinaryType reason into a shared private val so the getSupportLevel branch and getUnsupportedReasons stay in sync.
• CometRegExpReplace: pass the existing reasons into Incompatible(Some(...)) / Unsupported(Some(...)) via shared private vals so EXPLAIN matches the compatibility-guide text.
• CometStringLPad / CometStringRPad: share the two reason strings via a small PadReasons companion so getSupportLevel and getUnsupportedReasons cannot drift; align wording with backticks.
• CometSubstring / CometLeft / CometRight: lift the literal-argument fallback out of convert and into getSupportLevel so the dispatcher handles it uniformly; replace the convert-time withInfo with a declared Unsupported reason via a shared private val.

Tracking issues filed for follow-up

Higher-risk findings that need a semantics decision or larger work are filed as separate issues and cross-referenced from the support doc:

• [Bug] repeat throws on negative count where Spark returns empty string #4462 repeat throws on negative count where Spark returns empty string
• [Bug] translate uses graphemes vs Spark code points and ignores U+0000 deletion #4463 translate uses graphemes vs Spark code points; ignores U+0000 deletion
• [Bug] bit_length and octet_length error natively for BinaryType input instead of falling back #4464 bit_length and octet_length error natively for BinaryType input instead of falling back
• [Bug] decode ignores Spark 4.0 legacyCharsets and legacyErrorAction flags #4465 decode ignores Spark 4.0 legacyCharsets and legacyErrorAction flags
• [Doc] decode does not appear in auto-generated compatibility docs #4466 decode does not appear in auto-generated compatibility docs
• [Bug] CometCaseConversionBase gates compat inside convert() instead of getSupportLevel #4467 CometCaseConversionBase gates compat inside convert() instead of getSupportLevel

The umbrella Spark 4.0 collation issue is the existing #2190, which the lower/upper sub-bullets reference.

The audit skill update (require issues for prose-only findings, add Spark 4.1.1 to the version list) is split out into #4468.

Audit process

The audit was driven by 7 parallel agents using the audit-comet-expression skill, each handling a related group (case conversion + initcap; length family; trim; left/right/substring/substring_index; lpad/rpad; simple scalar funcs; complex serdes including concat_ws/regexp_replace/repeat/split/decode).

How are these changes tested?

• ./mvnw test -Dsuites="org.apache.comet.CometSqlFileTestSuite expressions/string/" -Dtest=none (42 tests pass)
• ./mvnw test -Dsuites="org.apache.comet.CometStringExpressionSuite" -Dtest=none (33 tests pass)
• Updated the affected fallback-reason assertions in left.sql, string_lpad.sql, string_rpad.sql, and CometStringExpressionSuite.scala to match the new shared reason strings.