Skip to content

[docgen]Refine ConfigOptionsDocGenerator and fix MDX formatting#2588

Open
Prajwal-banakar wants to merge 8 commits intoapache:mainfrom
Prajwal-banakar:fluss-docgen-refinement
Open

[docgen]Refine ConfigOptionsDocGenerator and fix MDX formatting#2588
Prajwal-banakar wants to merge 8 commits intoapache:mainfrom
Prajwal-banakar:fluss-docgen-refinement

Conversation

@Prajwal-banakar
Copy link
Contributor

@Prajwal-banakar Prajwal-banakar commented Feb 6, 2026

Purpose

Linked issue: close #2586

This PR refines the ConfigOptionsDocGenerator introduced in #2474 to produce correct, complete MDX documentation that fully replaces the manual configuration tables in configuration.md. It addresses all outstanding feedback from the #2474 review.

Brief change log

  • Annotation-driven sections: Generator now reads @Documentation.Section from ConfigOptions.java to group options into logical sections. Applied @Documentation.Section to all previously unannotated ConfigOption fields across the entire ConfigOptions.java file.

  • Environment-agnostic defaults: Generator reads @Documentation.OverrideDefault to replace system-dependent runtime values with documentation-friendly placeholders. Applied to client.scanner.io.tmpdir, table.auto-partition.time-zone, server.buffer.wait-timeout, client.writer.buffer.wait-timeout, log.flush.interval-messages, client.lookup.max-retries, client.writer.retries, and zookeeper.client.max-buffer-size.

  • Human-readable defaults: Wired ConfigDocUtils.formatDefaultValue() to format Duration values (including sub-second as milliseconds, e.g. 100 ms) and MemorySize values. Near-infinite values render as infinite.

  • Clean type names: Fixed inner enum type rendering — strips the outer class prefix so NoKeyAssigner renders instead of ConfigOptionsNoKeyAssigner.

  • List format: Output uses ### headings with bullet points instead of tables, as requested in the [doc-gen] Automate configuration documentation via fluss-docgen module #2474 review.

  • Deprecated options: Options annotated with @Deprecated render with a visible deprecation notice and remain in the documentation for reference.

  • Replaced manual tables: configuration.md now imports _partial_config.mdx via MDX import, replacing all manually maintained tables.

  • Removed redundant annotation files: Deleted ConfigSection.java and ConfigOverrideDefault.java since ConfigOptions.java uses the @Documentation.Section and @Documentation.OverrideDefault inner annotations
    already defined in Documentation.java.

Tests

Manually verified by running the ConfigOptionsDocGenerator main method and inspecting the resulting _partial_config.mdx file.

  • ConfigOptionsDocGeneratorTest verifies the generated output includes correct sections, @OverrideDefault values, human-readable Duration formatting, and absence of raw %s artifacts.
  • ./mvnw compile -pl fluss-docgen -am passes cleanly with 0 Checkstyle violations and 0 Spotless violations.

API and Format

This change does not affect the Fluss runtime API or storage format. It only affects the internal documentation generation tool and the resulting .mdx files.

Documentation

This PR replaces all manually maintained configuration tables in website/docs/maintenance/configuration.md with the auto-generated _partial_config.mdx. Future configuration changes in ConfigOptions.java will automatically reflect in the docs upon running the generator.

@Prajwal-banakar
Copy link
Contributor Author

Hi @wuchong,

I've addressed your feedback from the previous PR ( #2474 ) and continued the work here to fulfill the requirements mentioned in your #2474 (comment). I haven't removed the manual tables yet as I'm waiting for your feedback on the automated generation first!

@wuchong
Copy link
Member

wuchong commented Feb 10, 2026

Thank you @Prajwal-banakar , I will continue to review this after 0.9 RC1 is out. Busy preparing the release.

@Prajwal-banakar
Copy link
Contributor Author

Prajwal-banakar commented Feb 10, 2026

Hi Jark! Totally understand, Congratulations to the team on reaching this milestone!

@Prajwal-banakar Prajwal-banakar changed the title [docs]Refine ConfigOptionsDocGenerator and fix MDX formatting [docgen]Refine ConfigOptionsDocGenerator and fix MDX formatting Mar 4, 2026
@Prajwal-banakar Prajwal-banakar force-pushed the fluss-docgen-refinement branch from 1f21a7d to 98c73f4 Compare March 21, 2026 07:54
@Prajwal-banakar
Copy link
Contributor Author

Hi @wuchong , the PR is ready for review. One thing to flag issue #2912 was raised about _partial_config.mdx being regenerated on every full local project build since the exec plugin is bound to the compile phase, making it noisy in git status. Would it be better to change the phase binding to none so it only runs when explicitly called via
./mvnw compile -pl fluss-docgen -am? Happy to address any review comments!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DOCS] Enhance ConfigOptionsDocGenerator for cleaner MDX output and fixed formatting

2 participants