Skip to content

Add single-pass component tree collector and compiler plugin foundations#6260

Open
FarhanAliRaza wants to merge 6 commits intoreflex-dev:mainfrom
FarhanAliRaza:compiler-hooks
Open

Add single-pass component tree collector and compiler plugin foundations#6260
FarhanAliRaza wants to merge 6 commits intoreflex-dev:mainfrom
FarhanAliRaza:compiler-hooks

Conversation

@FarhanAliRaza
Copy link
Copy Markdown
Collaborator

@FarhanAliRaza FarhanAliRaza commented Mar 31, 2026

This branch introduces compiler plugin foundations for single-pass page compilation in Reflex.

Changes

  • New reflex/compiler/plugins.py module — Defines the plugin architecture:

    • CompilerPlugin protocol with three async hooks: eval_page, compile_page, and compile_component (async generator for pre/post tree traversal)
    • CompilerHooks dispatcher that runs hooks across an ordered plugin chain with stop-on-first-result semantics for eval_page and middleware-style enter/unwind for compile_component
    • BaseContext async context manager backed by ContextVar for task-local state, with each subclass getting its own context var
    • PageContext for mutable per-page compilation state (imports, hooks, custom code, dynamic imports, refs, app-wrap components)
    • CompileContext for top-level compilation orchestration across all pages
  • Comprehensive test suite (tests/units/compiler/test_plugins.py, 435 lines) — Covers plugin dispatch ordering, stop-on-result, component tree traversal (children before prop-components), context lifecycle/cleanup, data accumulation, duplicate route rejection, and factory isolation

  • Flaky Redis test fix — Replaces a racy modify_2_started event with a counter-based contenders_started mechanism that only fires after both contending tasks have started, fixing intermittent failures in test_oplock_contention_queue and test_oplock_contention_no_lease

  • Minor: Added asend to codespell ignore list, fixed benchmark script import, removed problematic __slots__ usage

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

Please delete options that are not relevant.

  • New feature (non-breaking change which adds functionality)

New Feature Submission:

  • Does your submission pass the tests?
  • Have you linted your code locally prior to submission?

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

closes #6210

Replace 6 separate recursive tree walks (_get_all_imports, _get_all_hooks,
_get_all_custom_code, _get_all_dynamic_imports, _get_all_refs,
_get_all_app_wrap_components) with a single collect_component_tree_artifacts
walk that gathers all compilation data in one pass.

Wire the new collector into app.py, compiler.py, and utils.py. Add
CompilerPlugin protocol, CompilerHooks dispatcher, and BaseContext/
PageContext/CompileContext types as foundations for the async plugin
pipeline.
@FarhanAliRaza FarhanAliRaza marked this pull request as draft March 31, 2026 16:50
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Mar 31, 2026

Greptile Summary

This PR lays the plugin-architecture foundations for single-pass page compilation in Reflex, introducing reflex/compiler/plugins.py with CompilerPlugin protocol, CompilerHooks dispatcher, and BaseContext/PageContext/CompileContext task-local context managers. It also fixes a flaky Redis oplock-contention test and ships a comprehensive 435-line test suite.

Key changes:

  • CompilerPlugin protocol with three async hooks (eval_page, compile_page, compile_component as an async generator for pre/post tree traversal)
  • CompilerHooks implements stop-on-first-result semantics for eval_page and middleware-style enter/unwind for compile_component
  • BaseContext.__init_subclass__ creates a per-subclass ContextVar, giving each context type task-local isolation without requiring manual registration
  • PageContext accumulates imports, hooks, custom code, dynamic imports, refs, and app-wrap components; CompileContext orchestrates multi-page compilation with duplicate-route detection
  • Redis test fix replaces a racy shared event (set by whichever of two tasks happened to run first) with a counter that only fires after both contenders have executed past their synchronous setup

Findings:

  • Prop-component traversal in compile_component discards the return value — no inline comment at the call site to warn plugin authors
  • Unwind loop's _compile_children call can receive None if a plugin yields a malformed tuple — a defensive assertion would give a clearer error
  • mark_contender_started is defined twice with identical logic in two test functions

Confidence Score: 5/5

Safe to merge — all remaining findings are P2 style and documentation suggestions with no impact on runtime correctness.

The core plugin machinery is well-designed and the test coverage is thorough. All three findings are P2: a missing inline comment on the intentional prop-component side-effect-only traversal, a defensive assertion that would improve error messages for malformed plugin generators (not triggerable by correctly-typed plugins), and duplicated test helper logic. None affect correctness or reliability.

reflex/compiler/plugins.py — the prop-component traversal asymmetry and the unwind-loop None-children path are worth a second look before the plugin API is widely adopted.

Important Files Changed

Filename Overview
reflex/compiler/plugins.py New module introducing plugin protocol, CompilerHooks dispatcher, BaseContext/PageContext/CompileContext — well-structured; minor issue with silently-discarded prop-component return values and a latent None-children path in the unwind loop.
tests/units/compiler/test_plugins.py Comprehensive 435-line test suite covering plugin dispatch ordering, tree traversal, context lifecycle, data accumulation, duplicate-route rejection, and factory isolation — thorough and well-organized.
tests/units/istate/manager/test_redis.py Flaky-test fix replaces racy single-event pattern with a counter that fires only after both contending tasks have started; correct for asyncio's cooperative scheduler. Minor: mark_contender_started logic is duplicated in two test functions.
pyproject.toml Adds "asend" to codespell's ignore list to suppress false positives from the async generator send API.

Sequence Diagram

sequenceDiagram
    participant CC as CompileContext.compile()
    participant CH as CompilerHooks
    participant PC as PageContext
    participant P as Plugin (ordered chain)

    CC->>CH: eval_page(page_fn) [stop-on-first]
    CH->>P: eval_page() — first plugin returning non-None wins
    P-->>CH: PageContext
    CH-->>CC: PageContext

    CC->>PC: async with page_ctx (attach ContextVar)

    loop For each structural child
        CC->>CH: compile_component(comp)
        CH->>P: plugin.compile_component() → AsyncGenerator
        Note over P: PRE: anext(gen) — enter phase (registration order)
        CH->>CH: _compile_children(structural_children) [recursive]
        CH->>CH: compile_component(prop_components) [side-effects only]
        Note over P: POST: gen.asend((comp, children)) — unwind (reverse order)
        CH-->>CC: compiled_component
    end

    CC->>CH: compile_page(page_ctx)
    CH->>P: compile_page() — all plugins, registration order
    P-->>CH: (void)

    CC->>PC: exit async with (detach ContextVar)
    CC->>CC: compiled_pages[route] = page_ctx
Loading

Reviews (2): Last reviewed commit: "removed the extra stuff no related to is..." | Re-trigger Greptile

@codspeed-hq
Copy link
Copy Markdown

codspeed-hq bot commented Mar 31, 2026

Merging this PR will not alter performance

✅ 8 untouched benchmarks


Comparing FarhanAliRaza:compiler-hooks (ac4af22) with main (57119bc)

Open in CodSpeed

@FarhanAliRaza FarhanAliRaza marked this pull request as ready for review April 1, 2026 11:07
@FarhanAliRaza FarhanAliRaza requested a review from masenf April 1, 2026 11:08
Copy link
Copy Markdown
Collaborator

@masenf masenf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a test case for the child replacement logic when traversing the tree; we might not strictly need it now, but it will be important when moving the StatefulComponent compilation into the plugin system.


if isinstance(compiled_component, Component):
for prop_component in compiled_component._get_components_in_props():
await self.compile_component(prop_component, **kwargs)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at first glance it seems like we might need to do something with the potential replacement value returned by this function, like applying it as the prop value on the compiled_component if it differs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Single-pass compiler: Introduce CompilerPlugin protocol, CompilerHooks, and context objects

2 participants