feat(stageleft): fix q! span attribution via hidden macro export

[shadaj]

Summary

When code inside a q!() panics at runtime, the backtrace now correctly points to the q!() definition site rather than the splice site.

Background: How stageleft works today (before this PR)

When you write:

#[stageleft::entry]
fn my_entry(_ctx: BorrowBounds<'_>) -> impl Quoted<'_, i32> {
    q!(crate::compute(x + 1))
}

The q!() macro captures the expression as a string and stores it in a closure. At splice time (when the entry macro is invoked), to_tokens() parses that string, rewrites relative paths (crate:: → final_crate::__staged::), and emits the expression as a TokenStream. The generated code looks like:

{
    use final_crate::__staged::*;
    let x__free = captured_value;
    final_crate::__staged::compute(x__free + 1)
}

The problem: all tokens in this output have Span::call_site() spans, so panics, backtraces, and compiler diagnostics point to the splice site — not the original q!() source location.

How it works after this PR

q!() now emits a hidden #[macro_export] macro_rules! alongside the closure. The macro body contains the original expression tokens with their source spans preserved. At splice time, instead of emitting the expression directly, we emit an invocation of this macro. Because macro_rules! preserves definition-site spans in its expansion, the compiled code's debug info points back to the q!() source — so backtraces show the correct file and line.

Full expansion of q!()

Given:

// src/lib.rs, line 5
q!(crate::compute(x + 1))

In non-Rust-Analyzer mode, q!() expands to:

{
    // Hidden macro with preserved source spans in the body
    #[allow(unexpected_cfgs, non_local_definitions, clippy::crate_in_macro_def)]
    #[cfg_attr(not(stageleft_macro), macro_export)]
    #[doc(hidden)]
    macro_rules! __stageleft_quote_src_lib_rs_5_7 {
        ([__sl_p0 = $($__sl_p0:ident)::*, x__free = $x__free:expr,]
         [__sl_p0 :: compute(x__free + 1)]) => {
            {
                #[allow(non_snake_case)]
                {
                    let x__free = $x__free;
                    $($__sl_p0)::*::compute(x__free + 1)
                }
            }
        };
    }

    move |__stageleft_ctx: &_, __output: &mut QuotedOutput, __props: &mut _| {
        // fn() -> T pointers for type inference
        let x__free = FreeVariableWithContext::uninitialized(&x, __stageleft_ctx);

        if true {
            // Capture free variables and store metadata
            __output.captures.push(Capture { ident: "x__free", tokens: ... });
            __output.module_path = module_path!();
            __output.crate_name = option_env!("STAGELEFT_FINAL_CRATE_NAME").unwrap_or(env!("CARGO_PKG_NAME"));
            __output.pkg_name = env!("CARGO_PKG_NAME");
            __output.tokens = "__sl_p0 :: compute (x__free + 1)";
            __output.macro_name = Some("__stageleft_quote_src_lib_rs_5_7");
            __output.relative_paths = &["crate"];
            *__props = Some(...);
            panic!("stageleft: q!() closure completed");
        }

        // Unreachable: type inference block.
        // Invokes the just-defined macro with original path prefixes + uninit values.
        #[allow(unreachable_code, unused_qualifications)]
        {
            let x__free = x__free();
            __stageleft_quote_src_lib_rs_5_7!(
                [__sl_p0 = crate, x__free = x__free,]
                [__sl_p0 :: compute(x__free + 1)]
            )
        }
    }
}

In Rust Analyzer mode, the expansion is simpler (no macro generation, no path rewriting):

{
    move |__stageleft_ctx: &_, __output: &mut QuotedOutput, __props: &mut _| {
        let x__free = FreeVariableWithContext::uninitialized(&x, __stageleft_ctx);

        if true {
            __output.captures.push(...);
            // tokens field is empty in RA mode
            __output.tokens = "";
            __output.macro_name = None;
            ...
            panic!("stageleft: q!() closure completed");
        }

        // Unreachable: uses original paths directly (they resolve locally)
        #[allow(unreachable_code, unused_qualifications)]
        {
            let x__free = x__free();
            crate::compute(x__free + 1)  // original expression, unmodified
        }
    }
}

At splice time

When the entry macro is invoked (e.g., my_entry!()), to_tokens() runs the closure via catch_unwind, extracts the metadata, and emits:

{
    use final_crate::__staged::__deps::*;
    use final_crate::__staged::*;
    use crate::*;  // brings #[macro_export] macros into scope
    __stageleft_quote_src_lib_rs_5_7!(
        [__sl_p0 = final_crate::__staged, x__free = captured_value,]
        [__sl_p0 :: compute(x__free + 1)]
    )
}

Fallback path

When the macro isn't accessible (e.g., q!() was in a #[cfg(test)] module or an example binary), the splice emits the expression directly with path placeholders resolved at runtime:

{
    use final_crate::__staged::__deps::*;
    use final_crate::__staged::*;
    {
        let x__free = captured_value;
        final_crate::__staged::compute(x__free + 1)
    }
}

This loses span attribution but is functionally correct.

Key design decisions

• Macro naming: The macro name is derived from the source file path (relative to crate root) + line + column, ensuring uniqueness and portability across machines.

• Path handling: Relative paths (crate::, self::, super::) in the q!() expression refer to items relative to where the q!() is written. But when the expression is spliced into a different crate (or a different module within the same crate), these paths would resolve incorrectly. Stageleft solves this by rewriting them to point through the __staged module, which re-exports all items as pub.

  In this PR, the rewriting is split into two parts:

  1. At q!() expansion time: Relative path prefixes are replaced with placeholder idents (__sl_p0, __sl_p1, etc.). For example, crate::module::Foo becomes __sl_p0::module::Foo. This happens in the same visitor pass that detects free variables.
  2. At splice time: The placeholders are resolved to concrete paths (e.g., __sl_p0 → final_crate::__staged) and passed as macro arguments using $($__sl_pN:ident)::* repetition, which allows macro_rules! to concatenate the resolved prefix with the remaining path segments.

  This two-phase approach is necessary because the hidden macro is defined at q!() time (when the final crate name isn't known), but invoked at splice time (when it is). The $($ident)::* pattern is used instead of :path because :path fragments cannot be concatenated with ::suffix in macro_rules!.

• Free variables: Passed as :expr parameters with name = value syntax. The macro body binds them with let name = $name;.

• Literal body match: The second [...] argument contains the expression tokens (post-path-rewrite). This serves as a coherence check — if downstream tooling mutates the AST, the macro pattern won't match.

• Fallback for inaccessible macros: When the q!() is in a #[cfg(test)] module or outside the crate (examples, trybuild), the macro may not be accessible. In these cases, the splice emits the expression directly with path resolution done at runtime.

• Rust Analyzer mode: All macro generation is skipped for performance; paths are left as-is for local resolution.

• Single-pass visitor: The FreeVariableVisitor now handles both free variable detection and path prefix rewriting in one traversal (when not in RA mode).

Other changes

• stageleft_tool: gen_macro emits STAGELEFT_FINAL_CRATE_MANIFEST_DIR for portable path resolution. New gen_staged(bool) unifies deps/test-module/lib-pub generation with a single source parse.
• stageleft_tool: Registers #[cfg(test)] module paths via a ctor, enabling the fallback detection at splice time.
• CI: Removed CARGO_PROFILE_*_STRIP: debuginfo so backtrace tests can verify line numbers.
• proc-macro2 bumped to 1.0.103 for Span::file() API.

[Benjscho]

Is there any extra runtime cost in using catch_unwind?

[shadaj]

Unfortunately, absolutely (especially with the global mutex). But a panic is the only way to get type inference to go through without executing the code being quoted. The earlier implementation (well before this PR, this diff is just a refactor) used MaybeUninit but we found out later that this was completely undefined behavior and we were getting lucky that it worked 🙃

[Benjscho]

Gotcha, thanks for the context!

[akainth015]

What on earth is this doing?

[akainth015]

The quoted crate, self and super identifiers are being mapped to what here? The order in which they appeared in the quoted code?

[shadaj]

Correct, and that index will then be used for the __sl_p{i} macro inputs that are used to feed the resolved paths at splice-time.

[luckyworkama]

It would be nice if you could update the PR description to document roughly how it works today as well as how it works after this change, I think that would be a nice addition.

[shadaj]

It would be nice if you could update the PR description to document roughly how it works today as well as how it works after this change, I think that would be a nice addition.

Added a summary that does a decent job of capturing the PR. (disclaimer: AI generated but I did a couple rounds of polish to make sure it actually reflects my creative intent)

[MingweiSamuel]

Are there no sort of tricks with Span::mixed_site() and resolved_at/located_at to make this work without the extra codegen?

[shadaj]

Are there no sort of tricks with Span::mixed_site() and resolved_at/located_at to make this work without the extra codegen?

Nope, basically the issue is that we would want the generated (trybuild) code to effectively have a foreign span. This is a special case where the foreign span originates in Rust code, so we can pull this off on stable with a declarative macro.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 3 comments.