Fix #714: Make TAGCOUNT always greater than the last instantiated Tag's count

[Technici4n]

This fixes #714:

     Testing Running tests...
Tag count of Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,1) is 1
Tag count of Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#1#2"{ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}}, ForwardDiff.Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}, Int64, 1}}}(Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,0),Dual{ForwardDiff.Tag{FDPrecompilationIssue.var"#3#4"{Int64}, Int64}}(0,2)) is 2
     Testing FDPrecompilationIssue tests passed

This adds an atomic operation for each Tag instantiation. This might be problematic when computing many fast derivatives in a tight loop?

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.76%. Comparing base (7e52ffa) to head (4cc51ad).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #724   +/-   ##
=======================================
  Coverage   90.75%   90.76%           
=======================================
  Files          11       11           
  Lines        1071     1072    +1     
=======================================
+ Hits          972      973    +1     
  Misses         99       99           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[devmotion]

Is there any way to opt out of precompilation? The design of tagcount is really that no method must exist at the start of a Julia session and tagcount is just defined when it is called with a specific F and V type for the first time.

[ChrisRackauckas-Claude]

Tested locally on master OrdinaryDiffEq.jl + this branch (with our local widen fix reverted): it does not fix SciML/OrdinaryDiffEq.jl#3381, the discourse MWE, or SciML/NonlinearSolve.jl#932. All three still throw the same MethodError: no method matching Float64(::Dual{...nested...}) (and #3381's AutoSpecialize variant fails with NoFunctionWrapperFoundError).

The Threads.atomic_max! only fires inside the runtime Tag(f, V) constructor; those bug classes either go through type-only Tag{F,V} construction (so atomic_max never engages) or are downstream buffer-type mismatches (DI's pushforward emulation writing into a Vector{Float64} cotangent buffer, FunctionWrappersWrappers slot signatures missing nested-Dual p) — orthogonal to tag-ordering.

Still a useful fix for #714, just not a structural fix for the SciML-side nested-Dual bugs.

[ChrisRackauckas]

Summary of that is, 3 different downstream MWEs, none of them fixed by this. I don't think this is a real fix.

[ChrisRackauckas]

Is there any way to opt out of precompilation? The design of tagcount is really that no method must exist at the start of a Julia session and tagcount is just defined when it is called with a specific F and V type for the first time.

If you do that, you can never precompile any ForwardDiff calls 😅 . At least not with tags you will end up using, so then precompilation is effectively ignored by ForwardDiff. That is a fix but it's not really what you want of course.

SciML/OrdinaryDiffEq.jl#3587 is super messy though, so I'm looking for if there's a better way to fix it downstream but I don't see one.

[Technici4n]

Yeah I wrote this a year ago but it is indeed not a great fix. Not sure what can be done to make the tags play well with precompilation, but solving this problem would be huge for the ForwardDiff UX 😅

[devmotion]

If you do that, you can never precompile any ForwardDiff calls

Only precompilation of tagcount is problematic, no other types or method specializations. It would be fine to run tagcount during precompilation but without keeping any precompiled version of the method around. That's what I was wondering - could one mark some methods as "I don't want to keep precompiled versions for them around".

[Technici4n]

Hmmm, not so sure. The tag count is used to nest duals in the right order. If you don't precompile the tag count you also can't precompile anything that nests duals. (Which includes any function you might want to differentiate)

[ChrisRackauckas]

I see. Yes we want to precompile the functions with the tag in there, but we don't care about precompiling the tag count call.

[AdityaPandeyCN]

If I remember correctly I was able to fix the bug with this
In ForwardDiff.jl/src/config.jl, replace lines 24-26:

@inline function ≺(::Type{Tag{F1,V1}}, ::Type{Tag{F2,V2}}) where {F1,V1,F2,V2}
    tagcount(Tag{F1,V1}) < tagcount(Tag{F2,V2})
end

with

@inline function ≺(::Type{Tag{F1,V1}}, ::Type{Tag{F2,V2}}) where {F1,V1,F2,V2}
    d1 = V1 <: Dual
    d2 = V2 <: Dual
    if d1 == d2
        return tagcount(Tag{F1,V1}) < tagcount(Tag{F2,V2})
    end
    return !d1
end

When I opened this issue #801 to understand the cause for this SciML/OrdinaryDiffEq.jl#3381 , Not sure if this still hold correctness.

[devmotion]

I'm missing any intuition and motivation for such a change. Why should we perform Dual supertype checks on the value type? Does precompilation generate some incorrect non-Dual tagcount definitions?