Support for LogExpFunctions && better errors

[lrnv]

Hey,

import TaylorDiff
import LogExpFunctions
t = TaylorDiff.make_seed(1.2, 1.0, Val{3}())
log1p(exp(t))
LogExpFunctions.log1pexp(t)

yields:

julia> import TaylorDiff

julia> import LogExpFunctions

julia> t = TaylorDiff.make_seed(1.2, 1.0, Val{3}())
TaylorScalar{Float64, 3}(1.2, (1.0, 0.0, 0.0))

julia> log1p(exp(t))
TaylorScalar{Float64, 3}(1.4632824673380311, (0.7685247834990176, 0.08894722032340283, -0.015923022053454108))

julia> LogExpFunctions.log1pexp(t)
1.4632824673380311

julia> 

Two questions:

1. How hard would it be to support LogExpFunctions
2. Why, why and why dont we error in those cases ? Took me ages to understand what was going on...

[tansongchen]

The problem is at https://github.com/JuliaStats/LogExpFunctions.jl/blob/master/src/basicfuns.jl#L259 where it defines log1pexp(x::Real) = _log1pexp(float(x)). In TaylorDiff.jl, float() converts a TaylorScalar to its primal type...

I might not be understanding the semantics of float() correctly, so maybe that should be fixed. But in general, we need to create an extension to fully support all the functions in LogExpFunctions. I will look into that next week.

However, it's hard to detect and throw error in these cases, because the freedom of multiple dispatch actually allow one to specify slightly different semantics to the same function interface. Therefore when combining two different packages, they might "just work" but also might not.

[lrnv]

Understood about the difficulty to error, indeed.

The comment on that line says "# ensures that BigInt/BigFloat, Int/Float64 etc. dispatch to the same algorithm", so float() for them is simply used to convert ints to floats, you you use it to return a true float.

You could look at what float() dispatches to on forwardiff's Dual{} type for example, to decide what to do. Thanks for your fast answer anyway, waiting for your input :)

[tansongchen]

I agree, looking at ForwardDiff should be a good reference.