Custom global equality selector

[jkrumbiegel]

This PR adds a way to globally change the function which determines the equality function to use for a given pair of reference, actual arguments. The reason is that it's currently annoying to annotate every image reference test with by if you don't want to use the default psnr_equality comparison which is too insensitive for my purposes.

You can either change it permanently with default_equality_selector! or temporarily with with_default_equality_selector. The setting is implemented as a simple Ref as the Julia compat is too low for ScopedValues. But ScopedValues might be overkill anyway.

Example:

# we have some image comparison function we want to use by default
custom_image_equality(reference, actual) = ...

# our selector picks `custom_image_equality` for images and `isequal` for the rest
custom_selector(actual, reference) = isequal
custom_selector(reference::AbstractArray{<:Colorant}, actual::AbstractArray{<:Colorant}) =
    custom_image_equality

default_equality_selector!(custom_selector)

# this test now uses `custom_image_equality`
@test_reference "image.png" some_image

[devmotion]

Maybe a - for ReferenceTests simpler - alternative would to not increase the API here but tell downstream packages to use custom functions or macros in their test that set by to some custom comparison function.

[jkrumbiegel]

Yeah that would be the alternative. I thought for the average user it would be a bit complex to set up a macro correctly that forwards everything but by.

[devmotion]

It could be documented in ReferenceTests how it can be done.

[devmotion]

For instance, I think the following would work (untested):

julia> macro test_ref(ref, actual, kwargs...)
           return esc(:($ReferenceTests.@test_reference($ref, $actual, by = $isequal, $(kwargs...))))
       end
@test_ref (macro with 1 method)

julia> @macroexpand @test_ref "test" img
:(ReferenceTests.test_reference(ReferenceTests.abspath(ReferenceTests.joinpath("/Users/david", "test")), img, by = isequal))

julia> @macroexpand @test_ref "test" img a = 5 b = 2
:(ReferenceTests.test_reference(ReferenceTests.abspath(ReferenceTests.joinpath("/Users/david", "test")), img, by = isequal, a = 5, b = 2))

julia> @macroexpand @test_ref "test" img a = 5 b = x
:(ReferenceTests.test_reference(ReferenceTests.abspath(ReferenceTests.joinpath("/Users/david", "test")), img, by = isequal, a = 5, b = x))

julia> @macroexpand @test_ref "test" img a = 5 b = x by = isapprox
:(ReferenceTests.test_reference(ReferenceTests.abspath(ReferenceTests.joinpath("/Users/david", "test")), img, by = isequal, a = 5, b = x, by = isapprox))

[oxinabox]

indeed I often create such functions/macros anyway to map to where in the directory structure the test files live etc.