Add 3 new element-wise vector operations

[alexander-novo]

Description

Add a new vector operations:

• scaleInv which performs like scale, but scales by the inverse of a diagonal matrix.
• max which computes a vector whose elements are the maximum of the corresponding elements in the two input vectors.
• abs which computes a vector whose elements are the absolute values of the corresponding element in the input vector.

Supplants #409 and is based on top of #411

Proposed changes

N/A

Checklist

• All tests pass (make test and make test_install per testing instructions). Code tested on
  • CPU backend
  • CUDA backend
  • HIP backend
• I have manually run the non-experimental examples and verified that residuals are close to machine precision. (In your build directory run:
  ./examples/<your_example>.exe -h to get instructions how to run examples). Code tested on:
  • CPU backend
  • CUDA backend
  • HIP backend
• Code compiles cleanly with flags -Wall -Wpedantic -Wconversion -Wextra.
• The new code follows Re::Solve style guidelines.
• There are unit tests for the new code.
• The new code is documented.
• The feature branch is rebased with respect to the target branch.
• I have updated CHANGELOG.md to reflect the changes in this PR. If this is a minor PR that is part of a larger fix already included in the file, state so.

Further comments

N/A

[pelesh]

If we are using BLAS-like naming scheme as in #411, I suggest we rename the methods as follows:

void scaleInv(const Vector* D, Vector* x); // multiply each x[i] with 1./D[i]
void max(const Vector* x, const Vector* y, Vector* z); // z[i] = max(x[i], y[i])
void abs(const Vector* x, Vector* y); // y[i] = abs(x[i])

I would allow in max, for example, to have x==z or y==z for more flexibility.

I also wouldn't hard-wire block size within kernels. Try something like

constexpr index_type block_size = 256;

within the proper namespace.

[alexander-novo]

I also wouldn't hard-wire block size within kernels. Try something like

constexpr index_type block_size = 256;

within the proper namespace.

The rest of the kernels in this module have hard-wired block sizes.

[alexander-novo]

Should be ready to review

[pelesh]

I fixed issues with HIP and CUDA kernels. Would like review by @shakedregev before merging.

[shakedregev]

Fix the naming stuff, otherwise looks good. I still need to run it.

[alexander-novo]

@shakedregev Can you show me where the floating point conventions are described? There are several other .5 values without the 0 in the same file.

[shakedregev]

@shakedregev Can you show me where the floating point conventions are described? There are several other .5 values without the 0 in the same file.

You're correct that the rules don't say that. We will update them. Almost everywhere in the code, including in the file we wrote 0.5, but yes it seems someone violated the unwritten convention there. The rationale is that you want it to be as obvious as possible that this is a number.