Move usm_ndarray into dpctl_ext.tensor

[vlad-perevezentsev]

This PR proposes to migrate the tensor interface (usm_ndarray, dlpack, flags) into dpctl_ext/tensor making dpnp independent of dpctl's tensor module.

Updates:

• Introduce dpctl_ext_capi.h
• Implement a clean CMake interface library DpctlExtCAPI to properly propagate generated headers to consumers
• Update remaining imports from dpctl.tensor to dpctl_ext.tensor
• Link all backend extensions against DpctlExtCAPI to ensure consistent access to the C-API

• Have you provided a meaningful PR description?
• Have you added a test, reproducer or referred to an issue with a reproducer?
• Have you tested your changes locally for CPU and GPU devices?
• Have you made sure that new changes do not introduce compiler warnings?
• Have you checked performance impact of proposed changes?
• Have you added documentation for your changes, if necessary?
• Have you added your changes to the changelog?

[antonwolfy]

I guess we have to remove the warning suppress, since it's the last PR removing migration of the source code

[ndgrigorian]

@antonwolfy @vlad-perevezentsev
A thought but should we consider flattening the directory structure? Since currently it's
dpnp -> dpctl_ext -> tensor (actual files...)

Maybe could we flatten to something like dpnp -> tensor (or dpctl_ext)

[antonwolfy]

@antonwolfy @vlad-perevezentsev A thought but should we consider flattening the directory structure? Since currently it's dpnp -> dpctl_ext -> tensor (actual files...)

Maybe could we flatten to something like dpnp -> tensor (or dpctl_ext)

dpnp.tensor sounds reasonable as for me.

[antonwolfy]

Based on warning report seems we still using import dpnp.tensor somewhere in code or tests.
Is it intentional?

[antonwolfy]

Why do we need that and how it worked previously without linking with dpctl lib?

[vlad-perevezentsev]

Previously dpnp4pybind11.hpp included dpctl_capi.h from the external dpctl package which was found via ${Dpctl_INCLUDE_DIRS} set by find_package(Dpctl).

Now dpnp4pybind11.hpp includes dpctl_ext_capi.h which is an internal header,and without linking to DpctlExtCAPI I got an error

dpnp4pybind11.hpp:41:10: fatal error: 'dpctl_ext_capi.h' file not found
   41 | #include "dpctl_ext_capi.h"

[antonwolfy]

But the error is about include, not about linkage.

[antonwolfy]

Is it a temporary approach, or do we need to define interface lib even when it will be exposed by dpctl when the migration is completed?

[ndgrigorian]

It's a design decision either way: dpctl exposed PyUSMArrayObject for numba-dpex's sake. So this way, usm_ndarray would be exposed to C-API and could be interacted with from C code conveniently.

It could be removed, in which case, the api methods like i.e., cdef api char* UsmNDArray_GetData could also be removed (barring causing any problems in the dpctl_ext).