CD pipeline does not work

[Haydnspass]

Unfortunately, the build pipeline does not work when I triggered it. Could you have a look?

https://github.com/TuragaLab/SplinePSF/actions/runs/651998012

[olbris]

That is really strange. I'll look into it. The Mac one is really bizarre...it fails to find the git tool using GitHub's own action to check out the repo? How can that happen? I kind of hope that is a transient problem with their container. But I will check.

[Haydnspass]

Taking a look at the logs another issue is that it does not find CUDA

[Haydnspass]

I simplified the build-upload.yaml 6ed6419
and currently testing it. I think the with github.ref statements are unnecessary and automatic in the git checkout.

[Haydnspass]

To summarise:

• CUDA not found for Windows and Mac

I tried to do a local build with CUDA but I now get:

Python 3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import spline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: /home/lucas/xconda3/envs/spline_test_rc_py38/lib/python3.8/site-packages/spline.cpython-38-x86_64-linux-gnu.so: undefined symbol: __fatbinwrap_38_cuda_device_runtime_compute_86_cpp1_ii_8b1a5d37
>>> 

I think I have seen this issue before ...

[olbris]

This is for the pip Linux build. If you change the runner to an older version, specifically to runs-on: ubuntu-18.04, it works. You only need to change it for the build-linux-pip job. The CUDA compiler is detected, the build succeeds, and if I download the wheel file and install it on Srini's workstation, spline imports and has CUDA compiled and available.

My theory is that something updated recently on the runner. On the Ubuntu 20 (ubuntu-latest) runner, if I was very explicit about telling cmake where to find the CUDA compiler, the build would then fail because the version of cudatoolkit we use could not handle gcc 9 (only 8 or earlier).

I have not merged this change into master yet. I was working on two branches, and they have a bunch of testing stuff that should not be merged. But it's just that one line, so you can make the change on master by hand.

That's all I can do today; I'll check in again on Monday morning (Eastern time).

[Haydnspass]

Hey @olbris ,

thanks! That was an easy solution. I refactored the build a bit so that it is more maintainable.
The only thing missing is now cuda compilation for windows. Currently, CMAKE does not detect the cuda compiler.

[olbris]

I will check that on my local Windows computer and see what's happening.

[olbris]

The conda build seemed to work properly on my Windows 10 computer. I was able to import spline, and CUDA was compiled and available. So I think overall the build works and it must be something about the environment in the GitHub runner. I suppose that's not really a surprise.

I did notice one strange thing, though. I think the Python version matrixing might not be quite right, at least for Windows. I found that the four tar.bz2 archives produced by the Windows build, at least on my computer, all contained the Python 3.7 binary extension. It would not import on 3.9, only 3.7 (I didn't try the other versions)

Unfortunately, I have to switch over to another project today. I will read any updates you add here, and I should be able to spend some more time tomorrow or Thursday.

[Haydnspass]

Yeah same observation for me ... This is however not related to Github but happens locally. Any idea where this could come from? We kind of need to solve this before we can submit ...

[Haydnspass]

Python version is fixed. CUDA still not found.