WIP Revive Windows support

[dhiltgen]

Proposed changes

This change partially addresses windows build issues in MLX. The remaining errors are related to add_kernel_node which may require a deeper change to keep MSVC+NVCC happy on windows.

To repro, on a windows system, install:

• Visual Studio 2022
• CUDA 13.0
• cuDNN

I've been building with:

$env:CUDNN_INCLUDE_PATH="C:\Program Files\NVIDIA\CUDNN\v9.15\include\13.0"
$env:CUDNN_LIBRARY_PATH="C:\Program Files\NVIDIA\CUDNN\v9.15\lib\13.0\x64"

cmake -B build -DMLX_BUILD_CUDA=on -DMLX_BUILD_GGUF=OFF -DBUILD_SHARED_LIBS=ON -DMLX_BUILD_PYTHON_STUBS=OFF -DMLX_BUILD_METAL=OFF  -DMLX_BUILD_TESTS=OFF .
cmake --build build --parallel 2>&1 | % ToString | Tee-Object build.log

Checklist

Put an x in the boxes that apply.

• I have read the CONTRIBUTING document
• I have run pre-commit run --all-files to format my code / installed pre-commit prior to committing changes
• I have added tests that prove my fix is effective or that my feature works
• I have updated the necessary documentation (if needed)

[awni]

@zcbenz just FYI @dhiltgen is part of the Ollama team they are hoping to add support for Windows with MLX Cuda back-end. If I understand correctly, there is one function that doesn't compile with MSVC / nvcc on windows still (add_kernel_node). Would be great if you could help take this over the finish line when you are back.

[zcbenz]

@dhiltgen Do you have recommendations for Windows laptops for CUDA development? I usually just do Windows dev in a virtual machine but I guess I have to get Apple buy me a gaming laptop now.