Missing Windows support

[TimDettmers]

Currently, the library uses precompiled Linux binaries. I am unsure how compatible these are with standard PyTorch installs on Windows. It might be that the binaries need to be compiled against mingw32/64 to create functional binaries for Windows.

The most helpful would be a case where a person is able to compile from source and use the library. This will require altering the Makefile file. If this works, we can add instructions on compiling for Windows as a first step before doing a full-scale Windows deployment of binaries on pip.

Since I do not have a Windows machine, any help is wanted on this!

[DeXtmL]

I am able to compile the csrc part on Windows but fail to link (Visual Studio)
FYI, this is the error message. I'm no expert on cuda, so I'm not quite sure where it goes wrong.

To successfully compile on windows MSVC, some parts need to be patched as following:

1. rename pythonInterface.c to pythonInterface.cpp, or visual studio will try using a C compiler for it.

2. add one missing template instantiation like this: (in SIMD.h)
   

3. get unistd.h and getopt.h for windows

4. get pthread for windows

finally, this is just a build test, so I'm not using the Makefile.
That's all for now.
Tested on CUDA toolkit 11.6, windows 11

[DeXtmL]

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module.
This is the end result:
https://github.com/DeXtmL/bitsandbytes-win-prebuilt

The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022.
I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers
Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:

in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.

Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

[TemporalLabsLLC-SOL]

Thank you. This might be the key.

[korakoe]

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt

The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:

in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.

Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

could you provide a makefile for this?

[bmaltais]

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt

The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:

in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.

Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

Where do you put the pre-built file to activate adam?

[davidvfx07]

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt

The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:

in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.

Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

An easy way to always return libbitsandbytes_cuda116.dll would be to insert
if torch.cuda.is_available(): return 'libbitsandbytes_cuda116.dll', None, None, None, None
above
119: if not torch.cuda.is_available(): return 'libsbitsandbytes_cpu.so', None, None, None, None

[davidvfx07]

After some really tedious debugging and tackling various hidden problems, I managed to compile the whole module. This is the end result: https://github.com/DeXtmL/bitsandbytes-win-prebuilt
The binaries are compiled against CUDAToolkit 11.6 and Visual Studio 2022. I am able to make inferences nearly identical to "normal" fp16 version. So this is kind of a confirmation for "it works" shout-out. No vigorous testing was conducted though. @TimDettmers Finally, the "cuda_setup" part of the source code is entirely incompatible with Windows, there are loads of hardcoded routines; so I used a quick makeshift patch instead of making it proper, that's also why I'm not posting my changes or making PR for now. If you are eager to test:

in cuda_setup/main.py:
make evaluate_cuda_setup() always return "libbitsandbytes_cuda116.dll"
in ./cextension.py:
change ct.cdll.LoadLibrary(binary_path) to ct.cdll.LoadLibrary(str(binary_path))

That should do the trick.
Hopefully this can help someone in the Windows territory; let's hope the official windows support come fast.

Where do you put the pre-built file to activate adam?

You put them in site-packages\bitsandbytes

[TinyBeeman]

Appreciate you doing this work, helped unblock me in a big way. I hope bitsandbytes supports Windows directly sooner rather than later, but this is a great stopgap.

[desis123]

@PinPointPing Thanks a lot, it just worked like a charm