breaks for 4 dimensions #980
Description
System Info
A100 cuda 12.1
Package Version Editable project location
absl-py 2.0.0
accelerate 0.26.1
aiohttp 3.9.1
aiosignal 1.3.1
annotated-types 0.6.0
antlr4-python3-runtime 4.9.3
anyio 4.0.0
argon2-cffi 23.1.0
argon2-cffi-bindings 21.2.0
arrow 1.3.0
asttokens 2.4.1
async-lru 2.0.4
async-timeout 4.0.3
attrs 23.1.0
Babel 2.13.1
beautifulsoup4 4.12.2
bitsandbytes 0.42.0
bleach 6.1.0
certifi 2022.12.7
cffi 1.16.0
chardet 5.2.0
charset-normalizer 2.1.1
click 8.1.7
colorama 0.4.6
comm 0.2.0
DataProperty 1.0.1
datasets 2.16.1
debugpy 1.8.0
decorator 5.1.1
defusedxml 0.7.1
dill 0.3.7
distro 1.9.0
einops 0.7.0
exceptiongroup 1.1.3
executing 2.0.1
fastjsonschema 2.19.0
filelock 3.9.0
fqdn 1.5.1
frozenlist 1.4.1
fsspec 2023.10.0
h11 0.14.0
httpcore 1.0.2
httpx 0.26.0
huggingface-hub 0.20.2
idna 3.4
ipykernel 6.26.0
ipython 8.17.2
ipywidgets 8.1.1
isoduration 20.11.0
jedi 0.19.1
Jinja2 3.1.2
joblib 1.3.2
json5 0.9.14
jsonlines 4.0.0
jsonpointer 2.4
jsonschema 4.19.2
jsonschema-specifications 2023.11.1
jupyter 1.0.0
jupyter_client 8.6.0
jupyter-console 6.6.3
jupyter_core 5.5.0
jupyter-events 0.9.0
jupyter-lsp 2.2.0
jupyter_server 2.10.1
jupyter_server_terminals 0.4.4
jupyterlab 4.0.8
jupyterlab-pygments 0.2.2
jupyterlab_server 2.25.1
jupyterlab-widgets 3.0.9
lm-eval 0.3.0 /home/cdhernandez/local/lm-evaluation-harness
MarkupSafe 2.1.3
matplotlib-inline 0.1.6
mbstrdecoder 1.1.3
mistune 3.0.2
mpmath 1.2.1
multidict 6.0.4
multiprocess 0.70.15
nbclient 0.9.0
nbconvert 7.11.0
nbformat 5.9.2
nest-asyncio 1.5.8
networkx 3.0rc1
ninja 1.10.2
nltk 3.8.1
notebook 7.0.6
notebook_shim 0.2.3
numexpr 2.8.8
numpy 1.24.1
nvidia-cublas-cu12 12.1.3.1
nvidia-cuda-cupti-cu12 12.1.105
nvidia-cuda-nvrtc-cu12 12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12 8.9.2.26
nvidia-cufft-cu12 11.0.2.54
nvidia-curand-cu12 10.3.2.106
nvidia-cusolver-cu12 11.4.5.107
nvidia-cusparse-cu12 12.1.0.106
nvidia-nccl-cu12 2.19.3
nvidia-nvjitlink-cu12 12.1.105
nvidia-nvtx-cu12 12.1.105
omegaconf 2.3.0
openai 1.7.2
overrides 7.4.0
packaging 23.2
pandas 2.1.4
pandocfilters 1.5.0
parso 0.8.3
pathvalidate 3.2.0
peft 0.7.1
pexpect 4.8.0
Pillow 9.3.0
pip 23.3.1
platformdirs 4.0.0
portalocker 2.8.2
prometheus-client 0.18.0
prompt-toolkit 3.0.41
psutil 5.9.6
ptyprocess 0.7.0
pure-eval 0.2.2
pyarrow 14.0.2
pyarrow-hotfix 0.6
pybind11 2.11.1
pycountry 23.12.11
pycparser 2.21
pydantic 2.5.3
pydantic_core 2.14.6
Pygments 2.16.1
pytablewriter 1.2.0
python-dateutil 2.8.2
python-json-logger 2.0.7
pytorch-triton 2.2.0+e28a256d71
pytz 2023.3.post1
PyYAML 6.0.1
pyzmq 25.1.1
qtconsole 5.5.1
QtPy 2.4.1
referencing 0.31.0
regex 2023.12.25
requests 2.31.0
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rouge-score 0.1.2
rpds-py 0.12.0
sacrebleu 1.5.0
safetensors 0.4.1
scikit-learn 1.3.2
scipy 1.11.4
segment-anything 1.0
Send2Trash 1.8.2
sentencepiece 0.1.99
setuptools 68.0.0
six 1.16.0
sniffio 1.3.0
soupsieve 2.5
sqlitedict 2.1.0
stack-data 0.6.3
sympy 1.11.1
tabledata 1.3.3
tcolorpy 0.1.4
terminado 0.18.0
threadpoolctl 3.2.0
tinycss2 1.2.1
tokenizers 0.15.0
tomli 2.0.1
torch 2.3.0.dev20240110+cu121
torchao 0.0.1
torchaudio 2.2.0.dev20240110+cu121
torchvision 0.18.0.dev20240110+cu121
tornado 6.3.3
tqdm 4.66.1
tqdm-multiprocess 0.0.11
traitlets 5.13.0
transformers 4.36.2
typepy 1.3.2
types-python-dateutil 2.8.19.14
typing_extensions 4.8.0
tzdata 2023.4
uri-template 1.3.0
urllib3 1.26.13
wcwidth 0.2.10
webcolors 1.13
webencodings 0.5.1
websocket-client 1.6.4
wheel 0.41.2
widgetsnbextension 4.0.9
xxhash 3.4.1
yarl 1.9.4
zstandard 0.22.0
Reproduction
https://github.com/TimDettmers/bitsandbytes/blob/main/bitsandbytes/functional.py#L445
if dims>3 it just leaves rows empty which causes an error if to_order isn't "row" or "column"
repro with
mod=bitsandbytes.nn.Linear8bitLt(1280, 3840, bias=True, has_fp16_weights=False).to(0).cuda()
mod(torch.randn((400,14,14,1280) , device='cuda', dtype=torch.float16))
UnboundLocalError: local variable 'rows' referenced before assignment
note even after fixing that there are further issues
AttributeError: 'NoneType' object has no attribute 'dtype'
Expected behavior
not breaking