Not working on Linux?

[omeganinjaduck]

Can anyone please confirm if this work on Linux? Thank you

[contentis]

It hasn't been tested as much on Linux, but it works on my Ubuntu (22.04) VM. In case you have a problem, you can open an issue, preferably with a reproducer.

[omeganinjaduck]

I was able to export the Tensorrt engine profile, it did run sucessfully but the speed is the same with non-tensortrt. Test on a AWS g5.xlarge A10 GPU with Ubuntu 22.04

[GuohongLi]

I meet the same problem with you. On my local ubuntu22.04+RTX4090, this extension get 4× acceleration. But on aws g5.xlarge A10 GPU with Ubuntu 22.04, only 25% improvement. Very strange, did you have a solution later on?

[GuohongLi]

@omeganinjaduck @contentis

[birep]

Working well for me for SD1.5 on ubuntu with nvidia driver 535 on a 3060 with 12gb vram.

For SDXL I'm getting out of memory errors while generating the engine (on both dev and master branch)

[aifartist]

I have it working on Ubuntu/4090 and it is extremely fast and I've since reduce the 512x512 gen time to .313 seconds. Trying with SDXL and it built the engine but getting "all tensors to be on the same device" errors. Perhaps my memory reduction changes are at fault. I want more strategies to be tried without the ol' 1GB workspace allocation error from occurring.

#59

[birep]

Are you on the dev branch of Automatic1111 when trying to use the SDXL engine? Last I heard the engine has to be generated on master but only works for inference on dev

[aifartist]

I'm on master. The build works but then:
File "/home/dwood/a1-231011/venv/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 122, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

[birep]

You could clone webui dev branch in another location and copy your generated engines there, that might fix your issue