I have been trying Mistral 3.5 on my 4x RTX 3090 rig with llama.cpp. Inference is slow (about 11 t/s) even without anything being offloaded to the CPU. Here is the llama-server command I used: ./llama-server --model ../downloaded_models/Mistral-Medium-3.5-128B-UD-Q4_K_XL-00001-of-00003.gguf --port 1
