Just a quick note that I got a nice result using am17an's MTP branch of llama.cpp on v100 32GB SXM module using one of those pcie card adapters. Pulled and built in one shot, and llama-server ran without a hitch. Tested using am17an's MTP GGUF, q8_0 kv cache and 200k cache limit acting as vscode cop
