My employer has a GPU node that is mostly sitting idle. It contains 8 NVIDIA Quadro RTX 6000 GPUs with a total of 192 GB VRAM, and 512 GB RAM, and approximately 112 CPU threads to play with. I want to suggest we repurpose it for local inference. I need to make a case for this with my boss. What ki
