I managed to get this model working on 2x 3090s with full 262k ctx and N=4, if anyone is interested to try it, thanks to this quant: https://huggingface.co/danielrmay/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-W4A16 Nemotron-Labs-3-Puzzle-75B-A9B (W4A16) on 2× RTX 3090 — vLLM, no CPU offload, full 262K,
