Hello r/LocalLLaMA, I put up an experimental PR which prefetches weights when offloading to CPU. Long story short from results it helps dense + smaller MoE models for PP (prompt processing). Give it a try if you are ram-rich and gpu-poor like me. https://github.com/ggml-org/llama.cpp/pull/21067
