Looking for some assistance /ideation. I am running qwen flash next q4 in my Mac mini m5 64gb. QFN doesn’t fit so this is done by having as many experts hot in cache as possible and streaming in the rest from ssd. I’m getting 17.5 tks decode and 390 tks pp. Have done a bunch of optimisations includi
