Single Request Results -------------------------------------------------------------------------------- Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem pp1024/tg128 3101.8 36.99 330.1 tok/s 27.2 tok/s 7.800 147.7 tok/s 226.63 GB pp4096/tg128 11506.3 51.50 356.0 tok/s 19.6 tok/s 18.04
