Repository navigation
4070 Ti Super + 32GB RAM - Test results on qwen3-8-flash-next-coder-iq1-m / qwen3-6-35b-a3b-ud-q3-k-m / qwen3-6-35b-a3b-ud-q4-k-m #47
Haroldyolo
started this conversation in
Show and tell
Replies: 1 comment
|
Thanks, these are complete reports and easy to compare. On a 16 GB card Qwen3.6 on strata-poly writes at about 165 tok/s with both sizes, and UD-Q3_K_M reads a 16K prompt at 3,615 tok/s against 3,078 for UD-Q4_K_M, which holds fewer experts on the card. Qwen3.8-Flash-Next Coder IQ1_M on the Strata project's engine is the much larger model, so its 59 tok/s is in line. I'll add the three to docs/COMMUNITY_BENCHMARKS.md with a link to this post. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Here are my results running the benchmarks on these tree models.
qwen3-8-flash-next-coder-iq1-m
`# Community benchmark on NVIDIA GeForce RTX 4070 Ti SUPER
Measured on 2026-10-18 with
tools/strata_bench.py(suite 1, 3 runs per test, medians with ranges). Speeds are the engine's own timings; first-token times are measured by the client over HTTP. Speed only: no correctness check was run.Hardware and software
Model and configuration
Method
Run by
python tools/strata_bench.pyagainst the HTTP API of a PolyStrata server (started by the benchmark from the install configuration); model loading is not timed. One warm-up chat and a 1K prompt are discarded first. Every prompt starts with a unique tag, so the conversation cache cannot help the fresh-prompt tests.tools/ab_engine.py, up to 100 new tokens, 3 runscached_tokens) and timings (prompt_n)Results
Ranges are the lowest and highest of the runs;
Runscounts requests that returned a token.Agent session (conversation cache)
Cache hit over the follow-up turns: 69.2 % (69.1-69.4); earlier-turn tokens read again per session: 74 (2-75) (earlier turns that the cache did not hold; the 14335 (14334-14335) new tokens of the files are read once); first turn (nothing cached) first token: 1.16 s (1.15-1.18).
Per-request data:
results.json. Engine log lines from the start:engine-log-excerpt.txt(when this run started the server).Correctness and limitations
Not measured: this is a speed suite. The conversation uses the model's own replies, so token counts differ a little between runs. Machine details come from the server,
nvidia-smi/rocm-smiand the OS; background load and power limits are not recorded. Total time of the suite: 102 s.`
qwen3-6-35b-a3b-ud-q3-k-m
`# Community benchmark on NVIDIA GeForce RTX 4070 Ti SUPER
Measured on 2026-10-18 with
tools/strata_bench.py(suite 1, 3 runs per test, medians with ranges). Speeds are the engine's own timings; first-token times are measured by the client over HTTP. Speed only: no correctness check was run.Hardware and software
Model and configuration
Method
Run by
python tools/strata_bench.pyagainst the HTTP API of a PolyStrata server (started by the benchmark from the install configuration); model loading is not timed. One warm-up chat and a 1K prompt are discarded first. Every prompt starts with a unique tag, so the conversation cache cannot help the fresh-prompt tests.tools/ab_engine.py, up to 100 new tokens, 3 runscached_tokens) and timings (prompt_n)Results
Ranges are the lowest and highest of the runs;
Runscounts requests that returned a token.Agent session (conversation cache)
Cache hit over the follow-up turns: 68.8 % (68.7-68.8); earlier-turn tokens read again per session: 241 (238-251) (earlier turns that the cache did not hold; the 14314 (14314-14314) new tokens of the files are read once); first turn (nothing cached) first token: 0.65 s (0.65-0.67).
Per-request data:
results.json. Engine log lines from the start:engine-log-excerpt.txt(when this run started the server).Correctness and limitations
Not measured: this is a speed suite. The conversation uses the model's own replies, so token counts differ a little between runs. Machine details come from the server,
nvidia-smi/rocm-smiand the OS; background load and power limits are not recorded. Total time of the suite: 57 s.`
qwen3-6-35b-a3b-ud-q4-k-m
`# Community benchmark on NVIDIA GeForce RTX 4070 Ti SUPER
Measured on 2026-10-18 with
tools/strata_bench.py(suite 1, 3 runs per test, medians with ranges). Speeds are the engine's own timings; first-token times are measured by the client over HTTP. Speed only: no correctness check was run.Hardware and software
Model and configuration
Method
Run by
python tools/strata_bench.pyagainst the HTTP API of a PolyStrata server (started by the benchmark from the install configuration); model loading is not timed. One warm-up chat and a 1K prompt are discarded first. Every prompt starts with a unique tag, so the conversation cache cannot help the fresh-prompt tests.tools/ab_engine.py, up to 100 new tokens, 3 runscached_tokens) and timings (prompt_n)Results
Ranges are the lowest and highest of the runs;
Runscounts requests that returned a token.Agent session (conversation cache)
Cache hit over the follow-up turns: 68.7 % (68.7-68.8); earlier-turn tokens read again per session: 275 (259-292) (earlier turns that the cache did not hold; the 14314 (14314-14314) new tokens of the files are read once); first turn (nothing cached) first token: 0.93 s (0.88-0.94).
Per-request data:
results.json. Engine log lines from the start:engine-log-excerpt.txt(when this run started the server).Correctness and limitations
Not measured: this is a speed suite. The conversation uses the model's own replies, so token counts differ a little between runs. Machine details come from the server,
nvidia-smi/rocm-smiand the OS; background load and power limits are not recorded. Total time of the suite: 83 s.`
@VecSzn Please let me know your thoughts and if there is something I can improve running these three. I am not sure which one fits me the best? It feels the flash coder is the "smartest" but I have no data to back this up
Thank you
All reactions