MLX Model Test Report

Generated 2026-05-10 11:49 · AFM MLX Backend · v0.9.13

mlx-model-test.sh --model mlx-community/Qwen3.5-35B-A3B-4bit --prompts Scripts/test-llm-comprehensive.txt

Test Runs
190
Passed
190
Failed
0
Best tok/s
119.6
Fastest
mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase

Performance Ranking (by tokens/sec)

Click a row to jump to its full response below.

# Test Model / Config Status Temp Load (s) Tokens Gen (s) Tokens/sec Prompt
1 102 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase stop-long-phrase OK 0.0 8.0 1143 9.56
119.6
Write a 3-paragraph essay about renewable energy. Start the ...
2 78 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-double-newline stop-double-newline OK 0.0 10.0 742 7.45
99.6
Write a short paragraph about the ocean. Then write a second...
3 161 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi tool-call-multi OK 0.0 9.0 1217 12.53
97.2
What is the weather in London and what time is it in Tokyo?
4 136 mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal think-normal OK 0.0 9.0 833 8.62
96.7
Solve step by step: If a train travels at 60 mph for 2.5 hou...
5 130 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json response-format-json OK 0.0 8.0 1305 13.5
96.6
Return a JSON object with keys "language", "year_created", a...
6 108 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate stop-immediate OK 0.0 7.0 666 6.89
96.6
Explain what gravity is.
7 170 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-nullable-param tool-call-nullable-param OK 0.0 8.0 587 6.09
96.4
What is the weather in Chicago?
8 158 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml tool-call-xml
--tool-call-parser xmlFunction
OK 0.0 9.0 762 7.93
96.1
What is the weather in Paris in celsius?
9 138 mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw think-raw
--raw
OK 0.0 9.0 833 8.68
96.0
Solve step by step: If a train travels at 60 mph for 2.5 hou...
10 180 mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual multilingual OK 0.3 11.0 1008 10.51
95.9
Translate "Hello, how are you?" into French, Spanish, Japane...
11 145 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens max-completion-tokens OK 0.0 9.0 2355 24.57
95.8
Explain the causes and consequences of the French Revolution...
12 172 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none tool-call-none OK 0.0 8.0 470 4.91
95.8
What is the current temperature in Berlin?
13 155 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto tool-call-auto OK 0.0 10.0 449 4.69
95.7
What is the weather in Tokyo?
14 82 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-period stop-period OK 0.0 9.0 741 7.75
95.6
Tell me about the sun in three sentences.
15 176 mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt long-prompt OK 0.0 25.0 831 8.72
95.3
Repeat after me exactly: The quick brown fox jumps over the ...
16 26 mlx-community/Qwen3.5-35B-A3B-4bit @ pirate pirate OK 0.7 4.0 1609 16.91
95.2
Tell me about quantum computing.
17 30 mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 eli5 OK 0.7 4.0 1227 12.9
95.1
Tell me about quantum computing.
18 150 mlx-community/Qwen3.5-35B-A3B-4bit @ verbose verbose
--verbose
OK 0.7 9.0 394 4.14
95.1
Hello, how are you?
19 132 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema response-format-schema OK 0.0 20.0 1582 16.64
95.0
Generate a fictional person profile.
20 134 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text response-format-text OK 0.0 25.0 370 3.89
95.0
Tell me the language Python, what year it was created, and w...
21 100 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key stop-json-object-key OK 0.0 8.0 4096 43.2
94.8
Generate a JSON object with keys "name", "age", and "city" f...
22 147 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role developer-role OK 0.0 9.0 4096 43.22
94.8
developer: You are a helpful coding assistant. Only respond ...
23 164 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex tool-call-complex OK 0.0 9.0 458 4.84
94.7
Search recursively for all Swift files under Sources/ with a...
24 148 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role developer-role OK 0.0 9.0 390 4.12
94.7
Write a Python function that reverses a string.
25 142 mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded non-streaming-seeded
--no-streaming --seed 123
OK 0.7 8.0 4096 43.31
94.6
Write a 4-line poem about the ocean.
26 2 mlx-community/Qwen3.5-35B-A3B-4bit @ greedy greedy OK 0.0 9.0 546 5.79
94.2
Explain why the sky is blue in exactly 3 bullet points.
27 32 mlx-community/Qwen3.5-35B-A3B-4bit @ json-output json-output OK 0.0 6.0 384 4.08
94.2
Respond with a valid JSON object containing keys "name", "ag...
28 106 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match stop-no-match OK 0.0 7.0 256 2.72
94.2
Explain the difference between TCP and UDP in a few sentence...
29 28 mlx-community/Qwen3.5-35B-A3B-4bit @ scientist scientist OK 0.3 4.0 2130 22.64
94.1
Tell me about quantum computing.
30 144 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens max-completion-tokens OK 0.0 9.0 938 9.97
94.1
max_completion_tokens: 100
31 116 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max stop-four-max OK 0.0 23.0 4096 43.55
94.0
List 10 facts about numbers, alternating between digit forma...
32 152 mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose very-verbose
--very-verbose
OK 0.7 9.0 424 4.51
94.0
Hello, how are you?
33 167 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-array-param tool-call-array-param OK 0.0 9.0 279 2.97
93.9
Create three todo items: Buy milk, Call dentist, Fix bug #42...
34 4 mlx-community/Qwen3.5-35B-A3B-4bit @ default default OK 0.7 4.0 559 5.96
93.7
Explain why the sky is blue in exactly 3 bullet points.
35 6 mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp high-temp OK 1.5 4.0 1148 12.28
93.5
Explain why the sky is blue in exactly 3 bullet points.
36 140 mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded streaming-seeded
--seed 123
OK 0.7 9.0 4096 43.86
93.4
Write a 4-line poem about the ocean.
37 60 mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized kv-quantized
--kv-bits 4
OK 0.7 24.0 874 9.36
93.3
Summarize the key ideas of machine learning in a few paragra...
38 120 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered stop-system-numbered OK 0.0 9.0 874 9.37
93.3
What are the main benefits of exercise?
39 12 mlx-community/Qwen3.5-35B-A3B-4bit @ min-p min-p
--min-p 0.05
OK 0.8 4.0 786 8.44
93.1
Explain why the sky is blue in exactly 3 bullet points.
40 178 mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars special-chars OK 0.0 26.0 3774 40.57
93.0
Repeat these characters exactly: <tag> "quotes" 'apostrophes...
41 20 mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty no-penalty
--presence-penalty 0.0
OK 0.8 5.0 4096 44.05
93.0
Write a long essay about the history of bread making across ...
42 68 mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming no-streaming
--no-streaming
OK 0.7 10.0 3516 37.89
92.8
Write a short poem about the moon.
43 182 mlx-community/Qwen3.5-35B-A3B-4bit @ code-python code-python OK 0.0 25.0 267 2.88
92.7
Write a function that finds all prime numbers up to n using ...
44 64 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 prefill-large-4096
--prefill-step-size 4096
OK 0.0 8.0 3832 41.61
92.1
Given this architecture, what are the top 3 performance bott...
45 58 mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv small-kv
--max-kv-size 2048
OK 0.7 12.0 1220 13.26
92.0
Summarize the key ideas of machine learning in a few paragra...
46 18 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 seed-42-run2
--seed 42
OK 0.7 4.0 4096 44.53
92.0
Write a limerick about a cat.
47 16 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 seed-42-run1
--seed 42
OK 0.7 3.0 4096 44.69
91.7
Write a limerick about a cat.
48 66 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 prefill-small-256
--prefill-step-size 256
OK 0.0 9.0 3832 41.85
91.6
Given this architecture, what are the top 3 performance bott...
49 62 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default prefill-default OK 0.0 24.0 3832 41.93
91.4
Given this architecture, what are the top 3 performance bott...
50 154 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto tool-call-auto OK 0.0 10.0 330 3.62
91.2
tools: [{"type":"function","function":{"name":"get_weather",...
51 10 mlx-community/Qwen3.5-35B-A3B-4bit @ top-k top-k
--top-k 30
OK 0.8 4.0 598 6.57
91.0
Explain why the sky is blue in exactly 3 bullet points.
52 8 mlx-community/Qwen3.5-35B-A3B-4bit @ top-p top-p
--top-p 0.9
OK 0.8 5.0 621 6.84
90.8
Explain why the sky is blue in exactly 3 bullet points.
53 54 mlx-community/Qwen3.5-35B-A3B-4bit @ long-output long-output OK 0.7 26.0 2000 22.04
90.8
Write a detailed recipe for chocolate chip cookies with ingr...
54 25 mlx-community/Qwen3.5-35B-A3B-4bit @ pirate pirate OK 0.7 4.0 1803 19.89
90.7
Name three primary colors and explain why they matter in des...
55 166 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-array-param tool-call-array-param OK 0.0 9.0 270 2.98
90.5
tools: [{"type":"function","function":{"name":"todowrite","d...
56 27 mlx-community/Qwen3.5-35B-A3B-4bit @ scientist scientist OK 0.3 4.0 4096 45.26
90.5
Name three primary colors and explain why they matter in des...
57 186 mlx-community/Qwen3.5-35B-A3B-4bit @ math math OK 0.0 10.0 1934 21.41
90.3
A store sells notebooks for $3.50 each. If you buy 5 or more...
58 157 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml tool-call-xml
--tool-call-parser xmlFunction
OK 0.0 9.0 330 3.66
90.2
tools: [{"type":"function","function":{"name":"get_weather",...
59 163 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex tool-call-complex OK 0.0 9.0 338 3.75
90.2
tools: [{"type":"function","function":{"name":"search_files"...
60 174 mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt minimal-prompt OK 0.7 16.0 553 6.14
90.0
Hi
61 56 mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs logprobs
--max-logprobs 5
OK 0.0 10.0 145 1.61
89.9
What is 1+1?
62 117 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate stop-system-pirate OK 0.0 25.0 4096 45.64
89.8
Name three primary colors and explain why they matter in des...
63 115 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max stop-four-max OK 0.0 23.0 818 9.12
89.7
Name three primary colors and explain why they matter in des...
64 70 mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode raw-mode
--raw
OK 0.7 10.0 1270 14.2
89.5
Think step by step: what is 17 * 23?
65 160 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi tool-call-multi OK 0.0 9.0 287 3.24
88.5
tools: [{"type":"function","function":{"name":"get_weather",...
66 190 mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format strict-format OK 0.0 10.0 334 3.78
88.4
Output EXACTLY 3 lines. Each line must contain exactly one w...
67 50 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 agent-cached-turn3
--enable-prefix-caching
OK 0.0 16.0 323 3.66
88.1
Write a unit test for the timeout feature you just described...
68 14 mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers combined-samplers
--top-k 50 --min-p 0.03 --top-p 0.95
OK 0.8 4.0 624 7.12
87.7
Explain why the sky is blue in exactly 3 bullet points.
69 121 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp stop-high-temp OK 1.0 8.0 1891 21.73
87.0
Name three primary colors and explain why they matter in des...
70 188 mlx-community/Qwen3.5-35B-A3B-4bit @ long-form long-form OK 0.7 10.0 3936 45.24
87.0
Write a detailed technical blog post explaining how Mixture-...
71 19 mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty no-penalty
--presence-penalty 0.0
OK 0.8 5.0 1302 14.98
86.9
Name three primary colors and explain why they matter in des...
72 3 mlx-community/Qwen3.5-35B-A3B-4bit @ default default OK 0.7 4.0 1122 12.93
86.8
Name three primary colors and explain why they matter in des...
73 169 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-nullable-param tool-call-nullable-param OK 0.0 8.0 191 2.2
86.8
tools: [{"type":"function","function":{"name":"get_weather",...
74 86 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi stop-cli-multi
--stop "```,DONE"
OK 0.0 16.0 83 0.96
86.7
Write a bash script that prints the current date inside a co...
75 128 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens stop-low-max-tokens OK 0.0 8.0 100 1.15
86.6
List 10 mountains, numbered 1 through 10, one per line.
76 33 mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list numbered-list OK 0.0 3.0 810 9.36
86.6
Name three primary colors and explain why they matter in des...
77 15 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 seed-42-run1
--seed 42
OK 0.7 3.0 1135 13.13
86.4
Name three primary colors and explain why they matter in des...
78 29 mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 eli5 OK 0.7 4.0 1185 13.78
86.0
Name three primary colors and explain why they matter in des...
79 114 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode stop-unicode OK 0.0 20.0 341 3.98
85.7
List 5 items about space using bullet points (•).
80 31 mlx-community/Qwen3.5-35B-A3B-4bit @ json-output json-output OK 0.0 6.0 810 9.48
85.4
Name three primary colors and explain why they matter in des...
81 80 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-word stop-word OK 0.0 9.0 1560 18.3
85.3
Name 5 programming languages and briefly describe each one.
82 173 mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt minimal-prompt OK 0.7 16.0 1349 15.82
85.2
Name three primary colors and explain why they matter in des...
83 13 mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers combined-samplers
--top-k 50 --min-p 0.03 --top-p 0.95
OK 0.8 4.0 2047 24.03
85.2
Name three primary colors and explain why they matter in des...
84 5 mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp high-temp OK 1.5 4.0 910 10.68
85.2
Name three primary colors and explain why they matter in des...
85 7 mlx-community/Qwen3.5-35B-A3B-4bit @ top-p top-p
--top-p 0.9
OK 0.8 5.0 1680 19.73
85.1
Name three primary colors and explain why they matter in des...
86 17 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 seed-42-run2
--seed 42
OK 0.7 4.0 1135 13.33
85.1
Name three primary colors and explain why they matter in des...
87 110 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars stop-special-chars OK 0.0 7.0 558 6.56
85.1
List 3 facts about the moon. Use **bold** markdown for empha...
88 139 mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded streaming-seeded
--seed 123
OK 0.7 9.0 1444 17.02
84.8
Name three primary colors and explain why they matter in des...
89 107 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate stop-immediate OK 0.0 7.0 817 9.67
84.5
Name three primary colors and explain why they matter in des...
90 109 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars stop-special-chars OK 0.0 7.0 824 9.79
84.2
Name three primary colors and explain why they matter in des...
91 141 mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded non-streaming-seeded
--no-streaming --seed 123
OK 0.7 8.0 1444 17.18
84.0
Name three primary colors and explain why they matter in des...
92 61 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default prefill-default OK 0.0 24.0 1584 18.9
83.8
Name three primary colors and explain why they matter in des...
93 48 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 agent-cached-turn2
--enable-prefix-caching
OK 0.0 6.0 126 1.51
83.6
Now add a --timeout flag to the CLI that sets a request time...
94 119 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered stop-system-numbered OK 0.0 9.0 1289 15.44
83.5
Name three primary colors and explain why they matter in des...
95 125 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 stop-seed-run2 OK 0.7 9.0 1135 13.65
83.1
Name three primary colors and explain why they matter in des...
96 63 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 prefill-large-4096
--prefill-step-size 4096
OK 0.0 8.0 1584 19.06
83.1
Name three primary colors and explain why they matter in des...
97 149 mlx-community/Qwen3.5-35B-A3B-4bit @ verbose verbose
--verbose
OK 0.7 9.0 1386 16.68
83.1
Name three primary colors and explain why they matter in des...
98 11 mlx-community/Qwen3.5-35B-A3B-4bit @ min-p min-p
--min-p 0.05
OK 0.8 4.0 1014 12.29
82.5
Name three primary colors and explain why they matter in des...
99 9 mlx-community/Qwen3.5-35B-A3B-4bit @ top-k top-k
--top-k 30
OK 0.8 4.0 859 10.43
82.4
Name three primary colors and explain why they matter in des...
100 76 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline stop-newline OK 0.0 24.0 173 2.12
81.5
What is the capital of France? Answer in one sentence.
101 118 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate stop-system-pirate OK 0.0 25.0 685 8.41
81.4
Tell me about treasure hunting on the high seas.
102 1 mlx-community/Qwen3.5-35B-A3B-4bit @ greedy greedy OK 0.0 9.0 810 9.98
81.2
Name three primary colors and explain why they matter in des...
103 175 mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt long-prompt OK 0.0 25.0 810 9.99
81.1
Name three primary colors and explain why they matter in des...
104 75 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline stop-newline OK 0.0 24.0 817 10.09
81.0
Name three primary colors and explain why they matter in des...
105 177 mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars special-chars OK 0.0 26.0 810 10.03
80.7
Name three primary colors and explain why they matter in des...
106 57 mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv small-kv
--max-kv-size 2048
OK 0.7 12.0 1590 19.73
80.6
Name three primary colors and explain why they matter in des...
107 131 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema response-format-schema OK 0.0 20.0 810 10.09
80.3
Name three primary colors and explain why they matter in des...
108 74 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi stop-multi OK 0.0 30.0 253 3.15
80.2
Write a Python hello world program in a code block, then wri...
109 89 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup stop-cli-api-dedup
--stop "3."
OK 0.0 24.0 810 10.12
80.1
Name three primary colors and explain why they matter in des...
110 123 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 stop-seed-run1 OK 0.7 8.0 1135 14.21
79.9
Name three primary colors and explain why they matter in des...
111 133 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text response-format-text OK 0.0 25.0 810 10.15
79.8
Name three primary colors and explain why they matter in des...
112 101 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase stop-long-phrase OK 0.0 8.0 810 10.16
79.7
Name three primary colors and explain why they matter in des...
113 87 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge stop-cli-api-merge
--stop "5."
OK 0.0 28.0 810 10.18
79.6
Name three primary colors and explain why they matter in des...
114 77 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-double-newline stop-double-newline OK 0.0 10.0 817 10.27
79.5
Name three primary colors and explain why they matter in des...
115 81 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-period stop-period OK 0.0 9.0 831 10.53
78.9
Name three primary colors and explain why they matter in des...
116 113 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode stop-unicode OK 0.0 20.0 810 10.27
78.8
Name three primary colors and explain why they matter in des...
117 91 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming stop-non-streaming
--no-streaming
OK 0.0 8.0 810 10.28
78.8
Name three primary colors and explain why they matter in des...
118 171 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none tool-call-none OK 0.0 8.0 810 10.29
78.7
Name three primary colors and explain why they matter in des...
119 135 mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal think-normal OK 0.0 9.0 810 10.29
78.7
Name three primary colors and explain why they matter in des...
120 99 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key stop-json-object-key OK 0.0 8.0 851 10.82
78.7
Name three primary colors and explain why they matter in des...
121 65 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 prefill-small-256
--prefill-step-size 256
OK 0.0 9.0 1584 20.19
78.5
Name three primary colors and explain why they matter in des...
122 129 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json response-format-json OK 0.0 8.0 810 10.39
78.0
Name three primary colors and explain why they matter in des...
123 103 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word stop-multi-word OK 0.0 10.0 810 10.39
78.0
Name three primary colors and explain why they matter in des...
124 159 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi tool-call-multi OK 0.0 9.0 810 10.4
77.9
Name three primary colors and explain why they matter in des...
125 137 mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw think-raw
--raw
OK 0.0 9.0 810 10.43
77.7
Name three primary colors and explain why they matter in des...
126 168 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-nullable-param tool-call-nullable-param OK 0.0 8.0 810 10.43
77.7
Name three primary colors and explain why they matter in des...
127 165 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-array-param tool-call-array-param OK 0.0 9.0 810 10.45
77.5
Name three primary colors and explain why they matter in des...
128 34 mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list numbered-list OK 0.0 3.0 489 6.32
77.3
List exactly 5 animals, one per line, numbered 1-5. No other...
129 104 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word stop-multi-word OK 0.0 10.0 374 4.84
77.3
Write a 5-step recipe for making tea. Label each step as "St...
130 143 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens max-completion-tokens OK 0.0 9.0 810 10.5
77.2
Name three primary colors and explain why they matter in des...
131 67 mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming no-streaming
--no-streaming
OK 0.7 10.0 963 12.48
77.2
Name three primary colors and explain why they matter in des...
132 111 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag stop-html-tag OK 0.0 8.0 810 10.51
77.1
Name three primary colors and explain why they matter in des...
133 146 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role developer-role OK 0.0 9.0 810 10.61
76.4
Name three primary colors and explain why they matter in des...
134 44 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 agent-no-cache-turn3 OK 0.0 5.0 149 1.95
76.3
Write a unit test for the timeout feature you just described...
135 179 mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual multilingual OK 0.3 11.0 785 10.36
75.8
Name three primary colors and explain why they matter in des...
136 156 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml tool-call-xml
--tool-call-parser xmlFunction
OK 0.0 9.0 810 10.7
75.7
Name three primary colors and explain why they matter in des...
137 73 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi stop-multi OK 0.0 30.0 810 10.71
75.6
Name three primary colors and explain why they matter in des...
138 153 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto tool-call-auto OK 0.0 10.0 810 10.73
75.5
Name three primary colors and explain why they matter in des...
139 59 mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized kv-quantized
--kv-bits 4
OK 0.7 24.0 772 10.29
75.0
Name three primary colors and explain why they matter in des...
140 162 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex tool-call-complex OK 0.0 9.0 810 10.81
74.9
Name three primary colors and explain why they matter in des...
141 79 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-word stop-word OK 0.0 9.0 810 10.84
74.7
Name three primary colors and explain why they matter in des...
142 22 mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty with-penalty
--presence-penalty 1.5
OK 0.8 4.0 4095 55.17
74.2
Write a long essay about the history of bread making across ...
143 85 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi stop-cli-multi
--stop "```,DONE"
OK 0.0 16.0 810 10.92
74.2
Name three primary colors and explain why they matter in des...
144 83 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only stop-cli-only
--stop "3."
OK 0.0 9.0 810 10.97
73.8
Name three primary colors and explain why they matter in des...
145 52 mlx-community/Qwen3.5-35B-A3B-4bit @ short-output short-output OK 0.7 27.0 50 0.68
73.8
Describe the entire history of Rome.
146 187 mlx-community/Qwen3.5-35B-A3B-4bit @ long-form long-form OK 0.7 10.0 1060 14.44
73.4
Name three primary colors and explain why they matter in des...
147 24 mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty repetition-penalty
--repetition-penalty 1.2
OK 0.8 7.0 4096 55.86
73.3
Write a long essay about the history of bread making across ...
148 90 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup stop-cli-api-dedup
--stop "3."
OK 0.0 24.0 234 3.25
72.1
List 10 cities, numbered 1 through 10, one per line.
149 55 mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs logprobs
--max-logprobs 5
OK 0.0 10.0 810 11.26
71.9
Name three primary colors and explain why they matter in des...
150 23 mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty repetition-penalty
--repetition-penalty 1.2
OK 0.8 7.0 1127 15.7
71.8
Name three primary colors and explain why they matter in des...
151 92 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming stop-non-streaming
--no-streaming
OK 0.0 8.0 1411 19.69
71.7
List 10 planets or celestial objects, numbered 1 through 10,...
152 21 mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty with-penalty
--presence-penalty 1.5
OK 0.8 4.0 1591 22.3
71.4
Name three primary colors and explain why they matter in des...
153 185 mlx-community/Qwen3.5-35B-A3B-4bit @ math math OK 0.0 10.0 810 11.5
70.4
Name three primary colors and explain why they matter in des...
154 69 mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode raw-mode
--raw
OK 0.7 10.0 753 10.74
70.1
Name three primary colors and explain why they matter in des...
155 124 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 stop-seed-run1 OK 0.7 8.0 329 4.69
70.1
List 10 flowers, numbered 1 through 10, one per line.
156 126 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 stop-seed-run2 OK 0.7 9.0 329 4.71
69.9
List 10 flowers, numbered 1 through 10, one per line.
157 71 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single stop-single OK 0.0 23.0 810 11.6
69.8
Name three primary colors and explain why they matter in des...
158 112 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag stop-html-tag OK 0.0 8.0 239 3.43
69.6
Write an HTML unordered list of 5 fruits using <ul> and <li>...
159 88 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge stop-cli-api-merge
--stop "5."
OK 0.0 28.0 245 3.52
69.6
List 10 countries, numbered 1 through 10, one per line.
160 189 mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format strict-format OK 0.0 10.0 810 11.67
69.4
Name three primary colors and explain why they matter in des...
161 42 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 agent-no-cache-turn2 OK 0.0 6.0 103 1.5
68.9
Now add a --timeout flag to the CLI that sets a request time...
162 122 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp stop-high-temp OK 1.0 8.0 262 3.83
68.4
List 10 random words, numbered 1 through 10, one per line.
163 53 mlx-community/Qwen3.5-35B-A3B-4bit @ long-output long-output OK 0.7 26.0 688 10.1
68.1
Name three primary colors and explain why they matter in des...
164 151 mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose very-verbose
--very-verbose
OK 0.7 9.0 516 7.92
65.1
Name three primary colors and explain why they matter in des...
165 84 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only stop-cli-only
--stop "3."
OK 0.0 9.0 209 3.29
63.6
List 10 types of cheese, numbered 1 through 10, one per line...
166 72 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single stop-single OK 0.0 23.0 199 3.14
63.4
List 10 fruits, numbered 1-10, one per line.
167 183 mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift code-swift OK 0.0 10.0 32768 542.45
60.4
Name three primary colors and explain why they matter in des...
168 40 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 agent-no-cache-turn1 OK 0.0 8.0 67 1.12
59.7
Read the file Sources/MacLocalAPI/main.swift and explain wha...
169 184 mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift code-swift OK 0.0 10.0 32768 555.02
59.0
Write a Swift async function that fetches JSON from a URL us...
170 46 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 agent-cached-turn1
--enable-prefix-caching
OK 0.0 6.0 59 1.0
58.9
Read the file Sources/MacLocalAPI/main.swift and explain wha...
171 41 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 agent-no-cache-turn2 OK 0.0 6.0 256 4.47
57.3
Name three primary colors and explain why they matter in des...
172 43 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 agent-no-cache-turn3 OK 0.0 5.0 256 4.5
56.9
Name three primary colors and explain why they matter in des...
173 181 mlx-community/Qwen3.5-35B-A3B-4bit @ code-python code-python OK 0.0 25.0 32768 580.74
56.4
Name three primary colors and explain why they matter in des...
174 105 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match stop-no-match OK 0.0 7.0 256 4.56
56.2
Name three primary colors and explain why they matter in des...
175 38 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested guided-json-nested
--guided-json '{"type":"object","properties":{"city":{"type":"string"},"population":{"type":"integer"},"landmarks":{"type":"array","items":{"type":"string"}}},"required":["city","population","landmarks"]}'
OK 0.0 9.0 34 0.66
51.6
Describe Tokyo as a structured record with at least 3 landma...
176 39 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 agent-no-cache-turn1 OK 0.0 8.0 256 4.98
51.5
Name three primary colors and explain why they matter in des...
177 47 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 agent-cached-turn2
--enable-prefix-caching
OK 0.0 6.0 256 5.21
49.1
Name three primary colors and explain why they matter in des...
178 49 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 agent-cached-turn3
--enable-prefix-caching
OK 0.0 16.0 256 5.27
48.5
Name three primary colors and explain why they matter in des...
179 45 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 agent-cached-turn1
--enable-prefix-caching
OK 0.0 6.0 256 5.3
48.3
Name three primary colors and explain why they matter in des...
180 127 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens stop-low-max-tokens OK 0.0 8.0 100 3.05
32.8
Name three primary colors and explain why they matter in des...
181 36 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple guided-json-simple
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]}'
OK 0.0 9.0 10 0.34
29.7
Generate a person record.
182 51 mlx-community/Qwen3.5-35B-A3B-4bit @ short-output short-output OK 0.7 27.0 50 2.34
21.4
Name three primary colors and explain why they matter in des...
183 98 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace stop-guided-json-brace
--guided-json '{"type":"object","properties":{"color":{"type":"string"},"hex":{"type":"string"}},"required":["color","hex"]}'
OK 0.0 8.0 7 0.38
18.6
Describe the color blue with its hex code.
184 94 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value stop-guided-json-value
--guided-json '{"type":"object","properties":{"cities":{"type":"array","items":{"type":"string"}}},"required":["cities"]}'
OK 0.0 9.0 3 0.29
10.4
List 5 major world cities as a JSON array. Include Tokyo.
185 37 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested guided-json-nested
--guided-json '{"type":"object","properties":{"city":{"type":"string"},"population":{"type":"integer"},"landmarks":{"type":"array","items":{"type":"string"}}},"required":["city","population","landmarks"]}'
OK 0.0 9.0 26 2.71
9.6
Name three primary colors and explain why they matter in des...
186 96 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma stop-guided-json-comma
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"},"city":{"type":"string"}},"required":["name","age","city"]}'
OK 0.0 8.0 3 0.34
8.8
Generate a person profile for someone named Alice who is 30 ...
187 97 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace stop-guided-json-brace
--guided-json '{"type":"object","properties":{"color":{"type":"string"},"hex":{"type":"string"}},"required":["color","hex"]}'
OK 0.0 8.0 15 2.7
5.6
Name three primary colors and explain why they matter in des...
188 35 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple guided-json-simple
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]}'
OK 0.0 9.0 13 2.68
4.8
Name three primary colors and explain why they matter in des...
189 93 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value stop-guided-json-value
--guided-json '{"type":"object","properties":{"cities":{"type":"array","items":{"type":"string"}}},"required":["cities"]}'
OK 0.0 9.0 12 2.61
4.6
Name three primary colors and explain why they matter in des...
190 95 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma stop-guided-json-comma
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"},"city":{"type":"string"}},"required":["name","age","city"]}'
OK 0.0 8.0 3 2.35
1.3
Name three primary colors and explain why they matter in des...

Full Responses

#102 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase 1143 tokens · 119.6 tok/s

#78 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-double-newline 742 tokens · 99.6 tok/s

#161 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi 1217 tokens · 97.2 tok/s

#136 mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal 833 tokens · 96.7 tok/s

#130 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json 1305 tokens · 96.6 tok/s

#108 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate 666 tokens · 96.6 tok/s

#170 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-nullable-param 587 tokens · 96.4 tok/s

#158 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml 762 tokens · 96.1 tok/s

#138 mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw 833 tokens · 96.0 tok/s

#180 mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual 1008 tokens · 95.9 tok/s

#145 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens 2355 tokens · 95.8 tok/s

#172 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none 470 tokens · 95.8 tok/s

#155 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto 449 tokens · 95.7 tok/s

#82 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-period 741 tokens · 95.6 tok/s

#176 mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt 831 tokens · 95.3 tok/s

#26 mlx-community/Qwen3.5-35B-A3B-4bit @ pirate 1609 tokens · 95.2 tok/s

#30 mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 1227 tokens · 95.1 tok/s

#150 mlx-community/Qwen3.5-35B-A3B-4bit @ verbose 394 tokens · 95.1 tok/s

#132 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema 1582 tokens · 95.0 tok/s

#134 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text 370 tokens · 95.0 tok/s

#100 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key 4096 tokens · 94.8 tok/s

#147 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role 4096 tokens · 94.8 tok/s

#164 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex 458 tokens · 94.7 tok/s

#148 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role 390 tokens · 94.7 tok/s

#142 mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded 4096 tokens · 94.6 tok/s

#2 mlx-community/Qwen3.5-35B-A3B-4bit @ greedy 546 tokens · 94.2 tok/s

#32 mlx-community/Qwen3.5-35B-A3B-4bit @ json-output 384 tokens · 94.2 tok/s

#106 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match 256 tokens · 94.2 tok/s

#28 mlx-community/Qwen3.5-35B-A3B-4bit @ scientist 2130 tokens · 94.1 tok/s

#144 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens 938 tokens · 94.1 tok/s

#116 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max 4096 tokens · 94.0 tok/s

#152 mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose 424 tokens · 94.0 tok/s

#167 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-array-param 279 tokens · 93.9 tok/s

#4 mlx-community/Qwen3.5-35B-A3B-4bit @ default 559 tokens · 93.7 tok/s

#6 mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp 1148 tokens · 93.5 tok/s

#140 mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded 4096 tokens · 93.4 tok/s

#60 mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized 874 tokens · 93.3 tok/s

#120 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered 874 tokens · 93.3 tok/s

#12 mlx-community/Qwen3.5-35B-A3B-4bit @ min-p 786 tokens · 93.1 tok/s

#178 mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars 3774 tokens · 93.0 tok/s

#20 mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty 4096 tokens · 93.0 tok/s

#68 mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming 3516 tokens · 92.8 tok/s

#182 mlx-community/Qwen3.5-35B-A3B-4bit @ code-python 267 tokens · 92.7 tok/s

#64 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 3832 tokens · 92.1 tok/s

#58 mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv 1220 tokens · 92.0 tok/s

#18 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 4096 tokens · 92.0 tok/s

#16 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 4096 tokens · 91.7 tok/s

#66 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 3832 tokens · 91.6 tok/s

#62 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default 3832 tokens · 91.4 tok/s

#154 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto 330 tokens · 91.2 tok/s

#10 mlx-community/Qwen3.5-35B-A3B-4bit @ top-k 598 tokens · 91.0 tok/s

#8 mlx-community/Qwen3.5-35B-A3B-4bit @ top-p 621 tokens · 90.8 tok/s

#54 mlx-community/Qwen3.5-35B-A3B-4bit @ long-output 2000 tokens · 90.8 tok/s

#25 mlx-community/Qwen3.5-35B-A3B-4bit @ pirate 1803 tokens · 90.7 tok/s

#166 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-array-param 270 tokens · 90.5 tok/s

#27 mlx-community/Qwen3.5-35B-A3B-4bit @ scientist 4096 tokens · 90.5 tok/s

#186 mlx-community/Qwen3.5-35B-A3B-4bit @ math 1934 tokens · 90.3 tok/s

#157 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml 330 tokens · 90.2 tok/s

#163 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex 338 tokens · 90.2 tok/s

#174 mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt 553 tokens · 90.0 tok/s

#56 mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs 145 tokens · 89.9 tok/s

#117 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate 4096 tokens · 89.8 tok/s

#115 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max 818 tokens · 89.7 tok/s

#70 mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode 1270 tokens · 89.5 tok/s

#160 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi 287 tokens · 88.5 tok/s

#190 mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format 334 tokens · 88.4 tok/s

#50 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 323 tokens · 88.1 tok/s

#14 mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers 624 tokens · 87.7 tok/s

#121 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp 1891 tokens · 87.0 tok/s

#188 mlx-community/Qwen3.5-35B-A3B-4bit @ long-form 3936 tokens · 87.0 tok/s

#19 mlx-community/Qwen3.5-35B-A3B-4bit @ no-penalty 1302 tokens · 86.9 tok/s

#3 mlx-community/Qwen3.5-35B-A3B-4bit @ default 1122 tokens · 86.8 tok/s

#169 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-nullable-param 191 tokens · 86.8 tok/s

#86 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi 83 tokens · 86.7 tok/s

#128 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens 100 tokens · 86.6 tok/s

#33 mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list 810 tokens · 86.6 tok/s

#15 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run1 1135 tokens · 86.4 tok/s

#29 mlx-community/Qwen3.5-35B-A3B-4bit @ eli5 1185 tokens · 86.0 tok/s

#114 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode 341 tokens · 85.7 tok/s

#31 mlx-community/Qwen3.5-35B-A3B-4bit @ json-output 810 tokens · 85.4 tok/s

#80 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-word 1560 tokens · 85.3 tok/s

#173 mlx-community/Qwen3.5-35B-A3B-4bit @ minimal-prompt 1349 tokens · 85.2 tok/s

#13 mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers 2047 tokens · 85.2 tok/s

#5 mlx-community/Qwen3.5-35B-A3B-4bit @ high-temp 910 tokens · 85.2 tok/s

#7 mlx-community/Qwen3.5-35B-A3B-4bit @ top-p 1680 tokens · 85.1 tok/s

#17 mlx-community/Qwen3.5-35B-A3B-4bit @ seed-42-run2 1135 tokens · 85.1 tok/s

#110 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars 558 tokens · 85.1 tok/s

#139 mlx-community/Qwen3.5-35B-A3B-4bit @ streaming-seeded 1444 tokens · 84.8 tok/s

#107 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate 817 tokens · 84.5 tok/s

#109 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars 824 tokens · 84.2 tok/s

#141 mlx-community/Qwen3.5-35B-A3B-4bit @ non-streaming-seeded 1444 tokens · 84.0 tok/s

#61 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-default 1584 tokens · 83.8 tok/s

#48 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 126 tokens · 83.6 tok/s

#119 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered 1289 tokens · 83.5 tok/s

#125 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 1135 tokens · 83.1 tok/s

#63 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-large-4096 1584 tokens · 83.1 tok/s

#149 mlx-community/Qwen3.5-35B-A3B-4bit @ verbose 1386 tokens · 83.1 tok/s

#11 mlx-community/Qwen3.5-35B-A3B-4bit @ min-p 1014 tokens · 82.5 tok/s

#9 mlx-community/Qwen3.5-35B-A3B-4bit @ top-k 859 tokens · 82.4 tok/s

#76 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline 173 tokens · 81.5 tok/s

#118 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate 685 tokens · 81.4 tok/s

#1 mlx-community/Qwen3.5-35B-A3B-4bit @ greedy 810 tokens · 81.2 tok/s

#175 mlx-community/Qwen3.5-35B-A3B-4bit @ long-prompt 810 tokens · 81.1 tok/s

#75 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-newline 817 tokens · 81.0 tok/s

#177 mlx-community/Qwen3.5-35B-A3B-4bit @ special-chars 810 tokens · 80.7 tok/s

#57 mlx-community/Qwen3.5-35B-A3B-4bit @ small-kv 1590 tokens · 80.6 tok/s

#131 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-schema 810 tokens · 80.3 tok/s

#74 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi 253 tokens · 80.2 tok/s

#89 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup 810 tokens · 80.1 tok/s

#123 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 1135 tokens · 79.9 tok/s

#133 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-text 810 tokens · 79.8 tok/s

#101 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase 810 tokens · 79.7 tok/s

#87 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge 810 tokens · 79.6 tok/s

#77 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-double-newline 817 tokens · 79.5 tok/s

#81 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-period 831 tokens · 78.9 tok/s

#113 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode 810 tokens · 78.8 tok/s

#91 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming 810 tokens · 78.8 tok/s

#171 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-none 810 tokens · 78.7 tok/s

#135 mlx-community/Qwen3.5-35B-A3B-4bit @ think-normal 810 tokens · 78.7 tok/s

#99 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key 851 tokens · 78.7 tok/s

#65 mlx-community/Qwen3.5-35B-A3B-4bit @ prefill-small-256 1584 tokens · 78.5 tok/s

#129 mlx-community/Qwen3.5-35B-A3B-4bit @ response-format-json 810 tokens · 78.0 tok/s

#103 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word 810 tokens · 78.0 tok/s

#159 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-multi 810 tokens · 77.9 tok/s

#137 mlx-community/Qwen3.5-35B-A3B-4bit @ think-raw 810 tokens · 77.7 tok/s

#168 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-nullable-param 810 tokens · 77.7 tok/s

#165 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-array-param 810 tokens · 77.5 tok/s

#34 mlx-community/Qwen3.5-35B-A3B-4bit @ numbered-list 489 tokens · 77.3 tok/s

#104 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word 374 tokens · 77.3 tok/s

#143 mlx-community/Qwen3.5-35B-A3B-4bit @ max-completion-tokens 810 tokens · 77.2 tok/s

#67 mlx-community/Qwen3.5-35B-A3B-4bit @ no-streaming 963 tokens · 77.2 tok/s

#111 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag 810 tokens · 77.1 tok/s

#146 mlx-community/Qwen3.5-35B-A3B-4bit @ developer-role 810 tokens · 76.4 tok/s

#44 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 149 tokens · 76.3 tok/s

#179 mlx-community/Qwen3.5-35B-A3B-4bit @ multilingual 785 tokens · 75.8 tok/s

#156 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-xml 810 tokens · 75.7 tok/s

#73 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi 810 tokens · 75.6 tok/s

#153 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-auto 810 tokens · 75.5 tok/s

#59 mlx-community/Qwen3.5-35B-A3B-4bit @ kv-quantized 772 tokens · 75.0 tok/s

#162 mlx-community/Qwen3.5-35B-A3B-4bit @ tool-call-complex 810 tokens · 74.9 tok/s

#79 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-word 810 tokens · 74.7 tok/s

#22 mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty 4095 tokens · 74.2 tok/s

#85 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi 810 tokens · 74.2 tok/s

#83 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only 810 tokens · 73.8 tok/s

#52 mlx-community/Qwen3.5-35B-A3B-4bit @ short-output 50 tokens · 73.8 tok/s

#187 mlx-community/Qwen3.5-35B-A3B-4bit @ long-form 1060 tokens · 73.4 tok/s

#24 mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty 4096 tokens · 73.3 tok/s

#90 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup 234 tokens · 72.1 tok/s

#55 mlx-community/Qwen3.5-35B-A3B-4bit @ logprobs 810 tokens · 71.9 tok/s

#23 mlx-community/Qwen3.5-35B-A3B-4bit @ repetition-penalty 1127 tokens · 71.8 tok/s

#92 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming 1411 tokens · 71.7 tok/s

#21 mlx-community/Qwen3.5-35B-A3B-4bit @ with-penalty 1591 tokens · 71.4 tok/s

#185 mlx-community/Qwen3.5-35B-A3B-4bit @ math 810 tokens · 70.4 tok/s

#69 mlx-community/Qwen3.5-35B-A3B-4bit @ raw-mode 753 tokens · 70.1 tok/s

#124 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 329 tokens · 70.1 tok/s

#126 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 329 tokens · 69.9 tok/s

#71 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single 810 tokens · 69.8 tok/s

#112 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag 239 tokens · 69.6 tok/s

#88 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge 245 tokens · 69.6 tok/s

#189 mlx-community/Qwen3.5-35B-A3B-4bit @ strict-format 810 tokens · 69.4 tok/s

#42 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 103 tokens · 68.9 tok/s

#122 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp 262 tokens · 68.4 tok/s

#53 mlx-community/Qwen3.5-35B-A3B-4bit @ long-output 688 tokens · 68.1 tok/s

#151 mlx-community/Qwen3.5-35B-A3B-4bit @ very-verbose 516 tokens · 65.1 tok/s

#84 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only 209 tokens · 63.6 tok/s

#72 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-single 199 tokens · 63.4 tok/s

#183 mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift 32768 tokens · 60.4 tok/s

#40 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 67 tokens · 59.7 tok/s

#184 mlx-community/Qwen3.5-35B-A3B-4bit @ code-swift 32768 tokens · 59.0 tok/s

#46 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 59 tokens · 58.9 tok/s

#41 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn2 256 tokens · 57.3 tok/s

#43 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn3 256 tokens · 56.9 tok/s

#181 mlx-community/Qwen3.5-35B-A3B-4bit @ code-python 32768 tokens · 56.4 tok/s

#105 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match 256 tokens · 56.2 tok/s

#38 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested 34 tokens · 51.6 tok/s

#39 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-no-cache-turn1 256 tokens · 51.5 tok/s

#47 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn2 256 tokens · 49.1 tok/s

#49 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn3 256 tokens · 48.5 tok/s

#45 mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 256 tokens · 48.3 tok/s

#127 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens 100 tokens · 32.8 tok/s

#36 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple 10 tokens · 29.7 tok/s

#51 mlx-community/Qwen3.5-35B-A3B-4bit @ short-output 50 tokens · 21.4 tok/s

#98 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace 7 tokens · 18.6 tok/s

#94 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value 3 tokens · 10.4 tok/s

#37 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-nested 26 tokens · 9.6 tok/s

#96 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma 3 tokens · 8.8 tok/s

#97 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace 15 tokens · 5.6 tok/s

#35 mlx-community/Qwen3.5-35B-A3B-4bit @ guided-json-simple 13 tokens · 4.8 tok/s

#93 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value 12 tokens · 4.6 tok/s

#95 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma 3 tokens · 1.3 tok/s