MLX Model Test Report

Generated 2026-02-25 16:08 · AFM MLX Backend · v0.9.5-050e836

mlx-model-test.sh --prompts Scripts/test-stop-sequences.txt --smart claude,codex

Test Runs
34
Passed
34
Failed
0
Best tok/s
53.7
Fastest
mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase

Performance Ranking (by tokens/sec)

Click a row to jump to its full response below.

# Model / Config Status codexclaude Temp Load (s) Tokens Gen (s) Tokens/sec Prompt
1 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase stop-long-phrase OK 54 0.0 1.0 1119 20.85
53.7
 Write a 3-paragraph essay about renewable energy. Start the ...
2 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-code-fence stop-code-fence OK 25 0.0 1.0 122 2.56
47.7
 Write a Python function that computes factorial, inside a co...
3 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-double-newline stop-api-double-newline OK 22 0.0 1.0 695 15.4
45.1
 Write a short paragraph about the ocean. Then write a second...
4 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-period stop-api-period OK 25 0.0 1.0 2041 45.28
45.1
 Tell me about the sun in three sentences.
5 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered stop-system-numbered OK 55 0.0 1.0 1010 22.69
44.5
 What are the main benefits of exercise?
6 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace stop-guided-json-brace
--guided-json '{"type":"object","properties":{"color":{"type":"string"},"hex":{"type":"string"}},"required":["color","hex"]}'		 OK 32 0.0 1.0 819 18.99
43.1
 Describe the color blue with its hex code.
7 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate stop-system-pirate OK 25 0.0 1.0 1113 25.81
43.1
 Tell me about treasure hunting on the high seas.
8 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-no-match stop-guided-json-no-match
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]}'		 OK 25 0.0 1.0 1150 26.94
42.7
 Generate a person record for Bob aged 25.
9 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-no-match stop-json-object-no-match OK 24 0.0 1.0 473 11.1
42.6
 Generate a JSON object with keys "fruit" and "color" for an ...
10 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only stop-cli-only
--stop "3."		 OK 25 0.0 1.0 309 7.32
42.2
 List 10 fruits, numbered 1 through 10, one per line.
11 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi stop-cli-multi
--stop "```,END"		 OK 55 0.0 1.0 326 7.74
42.1
 Write a Python hello world in a code block, then write END.
12 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate stop-immediate OK 53 0.0 1.0 111 2.66
41.7
 What is the capital of Japan?
13 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-word stop-api-word OK 23 0.0 1.0 2579 62.63
41.2
 Name 5 programming languages and briefly describe each one.
14 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match stop-no-match OK 22 0.0 1.0 134 3.28
40.8
 What is 2+2? Answer briefly.
15 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode stop-unicode OK 25 0.0 1.0 420 10.48
40.1
 List 5 items about space using bullet points (•).
16 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma stop-guided-json-comma
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"},"city":{"type":"string"}},"required":["name","age","city"]}'		 OK 25 0.0 1.0 628 15.91
39.5
 Generate a person profile for someone named Alice who is 30 ...
17 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens stop-low-max-tokens OK 2— 0.0 1.0 100 2.58
38.8
 List 10 mountains, numbered 1 through 10, one per line.
18 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars stop-special-chars OK 23 0.0 1.0 525 13.76
38.1
 List 3 facts about the moon. Use **bold** markdown for empha...
19 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key stop-json-object-key OK 35 0.0 1.0 711 19.6
36.3
 Generate a JSON object with keys "name", "age", and "city" f...
20 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-newline stop-api-newline OK 23 0.0 1.0 199 5.51
36.1
 What is the capital of France? Answer in one sentence.
21 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value stop-guided-json-value
--guided-json '{"type":"object","properties":{"cities":{"type":"array","items":{"type":"string"}}},"required":["cities"]}'		 OK 24 0.0 1.0 657 18.38
35.8
 List 5 major world cities as a JSON array. Include Tokyo.
22 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word stop-multi-word OK 22 0.0 1.0 585 16.93
34.6
 Write a 5-step recipe for making tea. Label each step as "St...
23 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-streaming stop-streaming OK 25 0.0 1.0 627 19.66
31.9
 List 10 planets or celestial objects, numbered 1 through 10,...
24 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge stop-cli-api-merge
--stop "5."		 OK 55 0.0 1.0 253 7.98
31.7
 List 10 countries, numbered 1 through 10, one per line.
25 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-single stop-api-single OK 35 0.0 1.0 435 13.74
31.6
 List 10 colors, numbered 1 through 10, one per line.
26 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming stop-non-streaming
--no-streaming		 OK 22 0.0 1.0 627 19.82
31.6
 List 10 planets or celestial objects, numbered 1 through 10,...
27 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max stop-four-max OK 25 0.0 1.0 359 11.38
31.5
 List 10 items, numbered 1 through 10. One per line.
28 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-multi stop-api-multi OK 25 0.0 1.0 471 15.15
31.1
 List 10 animals, numbered 1 through 10, one per line.
29 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag stop-html-tag OK 22 0.0 1.0 249 8.06
30.9
 Write an HTML unordered list of 5 fruits using <ul> and <li>...
30 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup stop-cli-api-dedup
--stop "3."		 OK 25 0.0 1.0 334 11.03
30.3
 List 10 cities, numbered 1 through 10, one per line.
31 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp stop-high-temp OK 32 1.0 1.0 692 22.86
30.3
 List 10 random words, numbered 1 through 10, one per line.
32 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 stop-seed-run2 OK 2— 0.0 1.0 253 8.74
28.9
 List 10 flowers, numbered 1 through 10, one per line.
33 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 stop-seed-run1 OK 2— 0.0 1.0 253 8.75
28.9
 List 10 flowers, numbered 1 through 10, one per line.
34 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-top-p stop-top-p OK 2— 0.0 2.0 357 12.56
28.4
 List 10 rivers, numbered 1 through 10, one per line.

AI Analysis (--smart)

▶ codex Analysis · avg score: 2.6/5

▶ claude Analysis · avg score: 3.9/5

Full Responses

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase 1119 tokens · 53.7 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-code-fence 122 tokens · 47.7 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-double-newline 695 tokens · 45.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-period 2041 tokens · 45.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered 1010 tokens · 44.5 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace 819 tokens · 43.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate 1113 tokens · 43.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-no-match 1150 tokens · 42.7 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-no-match 473 tokens · 42.6 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only 309 tokens · 42.2 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi 326 tokens · 42.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate 111 tokens · 41.7 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-word 2579 tokens · 41.2 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match 134 tokens · 40.8 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode 420 tokens · 40.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma 628 tokens · 39.5 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens 100 tokens · 38.8 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars 525 tokens · 38.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key 711 tokens · 36.3 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-newline 199 tokens · 36.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value 657 tokens · 35.8 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word 585 tokens · 34.6 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-streaming 627 tokens · 31.9 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge 253 tokens · 31.7 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-single 435 tokens · 31.6 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming 627 tokens · 31.6 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max 359 tokens · 31.5 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-multi 471 tokens · 31.1 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag 249 tokens · 30.9 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup 334 tokens · 30.3 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp 692 tokens · 30.3 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 253 tokens · 28.9 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 253 tokens · 28.9 tok/s

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ stop-top-p 357 tokens · 28.4 tok/s