MLX Model Test Report

Generated 2026-02-25 13:27 · AFM MLX Backend · v0.9.5

mlx-model-test.sh --prompts Scripts/test-stop-sequences.txt --smart claude,codex

Test Runs
34
Passed
34
Failed
0
Best tok/s
53.0
Fastest
mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase

Performance Ranking (by tokens/sec)

Click a row to jump to its full response below.

# Model / Config Status codexclaude Temp Load (s) Tokens Gen (s) Tokens/sec Prompt
1 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase stop-long-phrase OK 54 0.0 1.0 1119 21.13
53.0
Write a 3-paragraph essay about renewable energy. Start the ...
2 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-code-fence stop-code-fence OK 22 0.0 1.0 122 2.6
46.9
Write a Python function that computes factorial, inside a co...
3 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-double-newline stop-api-double-newline OK 23 0.0 1.0 695 15.45
45.0
Write a short paragraph about the ocean. Then write a second...
4 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-period stop-api-period OK 24 0.0 1.0 2041 45.9
44.5
Tell me about the sun in three sentences.
5 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered stop-system-numbered OK 54 0.0 1.0 1010 23.28
43.4
What are the main benefits of exercise?
6 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate stop-system-pirate OK 24 0.0 2.0 1113 26.19
42.5
Tell me about treasure hunting on the high seas.
7 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace stop-guided-json-brace
--guided-json '{"type":"object","properties":{"color":{"type":"string"},"hex":{"type":"string"}},"required":["color","hex"]}'
OK 34 0.0 1.0 819 19.48
42.0
Describe the color blue with its hex code.
8 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-no-match stop-json-object-no-match OK 25 0.0 1.0 473 11.31
41.8
Generate a JSON object with keys "fruit" and "color" for an ...
9 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi stop-cli-multi
--stop "```,END"
OK 55 0.0 1.0 326 7.8
41.8
Write a Python hello world in a code block, then write END.
10 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-no-match stop-guided-json-no-match
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"}},"required":["name","age"]}'
OK 25 0.0 2.0 1150 27.61
41.7
Generate a person record for Bob aged 25.
11 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate stop-immediate OK 53 0.0 1.0 111 2.7
41.1
What is the capital of Japan?
12 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only stop-cli-only
--stop "3."
OK 25 0.0 2.0 309 7.53
41.0
List 10 fruits, numbered 1 through 10, one per line.
13 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-word stop-api-word OK 23 0.0 1.0 2579 63.26
40.8
Name 5 programming languages and briefly describe each one.
14 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match stop-no-match OK 25 0.0 1.0 134 3.32
40.4
What is 2+2? Answer briefly.
15 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode stop-unicode OK 22 0.0 1.0 420 10.62
39.5
List 5 items about space using bullet points (•).
16 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma stop-guided-json-comma
--guided-json '{"type":"object","properties":{"name":{"type":"string"},"age":{"type":"integer"},"city":{"type":"string"}},"required":["name","age","city"]}'
OK 23 0.0 1.0 628 15.94
39.4
Generate a person profile for someone named Alice who is 30 ...
17 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens stop-low-max-tokens OK 25 0.0 1.0 100 2.57
38.8
List 10 mountains, numbered 1 through 10, one per line.
18 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars stop-special-chars OK 22 0.0 1.0 525 13.94
37.6
List 3 facts about the moon. Use **bold** markdown for empha...
19 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-newline stop-api-newline OK 25 0.0 1.0 199 5.58
35.7
What is the capital of France? Answer in one sentence.
20 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key stop-json-object-key OK 42 0.0 1.0 711 20.17
35.2
Generate a JSON object with keys "name", "age", and "city" f...
21 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value stop-guided-json-value
--guided-json '{"type":"object","properties":{"cities":{"type":"array","items":{"type":"string"}}},"required":["cities"]}'
OK 22 0.0 1.0 657 18.68
35.2
List 5 major world cities as a JSON array. Include Tokyo.
22 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word stop-multi-word OK 34 0.0 1.0 585 17.15
34.1
Write a 5-step recipe for making tea. Label each step as "St...
23 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-streaming stop-streaming OK 25 0.0 1.0 627 19.95
31.4
List 10 planets or celestial objects, numbered 1 through 10,...
24 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge stop-cli-api-merge
--stop "5."
OK 55 0.0 1.0 253 8.07
31.4
List 10 countries, numbered 1 through 10, one per line.
25 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming stop-non-streaming
--no-streaming
OK 25 0.0 1.0 627 20.12
31.2
List 10 planets or celestial objects, numbered 1 through 10,...
26 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-single stop-api-single OK 35 0.0 1.0 435 13.98
31.1
List 10 colors, numbered 1 through 10, one per line.
27 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max stop-four-max OK 25 0.0 1.0 359 11.59
31.0
List 10 items, numbered 1 through 10. One per line.
28 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-multi stop-api-multi OK 25 0.0 1.0 471 15.37
30.6
List 10 animals, numbered 1 through 10, one per line.
29 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag stop-html-tag OK 23 0.0 1.0 249 8.15
30.6
Write an HTML unordered list of 5 fruits using <ul> and <li>...
30 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup stop-cli-api-dedup
--stop "3."
OK 25 0.0 1.0 334 11.03
30.3
List 10 cities, numbered 1 through 10, one per line.
31 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 stop-seed-run1 OK 25 0.0 1.0 253 8.87
28.5
List 10 flowers, numbered 1 through 10, one per line.
32 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp stop-high-temp OK 35 1.0 1.0 419 14.69
28.5
List 10 random words, numbered 1 through 10, one per line.
33 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 stop-seed-run2 OK 24 0.0 1.0 253 8.87
28.5
List 10 flowers, numbered 1 through 10, one per line.
34 mlx-community/Qwen3.5-35B-A3B-4bit @ stop-top-p stop-top-p OK 22 0.0 1.0 357 12.78
27.9
List 10 rivers, numbered 1 through 10, one per line.

AI Analysis (--smart)

codex Analysis · avg score: 2.6/5

claude Analysis · avg score: 4.0/5

Full Responses

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-long-phrase 1119 tokens · 53.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-code-fence 122 tokens · 46.9 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-double-newline 695 tokens · 45.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-period 2041 tokens · 44.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-numbered 1010 tokens · 43.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-system-pirate 1113 tokens · 42.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-brace 819 tokens · 42.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-no-match 473 tokens · 41.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-multi 326 tokens · 41.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-no-match 1150 tokens · 41.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-immediate 111 tokens · 41.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-only 309 tokens · 41.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-word 2579 tokens · 40.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-no-match 134 tokens · 40.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-unicode 420 tokens · 39.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-comma 628 tokens · 39.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-low-max-tokens 100 tokens · 38.8 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-special-chars 525 tokens · 37.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-newline 199 tokens · 35.7 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-json-object-key 711 tokens · 35.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-guided-json-value 657 tokens · 35.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-multi-word 585 tokens · 34.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-streaming 627 tokens · 31.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-merge 253 tokens · 31.4 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-non-streaming 627 tokens · 31.2 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-single 435 tokens · 31.1 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-four-max 359 tokens · 31.0 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-api-multi 471 tokens · 30.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-html-tag 249 tokens · 30.6 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-cli-api-dedup 334 tokens · 30.3 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run1 253 tokens · 28.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-high-temp 419 tokens · 28.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-seed-run2 253 tokens · 28.5 tok/s

mlx-community/Qwen3.5-35B-A3B-4bit @ stop-top-p 357 tokens · 27.9 tok/s