MLX Model Test Report — 2026-02-27 11:16

mlx-model-test.sh --model mlx-community/Qwen3.5-35B-A3B-4bit --prompts /Volumes/edata/dev/git/NIGHTLY/maclocal-api/Scripts/test-llm-comprehensive.txt --tests 4,7,23 --smart 1:claude

Performance Ranking (by tokens/sec)

AI Analysis (--smart)

#	Model / Config	Status	claude	Temp	Load (s)	Tokens	Gen (s)	Tokens/sec	Prompt
1	mlx-community/Qwen3.5-35B-A3B-4bit @ top-p top-p --top-p 0.9	OK	5	0.8	1.0	590	5.36	110.1	Explain why the sky is blue in exactly 3 bullet points.
2	mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers combined-samplers --top-k 50 --min-p 0.03 --top-p 0.95	OK	5	0.8	1.0	641	5.9	108.6	Explain why the sky is blue in exactly 3 bullet points.
3	mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 agent-cached-turn1 --enable-prefix-caching	OK	4	0.0	1.0	67	1.13	59.5	Read the file Sources/MacLocalAPI/main.swift and explain wha...

▶ Quality, Anomalies & Recommendations · avg score: 4.2/5

Full Responses

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ top-p 590 tokens · 110.1 tok/s

CLI afm: --top-p 0.9

API variant: top-p temp: 0.8 max_tokens: 4096

Perf finish: stop load: 1.0s gen: 5.36s prompt_tok: 35 comp_tok: 590 tok/s: 110.1

🎯 Test Intent

Tests nucleus sampling (top_p=0.9). Feature: --top-p CLI flag.
Expected: coherent 3-bullet response, quality between greedy and high-temp.

🤖 AI Scores

5/5 ✅ claude: Exactly 3 bullet points, scientifically accurate explanation of Rayleigh scattering, coherent and well-structured. Meets expected outcome perfectly.

PROMPT Explain why the sky is blue in exactly 3 bullet points.

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ combined-samplers 641 tokens · 108.6 tok/s

CLI afm: --top-k 50 --min-p 0.03 --top-p 0.95

API variant: combined-samplers temp: 0.8 max_tokens: 4096

Perf finish: stop load: 1.0s gen: 5.9s prompt_tok: 35 comp_tok: 641 tok/s: 108.6

🎯 Test Intent

Tests combined sampler chain: top-k → min-p → temperature (llama.cpp order).
Feature: all three sampling flags applied together. Expected: coherent response.
Verifies the sampler chain doesn't conflict or crash when all are combined.

🤖 AI Scores

5/5 ✅ claude: Exactly 3 bullet points explaining Rayleigh scattering, correct science, concise and well-structured. Combined samplers (top_k+min_p+top_p+temp) produced coherent output.

PROMPT Explain why the sky is blue in exactly 3 bullet points.

▶ mlx-community/Qwen3.5-35B-A3B-4bit @ agent-cached-turn1 67 tokens · 59.5 tok/s

CLI afm: --enable-prefix-caching system: You are an expert coding assistant. You have access to the following tools:\n\n1...

API variant: agent-cached-turn1 temp: 0.0 max_tokens: 4096

Perf finish: stop load: 1.0s gen: 1.13s prompt_tok: 224 comp_tok: 67 tok/s: 59.5

🎯 Test Intent

Cached turn 1 (caching enabled). Feature: --enable-prefix-caching.
Expected: same quality as no-cache turn 1. This is the first request with this
system prompt, so TTFT may be similar (cache miss). The cache gets populated here.

🤖 AI Scores

4/5 👍 claude: Model correctly invoked read_file tool as expected for an agent/tool-call scenario. Reasoning shows understanding. Content is appropriate first step - reading before explaining. Minor: didn't complete the explanation (would need tool result), but this is expected behavior for a single-turn tool-use test with prefix caching.

PROMPT Read the file Sources/MacLocalAPI/main.swift and explain what it does.