MLX Model Test Report

Generated 2026-02-25 21:32 · AFM MLX Backend · v0.9.5-0cfba17

mlx-model-test.sh --prompts Scripts/test-vlm-media.txt

Test Runs

Passed

Failed

Best tok/s

39.1

Fastest

mlx-community/Qwen3.5-35B-A3B-4bit @ image-system-prompt

Performance Ranking (by tokens/sec)

Click a row to jump to its full response below.

#	Model / Config	Status	Temp	Load (s)	Tokens	Gen (s)	Tokens/sec	Prompt
1	mlx-community/Qwen3.5-35B-A3B-4bit @ image-system-prompt image-system-prompt --raw --vlm	OK	0.7	1.0	1932	49.42	39.1	Describe the animal in this image. What breed might it be?
2	mlx-community/Qwen3.5-35B-A3B-4bit @ image-describe image-describe --raw --vlm	OK	0.7	2.0	904	25.32	35.7	Describe this image in detail.
3	mlx-community/Qwen3.5-35B-A3B-4bit @ image-question image-question --raw --vlm	OK	0.7	1.0	512	16.06	31.9	What animal is in this image? Answer in one word.
4	mlx-community/Qwen3.5-35B-A3B-4bit @ image-greedy image-greedy --raw --vlm	OK	0.0	1.0	512	16.12	31.8	What is in this image? Reply in one sentence.
5	mlx-community/Qwen3.5-35B-A3B-4bit @ image-stop image-stop --raw --vlm	OK	0.7	1.0	16	4.3	3.7	Describe this image in detail

Model	Error	Config	Load (s)
mlx-community/Qwen3.5-35B-A3B-4bit @ image-guided-json image-guided-json	Error code: 400 - {'error': {'type': 'mlx_error', 'message': 'The operation couldn’t be completed. (Jinja.TemplateException error 1.)'}}	t=0.7 --raw --vlm	1.0
mlx-community/Qwen3.5-35B-A3B-4bit @ video-describe video-describe	Error code: 400 - {'error': {'message': 'Unable to load image from URL: video/mp4;base64,AAAAIGZ0eXBpc29tAAACAGlzb21pc28yYXZjMW1wNDEAAAAIZnJlZQAPNwdtZGF0AAACrwYF//+r3EXpvebZSLeWLNgg2SPu73gyNjQgLSBjb3J	t=0.7 --raw --vlm	1.0