MLX LM
A Python package for generating text with, and fine-tuning, large language models on Apple silicon using MLX.
The HTTP server is "not recommended for production as it only implements basic security checks" (mlx_lm/SERVER.md).
For multimodal models mlx-lm loads the text model only: its model files drop the vision (and audio) towers. Many mlx-community conversions of vision models are made for the separate mlx-vlm package.
Embedding models are not served by mlx-lm, so none are listed.
At a glance
- Built by
- ml-explore (Apple machine learning research)
- Status
- Active
- Latest release
- v0.32.2 (2026-10-16)
- License
- MIT
- Written in
- Python
- Serves
- Language
- Hardware
- Apple Silicon
- API
- OpenAI-like HTTP server (mlx_lm.server); Python API
- Quantization
- Affine 2–8-bit, MXFP4, NVFP4, MXFP8, DWQ, AWQ, GPTQ
From MLX LM's own repo and docs, checked 21 Oct 2026.
Get started
pip install mlx-lmmlx_lm.server --model {model}Replace {model} with the model ID below.
What it's good at
- Hugging Face in one flag
Any MLX-compatible model loads with --model <repo>; thousands of ready-converted models live in the mlx-community Hugging Face org. source ↗
- Quantize and upload yourself
mlx_lm.convert -q quantizes a Hugging Face model (affine, MXFP4, NVFP4 or MXFP8 modes) and can upload it; learned methods DWQ, AWQ and GPTQ are also included. source ↗
- Fine-tuning built in
LoRA and full fine-tuning, including on quantized models, plus distributed inference and fine-tuning with mx.distributed. source ↗
- Long-context tools
A rotating fixed-size KV cache (--max-kv-size), prompt caching to files, and a configurable prefill step size to cap peak memory. source ↗
Catalog models it runs 20
mlx_lm.server --model Qwen/Qwen3.8-27BText only: mlx-lm's qwen3_5 model drops the vision tower.
mlx_lm.server --model mlx-community/DeepSeek-V4.1-Flash-MLX-4bitArchitecture DeepseekV41ForCausalLM (deepseek_v41) was added in PR #1895.
mlx_lm.server --model mlx-community/gemma-3-text-27b-it-4bitText only: mlx-lm's gemma3 model drops the vision tower.
mlx_lm.server --model google/gemma-4-31B-itText only: mlx-lm's gemma4 model drops the vision and audio towers.
mlx_lm.server --model mlx-community/Meta-Llama-3.1-8B-Instruct-4bitmlx_lm.server --model mlx-community/Llama-3.3-70B-Instruct-4bitmlx_lm.server --model meta-llama/Llama-4-Maverick-17B-128E-InstructText only: mlx-lm's llama4 model removes the vision weights.
mlx_lm.server --model meta-llama/Llama-4-Scout-17B-16E-InstructText only: mlx-lm's llama4 model removes the vision weights.
mlx_lm.server --model meta-models/Muse-Glimmer-30BText only: mlx-lm's muse_glimmer model is a text-only port that drops the vision tower.
mlx_lm.server --model mlx-community/Muse-Spark-Open-816B-A42B-4bitThe mlx-community card asks for mlx-lm 0.32.2 or newer.
mlx_lm.server --model mistralai/Ministral-3-14B-Instruct-2512Text only: mlx-lm's mistral3 model (ministral3 text model) skips the vision tower.
mlx_lm.server --model mistralai/Ministral-3-3B-Instruct-2512Text only: mlx-lm's mistral3 model (ministral3 text model) skips the vision tower.
mlx_lm.server --model mistralai/Ministral-3-8B-Instruct-2512Text only: mlx-lm's mistral3 model (ministral3 text model) skips the vision tower.
mlx_lm.server --model mistralai/Mistral-Medium-3.5-128BText only: mlx-lm's mistral3 model (ministral3 text model) skips the vision tower.
mlx_lm.server --model mistralai/Mistral-Small-4-119B-2603Text only: mlx-lm's mistral3 model (mistral4 text model) skips the vision tower.
mlx_lm.server --model moonshotai/Kimi-K3Architecture kimi_k3 is in mlx-lm, text only (vision weights are dropped); no full mlx-community conversion.
mlx_lm.server --model mlx-community/Nemotron-3-Ultra-550B-A55B-4bitArchitecture nemotron_h.
mlx_lm.server --model mlx-community/gpt-oss-120b-MXFP4-Q8mlx_lm.server --model mlx-community/gpt-oss-20b-MXFP4-Q8mlx_lm.server --model mlx-community/GLM-5.3-4bitConverted with mlx-lm 0.31.3 (architecture glm_moe_dsa).
Only models with an official source are listed. MLX LM may run others with the same architecture; see its supported-models page.