Inference engines
Software for running open-weights models on your own hardware. 11 engines, and which of the catalog's 45 open models each one runs, from the engine's own docs and model lists or the maker's model card.
Engines
- vLLM32 models
A Python library and OpenAI-compatible server for high-throughput LLM inference and serving on GPUs and other accelerators.
NVIDIA GPUAMD GPUIntel GPUx86 / ARM / PowerPC CPU+4 - llama.cpp31 models
A dependency-free C/C++ engine for running GGUF models locally or in the cloud, from laptops and phones to multi-GPU servers.
CPU (x86, ARM, RISC-V)Apple Silicon (Metal)NVIDIA GPUAMD GPU+6 - SGLang26 models
An open-source serving framework for large language, vision-language and diffusion models, aimed at large-scale serving, agentic workloads and RL rollouts.
NVIDIA GPUAMD GPUGoogle TPUIntel GPU+4 - Ollama25 models
A desktop app, CLI and local server that pulls ready-packaged open models from its own library and runs them with one command; aimed at developers working on their own machine.
NVIDIA GPUAMD GPUApple Silicon (Metal)Vulkan GPUs+1 - TensorRT-LLM21 models
NVIDIA's open-source library and server for fast LLM (and visual generation) inference on NVIDIA GPUs, with a Python API and an OpenAI-compatible trtllm-serve.
NVIDIA GPU - MLX LM20 models
A Python package for generating text with, and fine-tuning, large language models on Apple silicon using MLX.
Apple Silicon - ComfyUI12 models
A node-graph app and local server for running open diffusion models (image, video, audio, 3D) as reusable workflows, aimed at creators and pipeline builders.
NVIDIA GPUAMD GPUIntel GPUApple Silicon+4 - Diffusers12 models
Hugging Face's PyTorch library of pretrained diffusion pipelines for image, video and audio generation, for developers who run models from Python.
NVIDIA GPUApple Silicon (MPS)Intel GaudiAWS Inferentia/Trainium (via Optimum Neuron)+2 - LMDeploy10 models
A toolkit for compressing, deploying and serving LLMs and VLMs, with a C++/CUDA engine (TurboMind) and a pure-Python PyTorch engine.
NVIDIA GPUHuawei Ascend NPUMetaX Maca GPUCambricon MLU - Text Embeddings Inference3 models
A Rust server from Hugging Face for serving open text-embedding, re-ranker and sequence-classification models in production.
NVIDIA GPUAMD GPUIntel CPU / XPU / Gaudi HPUCPU+1 - Text Generation Inference (TGI)5 models
A Rust, Python and gRPC server for serving open LLMs, used by Hugging Face for HuggingChat and Inference Endpoints; now archived, with vLLM and SGLang recommended instead.
archivedNVIDIA GPUAMD GPUAWS InferentiaIntel GPU+2
Language models 25
| Model | vLLM | llama.cpp | SGLang | Ollama | TensorRT-LLM | MLX LM | LMDeploy | TGI |
|---|---|---|---|---|---|---|---|---|
| Qwen3.8-27BAlibaba | ✓ | ✓ | ✓ | ✓ | partial | partial | ✓ | · |
| Qwen3.8-FlashAlibaba | ✓ | ✓ | ✓ | ✓ | partial | · | · | · |
| Qwen3.8-MaxAlibaba | ✓ | ✓ | ✓ | · | partial | · | · | · |
| Command A+Cohere | ✓ | ✓ | ✓ | · | · | · | · | · |
| DeepSeek-V4-ProDeepSeek | ✓ | ✓ | ✓ | partial | partial | · | ✓ | · |
| DeepSeek-V4.1-FlashDeepSeek | ✓ | · | partial | partial | partial | ✓ | · | · |
| Gemma 3 27BGoogle | ✓ | ✓ | partial | ✓ | ✓ | partial | ✓ | ✓ |
| Gemma 4 31BGoogle | ✓ | ✓ | ✓ | ✓ | partial | partial | · | · |
| Llama 3.1 8BMeta | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Llama 3.3 70BMeta | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Llama 4 MaverickMeta | ✓ | ✓ | ✓ | ✓ | ✓ | partial | ✓ | ✓ |
| Llama 4 ScoutMeta | ✓ | ✓ | ✓ | ✓ | ✓ | partial | ✓ | ✓ |
| Muse Glimmer 30BMeta | ✓ | ✓ | ✓ | ✓ | · | partial | · | · |
| Muse Spark OpenMeta | partial | ✓ | partial | partial | · | ✓ | · | · |
| Ministral 3 14BMistral AI | ✓ | ✓ | ✓ | ✓ | ✓ | partial | · | · |
| Ministral 3 3BMistral AI | ✓ | ✓ | ✓ | ✓ | ✓ | partial | · | · |
| Ministral 3 8BMistral AI | ✓ | ✓ | ✓ | ✓ | ✓ | partial | · | · |
| Mistral Large 3Mistral AI | ✓ | ✓ | · | partial | ✓ | · | · | · |
| Mistral Medium 3.5Mistral AI | ✓ | ✓ | ✓ | ✓ | ✓ | partial | · | · |
| Mistral Small 4Mistral AI | ✓ | ✓ | ✓ | · | · | partial | · | · |
| Kimi K3Moonshot AI | ✓ | ✓ | ✓ | partial | partial | partial | · | · |
| NVIDIA Nemotron 3 UltraNVIDIA | ✓ | ✓ | ✓ | partial | partial | ✓ | · | · |
| gpt-oss-120bOpenAI | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · |
| gpt-oss-20bOpenAI | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · |
| GLM-5.3Z.ai | ✓ | ✓ | ✓ | partial | partial | ✓ | ✓ | · |
Embedding models 8
| Model | vLLM | llama.cpp | SGLang | Ollama | TEI |
|---|---|---|---|---|---|
| Qwen3-Embedding-8BAlibaba | ✓ | ✓ | ✓ | ✓ | ✓ |
| EmbeddingGemma 2Google | ✓ | ✓ | ✓ | ✓ | · |
| jina-embeddings-v5-omni-smallJina AI | ✓ | partial | · | · | · |
| jina-embeddings-v5-text-smallJina AI | ✓ | ✓ | · | · | partial |
| Nomic Embed Text v2Nomic | ✓ | ✓ | · | ✓ | ✓ |
| NVIDIA Nemotron 3.5 Embed 1BNVIDIA | ✓ | ✓ | · | · | · |
| NVIDIA Nemotron 3.5 Embed 8BNVIDIA | ✓ | ✓ | · | · | · |
| Nomic Embed Multimodal 7BNomic | · | · | · | · | · |
Image models 8
| Model | ComfyUI | Diffusers |
|---|---|---|
| Qwen-Image-2.1Alibaba | ✓ | ✓ |
| FLUX 3 Image KleinBlack Forest Labs | ✓ | partial |
| FLUX.2 [dev]Black Forest Labs | ✓ | ✓ |
| FLUX.2 [klein] 4BBlack Forest Labs | ✓ | ✓ |
| Ideogram 4.0Ideogram | ✓ | partial |
| Krea 2Krea | ✓ | ✓ |
| Stable Diffusion 3.5 LargeStability AI | ✓ | ✓ |
| Stable Diffusion 3.5 MediumStability AI | ✓ | ✓ |
Video models 3
Audio models 1
| Model | ComfyUI | Diffusers |
|---|---|---|
| Stable Audio 3.0 MediumStability AI | ✓ | partial |
✓ links to the source that says the engine runs the model. “partial” has a stated limit (hover for it). A dot means we found no official source either way, not that it doesn't work: engines that support a model's architecture often run it before their docs say so.