Text Embeddings Inference

A Rust server from Hugging Face for serving open text-embedding, re-ranker and sequence-classification models in production.

Text only: multimodal embedding models (image, audio, video inputs) are not among the supported architectures (README, Supported Models).

Apple Silicon Metal runs via the Homebrew binary or cargo build; the Docker CPU image on ARM64 has no Metal/MPS (README, ARM64 section).

At a glance

Built by: Hugging Face
Status: Active
Latest release: v1.9.4 (2026-09-15)
License: Apache-2.0
Written in: Rust
Serves: Embedding
Hardware: NVIDIA GPU, AMD GPU, Intel CPU / XPU / Gaudi HPU, CPU, Apple Silicon (Metal)
API: HTTP (/embed, /rerank, /predict, OpenAI-compatible /v1/embeddings) or gRPC
Quantization: —

From Text Embeddings Inference's own repo and docs, checked 21 Oct 2026.

Install

docker pull ghcr.io/huggingface/text-embeddings-inference:cuda-1.9

Run a model

docker run --gpus all -p 8080:80 -v $PWD/data:/data ghcr.io/huggingface/text-embeddings-inference:cuda-1.9 --model-id {model}

Replace {model} with the model ID below.

Embeddings, re-rankers and classifiers
Besides dense embeddings (/embed) it serves re-rankers (/rerank), sequence classifiers (/predict) and SPLADE sparse embeddings (/embed_sparse). source ↗
Token-based dynamic batching
Requests are batched by total tokens (--max-batch-tokens), with Flash Attention, Candle and cuBLASLt kernels and no graph-compilation step. source ↗
One Docker image per GPU generation
Separate images for CPU (x86 and arm64), Turing, Ampere 8.0/8.6, Hopper and Blackwell; Turing and Blackwell images are marked experimental. source ↗
Architecture-based support
Supports BERT, CamemBERT, XLM-RoBERTa, NomicBERT, JinaBERT, Mistral, Alibaba GTE, Qwen2, Qwen3, MPNet, ModernBERT and Gemma3 (text-only) models; other architectures will not load. source ↗
Production telemetry
Ships OpenTelemetry distributed tracing and Prometheus metrics; a gRPC API is available via the -grpc image tags. source ↗

Qwen3-Embedding-8BAlibaba · Embeddingsince v1.7.2source ↗
docker run --gpus all -p 8080:80 -v $PWD/data:/data ghcr.io/huggingface/text-embeddings-inference:cuda-1.9 --model-id Qwen/Qwen3-Embedding-8B
Listed in TEI's supported-models table; Qwen's model card gives TEI GPU and CPU commands.
jina-embeddings-v5-text-smallJina AI · Embeddingpartialsource ↗
docker run --gpus all -p 8080:80 -v $PWD/data:/data ghcr.io/huggingface/text-embeddings-inference:cuda-1.9 --model-id jinaai/jina-embeddings-v5-text-small-retrieval
Use Jina's per-task merged repos (-retrieval, -text-matching, -classification, -clustering) with --pooling last-token; the base repo with switchable task adapters is not the TEI path.
Nomic Embed Text v2Nomic · Embeddingsince v1.8.0source ↗
docker run --gpus all -p 8080:80 -v $PWD/data:/data ghcr.io/huggingface/text-embeddings-inference:cuda-1.9 --model-id nomic-ai/nomic-embed-text-v2-moe
Listed in TEI's supported-models table (NomicBERT MoE).

Only models with an official source are listed. Text Embeddings Inference may run others with the same architecture; see its supported-models page.