Embeddings providers
The complete embeddings-provider reference for the RAG API — how to select a provider and the exact keys, endpoints, and models for OpenAI, Azure OpenAI, HuggingFace (local and TEI), Ollama, Google GenAI, Google VertexAI, and AWS Bedrock.
The RAG API embeds document chunks with a pluggable embeddings provider, chosen
with EMBEDDINGS_PROVIDER. This page documents the shared selection settings and
then the exact keys and models for each provider.
Selecting a provider
| Name | Purpose | Default | Required |
|---|---|---|---|
EMBEDDINGS_PROVIDER | Backend: openai, azure, huggingface, huggingfacetei, ollama, google_genai, vertexai, bedrock | openai | No |
EMBEDDINGS_MODEL | Model name/URI (provider-specific; see each section) | (per provider) | No |
EMBEDDINGS_CHUNK_SIZE | Input tokens per embedding request (batching to avoid rate limits) | 200 | No |
RAG_CHECK_EMBEDDING_CTX_LENGTH | Validate input token length before embedding | true | No |
Dedicated RAG keys take precedence
For providers the app also uses (OpenAI, Azure, Google), the RAG API prefers a
dedicated RAG_* key so its credentials don't collide with the app's. Where a
RAG_* key is unset it falls back to the shared key (e.g. OPENAI_API_KEY).
OpenAI
EMBEDDINGS_PROVIDER=openai
| Name | Purpose | Default | Required |
|---|---|---|---|
RAG_OPENAI_API_KEY | OpenAI API key (preferred) | value of OPENAI_API_KEY | Yes* |
OPENAI_API_KEY | Fallback API key | (unset) | No |
RAG_OPENAI_BASEURL | Custom endpoint (proxy, local, or OpenAI-compatible server) | (unset) | No |
RAG_OPENAI_PROXY | HTTP proxy for OpenAI requests | (unset) | No |
* One of RAG_OPENAI_API_KEY / OPENAI_API_KEY is required for this provider.
Common models: text-embedding-3-small (default, 1536 dims), text-embedding-3-large
(3072 dims), text-embedding-ada-002 (legacy).
Azure OpenAI
EMBEDDINGS_PROVIDER=azure
| Name | Purpose | Default | Required |
|---|---|---|---|
RAG_AZURE_OPENAI_ENDPOINT | Azure resource endpoint (https://<res>.openai.azure.com/) | value of AZURE_OPENAI_ENDPOINT | Yes |
AZURE_OPENAI_ENDPOINT | Fallback endpoint | (unset) | No |
RAG_AZURE_OPENAI_API_KEY | Azure API key (preferred) | value of AZURE_OPENAI_API_KEY | Yes |
AZURE_OPENAI_API_KEY | Fallback API key | (unset) | No |
RAG_AZURE_OPENAI_API_VERSION | Azure OpenAI API version | 2023-05-15 | No |
EMBEDDINGS_MODEL is your Azure deployment name (e.g. text-embedding-3-small).
HuggingFace (local / sentence-transformers)
EMBEDDINGS_PROVIDER=huggingface
| Name | Purpose | Default | Required |
|---|---|---|---|
EMBEDDINGS_MODEL | HuggingFace Hub model ID | sentence-transformers/all-MiniLM-L6-v2 | No |
HF_TOKEN | HuggingFace token (for gated/private models) | (unset) | No |
Models download on first run and cache under ~/.cache/huggingface/hub/. Examples:
all-MiniLM-L6-v2 (384 dims, fast), all-mpnet-base-v2 (768 dims, higher quality),
intfloat/multilingual-e5-small (multilingual).
HuggingFace TEI (inference endpoint)
EMBEDDINGS_PROVIDER=huggingfacetei
| Name | Purpose | Default | Required |
|---|---|---|---|
EMBEDDINGS_MODEL | URL of a running Text Embeddings Inference service | http://huggingfacetei:3000 | No |
The model is configured on the TEI service itself, not here. Requires a separate TEI container (a lightweight GPU inference service).
Ollama (local)
EMBEDDINGS_PROVIDER=ollama
| Name | Purpose | Default | Required |
|---|---|---|---|
OLLAMA_BASE_URL | Ollama service base URL | http://ollama:11434 | No |
EMBEDDINGS_MODEL | Ollama embedding model (must be an embedding model) | nomic-embed-text | No |
Pull the model into the Ollama container first (ollama pull nomic-embed-text).
Google GenAI (Gemini)
EMBEDDINGS_PROVIDER=google_genai
| Name | Purpose | Default | Required |
|---|---|---|---|
RAG_GOOGLE_API_KEY | Google API key (highest priority) | value of GOOGLE_KEY/GOOGLE_API_KEY | Yes |
GOOGLE_KEY | Fallback key | value of GOOGLE_API_KEY | No |
GOOGLE_API_KEY | Final fallback key | (unset) | No |
EMBEDDINGS_MODEL | Google GenAI embedding model | gemini-embedding-001 | No |
Priority: RAG_GOOGLE_API_KEY → GOOGLE_KEY → GOOGLE_API_KEY.
Google VertexAI (GCP)
EMBEDDINGS_PROVIDER=vertexai
| Name | Purpose | Default | Required |
|---|---|---|---|
GOOGLE_APPLICATION_CREDENTIALS | Path to a GCP service-account JSON file | (unset) | Yes |
GOOGLE_CLOUD_PROJECT | GCP project ID | (unset) | Yes |
GOOGLE_CLOUD_LOCATION | GCP region for the VertexAI endpoint | us-central1 | No |
RAG_GOOGLE_API_KEY | API-key auth instead of the service-account file | (unset) | No |
EMBEDDINGS_MODEL | VertexAI embedding model | gemini-embedding-001 | No |
AWS Bedrock
EMBEDDINGS_PROVIDER=bedrock
| Name | Purpose | Default | Required |
|---|---|---|---|
AWS_ACCESS_KEY_ID | AWS access key (needs bedrock:InvokeModel) | (unset) | Yes |
AWS_SECRET_ACCESS_KEY | AWS secret key | (unset) | Yes |
AWS_DEFAULT_REGION | Bedrock region | us-east-1 | No |
AWS_SESSION_TOKEN | Session token for temporary STS credentials | (unset) | No |
EMBEDDINGS_MODEL | Bedrock embedding model ID | amazon.titan-embed-text-v1 | No |
Common models: amazon.titan-embed-text-v1, amazon.titan-embed-text-v2:0,
cohere.embed-english-v3. Prefer IAM roles over static keys in production.
Choosing dimensions consistently
Don't change model dimensions on a populated store
The embedding dimension is fixed per model (e.g. 1536 for
text-embedding-3-small, 384 for all-MiniLM-L6-v2). Switching to a model with a
different dimension after documents are indexed will make existing vectors
incompatible — you must re-index. Keep EMBEDDINGS_MODEL stable, or plan a full
re-index when you change it.
Last updated on