## What's Changed * Fix auto-truncate false setting by @vrdn-23 in https://github.com/huggingface/text-embeddings-inference/pull/836 * Set `pad_token_id` as nullable & add support for `rope_parameters` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/832 * docs: add Homebrew installation to README by @Peredery in https://github.com/huggingface/text-embeddings-inference/pull/834 * feat: support pplx-embed-v1 by @mkrimmel-pplx in https://github.com/huggingface/text-embeddings-inference/pull/824 ## New Contributors * @Peredery made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/834 * @mkrimmel-pplx made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/824 **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.9.1...v1.9.2

## What's Changed ### 🚨 Fix * Fix support for containers w/ CUDA 13.0+ by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/831 > When releasing ghcr.io/huggingface/text-embeddings-inference:cuda-1.9 with CUDA 12.9 and `cuda-compat-12-9` there was an issue when running that same container on instances with CUDA 13.0+, as the `cuda-compat-12-9` set in `LD_LIBRARY_PATH` was leading to a `CUDA_ERROR_SYSTEM_DRIVER_MISMATCH = 803`, which is now solved with a custom entrypoint that dynamically includes the `cuda-compat` on the `LD_LIBRARY_PATH` depending on the instance CUDA version. **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.9.0...v1.9.1

## Notable Changes * Add support for Gemma3 (text-only) architecture * Intel updates to Synapse 1.21.3 and IPEX 2.8 * Extend ONNX Runtime support in `OrtRuntime` * Support `position_ids` and `past_key_values` as inputs * Handle `padding_side` and `pad_token_id` ## What's Changed * Adjust HPU warmup: use dummy inputs with shape more close to real scenario by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/689 * Add `extra_args` to `trufflehog` to exclude unverified results by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/696 * Update GitHub templates & fix mentions to Text Embeddings Inference by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/697 * Disable Flash Attention with `USE_FLASH_ATTENTION` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/692 * Add support for `position_ids` and `past_key_values` in `OrtBackend` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/700 * HPU upgrade to Synapse 1.21.3 by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/703 * Upgrade to IPEX 2.8 by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/702 * Parse `modules.json` to identify default `Dense` modules by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/701 * Add `padding_side` and `pad_token_id` in `OrtBackend` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/705 * Update `docs/openapi.json` for v1.8.0 by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/708 * Add Gemma3 architecture (text-only) by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/711 * Update `version` to 1.8.1 by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/712 **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.8.0...v1.8.1

## Noticeable Changes Qwen3 support included for Intel HPU, and fixed for CPU / Metal / CUDA. ## What's Changed * Default to Qwen3 in `README.md` and `docs/` examples by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/641 * Fix Qwen3 by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/646 * Add integration tests for Gaudi by @baptistecolle in https://github.com/huggingface/text-embeddings-inference/pull/598 * Fix Qwen3-Embedding batch vs single inference inconsistency by @lance-miles in https://github.com/huggingface/text-embeddings-inference/pull/648 * Fix FlashQwen3 by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/650 * Make flake work on metal by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/654 * Fixing metal backend. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/655 * Qwen3 hpu support by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/656 * change HPU warmup logic: seq length should be with exponential growth by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/659 * Update `version` to 1.7.3 by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/666 * Add last token pooling support for ORT. by @tpendragon in https://github.com/huggingface/text-embeddings-inference/pull/664 ## New Contributors * @lance-miles made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/648 * @tpendragon made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/664 **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.7.2...v1.7.3

## What's Changed * [Docs] Update quick tour by @NielsRogge in https://github.com/huggingface/text-embeddings-inference/pull/574 * Update `README.md` and `supported_models.md` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/572 * Back with linting. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/577 * [Docs] Add cloud run example by @NielsRogge in https://github.com/huggingface/text-embeddings-inference/pull/573 * Fixup by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/578 * Fixing the tokenization routes token (offsets are in bytes, not in by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/576 * Removing requirements file. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/585 * Removing candle-extensions to live on crates.io by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/583 * Bump `sccache` to 0.10.0 and `sccache-action` to 0.0.9 by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/586 * optimize the performance of FlashBert Path for HPU by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/575 * Revert "Removing requirements file. (#585)" by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/588 * Get opentelemetry trace id from request headers by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/425 * Add argument for configuring Prometheus port by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/589 * Adding missing `head.` prefix in the weight name in `ModernBertClassificationHead` by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/591 * Fixing the CI (grpc path). by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/593 * fix xpu env issue that cannot find right libur_loader.so.0 by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/595 * enable flash mistral model for HPU device by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/594 * remove optimum-habana dependency by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/599 * Support NomicBert MoE by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/596 * Remove duplicate short option '-p' to fix router executable by @cebtenzzre in https://github.com/huggingface/text-embeddings-inference/pull/602 * Update `text-embeddings-router --help` output by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/603 * Warmup padded models too. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/592 * Add support for JinaAI Re-Rankers V1 by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/582 * Gte diffs by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/604 * Fix the weight name in GTEClassificationHead by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/606 * upgrade pytorch and ipex to 2.7 version by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/607 * upgrade HPU FW to 1.21; upgrade transformers to 4.51.3 by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/608 * Patch DistilBERT variants with different weight keys by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/614 * add offline modeling for model `jinaai/jina-embeddings-v2-base-code` to avoid `auto_map` to other repository by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/612 * Add mean pooling strategy for Modernbert classifier by @kwnath in https://github.com/huggingface/text-embeddings-inference/pull/616 * Using serde for pool validation. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/620 * Preparing the update to 1.7.1 by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/623 ## New Contributors * @NielsRogge made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/574 * @cebtenzzre made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/602 * @kwnath made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/616 **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.7.0...v1.7.1

## Notable changes - Upgrade dependencies heavily (candle 0.5 -> 0.8 and related) - Added ModernBert support by @kozistr ! ## What's Changed * Moving cublaslt into TEI extension for easier upgrade of candle globally by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/542 * Upgrade candle2 by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/543 * Upgrade candle3 by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/545 * Fixing the static-linking. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/547 * Fix linking bis by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/549 * Make `sliding_window` for `Qwen2` optional by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/546 * Optimize the performance of FlashBert on HPU by using fast mode softmax by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/555 * Fixing cudarc to the latest unified bindings. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/558 * Fix typos / formatting in CLI args in Markdown files by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/552 * Use custom `serde` deserializer for JinaBERT models by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/559 * Implement the `ModernBert` model by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/459 * Fixing FlashAttention ModernBert. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/560 * Enable ModernBert on metal by @ivarflakstad in https://github.com/huggingface/text-embeddings-inference/pull/562 * Fix `{Bert,DistilBert}SpladeHead` when loading from Safetensors by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/564 * add related docs for intel cpu/xpu/hpu container by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/550 * Update the doc for submodule. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/567 * Update `docs/source/en/custom_container.md` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/568 * Preparing for release 1.7.0 (candle update + modernbert). by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/570 ## New Contributors * @ivarflakstad made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/562 **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.6.1...v1.7.0

## What's Changed * Enable intel devices CPU/XPU/HPU for python backend by @yuanwu2017 in https://github.com/huggingface/text-embeddings-inference/pull/245 * add reranker model support for python backend by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/386 * (FIX): CI Security Fix - branchname injection by @glegendre01 in https://github.com/huggingface/text-embeddings-inference/pull/479 * Upgrade TEI. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/501 * Pin `cargo-chef` installation to 0.1.62 by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/469 * add `TRUST_REMOTE_CODE` param to python backend. by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/485 * Enable splade embeddings for Python backend by @pi314ever in https://github.com/huggingface/text-embeddings-inference/pull/493 * Hpu bucketing by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/489 * Optimize flash bert path for hpu device by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/509 * upgrade ipex to 2.6 version for cpu/xpu by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/510 * fix bug for `MaskedLanguageModel` class` by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/513 * Fix double incrementing `te_request_count` metric by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/486 * Add intel based images to the CI by @baptistecolle in https://github.com/huggingface/text-embeddings-inference/pull/518 * Fix typo on intel docker image by @baptistecolle in https://github.com/huggingface/text-embeddings-inference/pull/529 * chore: Upgrade to tokenizers 0.21.0 by @lightsofapollo in https://github.com/huggingface/text-embeddings-inference/pull/512 * feat: add support for "model_type": "gte" by @anton-pt in https://github.com/huggingface/text-embeddings-inference/pull/519 * Update `README.md` to include ONNX by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/507 * Fusing both Gte Configs. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/530 * Add `HF_HUB_USER_AGENT_ORIGIN` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/534 * Use `--hf-token` instead of `--hf-api-token` by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/535 * Fixing the tests. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/531 * Support classification head for DistilBERT by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/487 * add CLI flag `disable-spans` to toggle span trace logging by @obloomfield in https://github.com/huggingface/text-embeddings-inference/pull/481 * feat: support HF_ENDPOINT environment when downloading model by @StrayDragon in https://github.com/huggingface/text-embeddings-inference/pull/505 * Small fixup. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/537 * Fix `VarBuilder` handling in GTE e.g. `gte-multilingual-reranker-base` by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/538 * make a WA in case Bert model do not have `safetensor` file by @kaixuanliu in https://github.com/huggingface/text-embeddings-inference/pull/515 * Add missing `match` on `onnx/model.onnx` download by @alvarobartt in https://github.com/huggingface/text-embeddings-inference/pull/472 * Fixing the impure flake devShell to be able to run python code. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/539 * Prepare for release. by @Narsil in https://github.com/huggingface/text-embeddings-inference/pull/540 ## New Contributors * @yuanwu2017 made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/245 * @kaixuanliu made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/386 * @Narsil made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/501 * @pi314ever made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/493 * @baptistecolle made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/518 * @lightsofapollo made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/512 * @anton-pt made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/519 * @obloomfield made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/481 * @StrayDragon made their first contribution in https://github.com/huggingface/text-embeddings-inference/pull/505 **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.6.0...v1.6.1

## What's Changed * feat: support multiple backends at the same time by @OlivierDehaene in https://github.com/huggingface/text-embeddings-inference/pull/440 * feat: GTE classification head by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/441 * feat: Implement GTE model to support the non-flash-attn version by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/446 * feat: Implement MPNet model (#363) by @kozistr in https://github.com/huggingface/text-embeddings-inference/pull/447 **Full Changelog**: https://github.com/huggingface/text-embeddings-inference/compare/v1.5.1...v1.6.0