ITADN

Inconsistency between HTTP and gRPC output normalization (Qwen3-Embedding)

#807OpenKyogre-Primal 创建于 2026-01-27
K
Kyogre-Primalcommented
### System Info I noticed a significant inconsistency in the output embeddings between the HTTP and gRPC interfaces when using the same model and start-up parameters. The HTTP /embed endpoint returns L2-normalized vectors by default, whereas the gRPC tei.v1.Embed/Embed method returns raw, unnormalized vectors. ### Information - [x] Docker - [ ] The CLI directly ### Tasks - [x] An officially supported command - [ ] My own modifications ### Reproduction Start the HTTP server: Bash docker run --gpus '"device=0"' -d -p 8080:80 \ -v ~/model:/data \ --name tei-http \ ghcr.io/huggingface/text-embeddings-inference:turing-1.8 \ --model-id /data/Qwen/Qwen3-Embedding-0.6B --dtype float16 --auto-truncate --max-batch-tokens 8192 Querying via curl: curl 127.0.0.1:8080/embed -X POST -d '{"inputs":"1"}' -H 'Content-Type: application/json' Result A (Normalized) Start the gRPC server: Bash docker run --gpus '"device=0"' -d -p 8080:80 \ -v ~/model:/data \ --name tei-grpc \ ghcr.io/huggingface/text-embeddings-inference:turing-1.8-grpc \ --model-id /data/Qwen/Qwen3-Embedding-0.6B --dtype float16 --auto-truncate --max-batch-tokens 8192 Querying via grpcurl (using proto file): grpcurl -d '{"inputs": "1"}' -plaintext -proto tei.proto 0.0.0.0:8080 tei.v1.Embed/Embed Result B (Unnormalized) ### Expected behavior Expected behavior Both interfaces should respect the same default normalization settings (or explicitly document the difference). Currently, Result B is the unnormalized version of Result A. Environment: Model: Qwen3-Embedding-0.6B Image: ghcr.io/huggingface/text-embeddings-inference:turing-1.8-grpc GPU: Turing architecture 2080Ti Additional context I also tried to pass "normalize": true in the gRPC request, but it seems the current Protobuf definition for EmbedRequest does not support this field, making it impossible to request normalization via gRPC if it's not applied by default.
5 条评论