Inconsistency between HTTP and gRPC output normalization (Qwen3-Embedding)
### System Info
I noticed a significant inconsistency in the output embeddings between the HTTP and gRPC interfaces when using the same model and start-up parameters. The HTTP /embed endpoint returns L2-normalized vectors by default, whereas the gRPC tei.v1.Embed/Embed method returns raw, unnormalized vectors.
### Information
- [x] Docker
- [ ] The CLI directly
### Tasks
- [x] An officially supported command
- [ ] My own modifications
### Reproduction
Start the HTTP server:
Bash
docker run --gpus '"device=0"' -d -p 8080:80 \
-v ~/model:/data \
--name tei-http \
ghcr.io/huggingface/text-embeddings-inference:turing-1.8 \
--model-id /data/Qwen/Qwen3-Embedding-0.6B --dtype float16 --auto-truncate --max-batch-tokens 8192
Querying via curl: curl 127.0.0.1:8080/embed -X POST -d '{"inputs":"1"}' -H 'Content-Type: application/json' Result A (Normalized)
Start the gRPC server:
Bash
docker run --gpus '"device=0"' -d -p 8080:80 \
-v ~/model:/data \
--name tei-grpc \
ghcr.io/huggingface/text-embeddings-inference:turing-1.8-grpc \
--model-id /data/Qwen/Qwen3-Embedding-0.6B --dtype float16 --auto-truncate --max-batch-tokens 8192
Querying via grpcurl (using proto file): grpcurl -d '{"inputs": "1"}' -plaintext -proto tei.proto 0.0.0.0:8080 tei.v1.Embed/Embed Result B (Unnormalized)
### Expected behavior
Expected behavior Both interfaces should respect the same default normalization settings (or explicitly document the difference). Currently, Result B is the unnormalized version of Result A.
Environment:
Model: Qwen3-Embedding-0.6B
Image: ghcr.io/huggingface/text-embeddings-inference:turing-1.8-grpc
GPU: Turing architecture 2080Ti
Additional context I also tried to pass "normalize": true in the gRPC request, but it seems the current Protobuf definition for EmbedRequest does not support this field, making it impossible to request normalization via gRPC if it's not applied by default.
5 条评论