server: merge tokens split across UTF-8 boundaries in JSON output
The fix in #3751 (PR #3751) added UTF-8 boundary merging for `examples/cli/cli.cpp` to prevent CJK/multi-byte characters that span whisper token boundaries from producing invalid UTF-8 in JSON output.
**The same bug exists in `examples/server/server.cpp`** at line ~1090:
```cpp
json word = json{{"word", whisper_full_get_token_text(ctx, i, j)}};
```
The server's JSON `"words"` array emits per-token text via `whisper_full_get_token_text()` without merging split multi-byte sequences. When a CJK character (3 bytes) lands across two adjacent tokens, the server will emit partial byte sequences as separate word entries — producing invalid UTF-8 in the API response.
The fix should apply the same `utf8_trailing_bytes_needed()` merge logic from cli.cpp to the server's word emission loop.
Found during whole-codebase audit of #3751.
0 条评论