Update `llama_cpp_server.py` fixing bugs with non-streaming response
This commit fixes the following bugs:
- `max_tokens` as `-1` return a error from python server, set it a default value as 8192 tokens for maximum capacity, but tests can be done with this;
- `data["content"]` doesn't exist in llama-cpp-python server for response, better use the entire response json data because it's the same structure used by extractor.
合并状态:已合并 合并于 2025-02-17 关闭于 2025-02-17 1 条评论