ITADN

Update `llama_cpp_server.py` fixing bugs with non-streaming response

#85Pull Requestorangethewell 创建于 2024-12-25已合并
O
orangethewellcommented
This commit fixes the following bugs: - `max_tokens` as `-1` return a error from python server, set it a default value as 8192 tokens for maximum capacity, but tests can be done with this; - `data["content"]` doesn't exist in llama-cpp-python server for response, better use the entire response json data because it's the same structure used by extractor.
合并状态:已合并 合并于 2025-02-17 关闭于 2025-02-17 1 条评论