ITADN

Fix GEMMA_2, LLAMA_3, and PHI_3 MessagesFormatter

#73Pull Requestwoheller69 创建于 2024-07-07
W
woheller69commented
Gemma needs another \n in the prompt template to avoid slow processing of follow up prompts, see #54 ![Screenshot from 2024-07-07 19-16-28](https://github.com/Maximilian-Winter/llama-cpp-agent/assets/68678880/8b69d107-839f-409c-a9b8-13d8d19666b9) As can be seen in the attached screenshot the prompt processing is much faster. Before: The new user prompt plus the models previous answer are processed (42 tokens) After: Only the new user prompt is processed (18 tokens) The longer the models previous answer the more relevant... Probably there are similar issues in all other prompt templates.
合并状态:未合并 关闭于 2024-12-10 14 条评论