[Feature] monty support complimentary to Pyodide
enhancement
### What feature would you like to see?
`ProgramOfThought` is an extremely useful module. In many cases, a full Python interpreter makes it possible to run math operations and chain tool calls as code.
However, the main drawback of this module when serving many user requests is its speed. Pyodide is a complete but heavy interpreter implementation. The original Pyodide authors were willing to accept a startup time of nearly 3s (https://github.com/pydantic/monty#alternatives) because they assumed users would keep the interpreter running for a long time. On top of that, DSPy adds extra overhead by starting Deno for WebAssembly.
One important observation, especially based on my anecdotal experience, is that Python is usually used inside `ProgramOfThought` for math or API calls. Because of that, I suspect DSPy could benefit significantly from a lightweight, fast, even somewhat limited interpreter. I’d like to suggest an alternative interpreter for `ProgramOfThought`: monty (https://github.com/pydantic/monty).
If this feature request sounds useful to you, I’d be happy to add support for it.
### Would you like to contribute?
- [x] Yes, I'd like to help implement this.
- [ ] No, I just want to request it.
### Additional Context
_No response_
0 条评论