ITADN

[Feature Request] Add Olostep web search and scraping to SearchToolkit

#4065Closedumerkay 创建于 2026-05-22
Needs Triage
U
umerkaycommented
### Required prerequisites - [x] I have searched the [Issue Tracker](https://github.com/camel-ai/camel/issues) and [Discussions](https://github.com/camel-ai/camel/discussions) that this hasn't already been reported. (+1 or comment there if it has.) - [ ] Consider asking first in a [Discussion](https://github.com/camel-ai/camel/discussions/new). ### Motivation ### Required prerequisites - [x] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.) - [x] Consider asking first in a Discussion. ### Motivation The `SearchToolkit` currently supports DuckDuckGo, Brave, Bocha, Baidu, Tavily, Wikipedia, and Google as search backends. This PR adds [Olostep](https://www.olostep.com) as an additional option — covering both **search** and **scraping** in two new methods: **`search_olostep(query, num_results=5)`** Calls `POST https://api.olostep.com/v1/searches` and returns a list of `{url, title, description}` dicts. Drop-in alternative to `search_brave` and `search_bocha`. Requires `OLOSTEP_API_KEY`. **`scrape_olostep(url)`** Uses the official `olostep` Python SDK (`SyncOlostepClient`) to scrape a URL and return clean Markdown content. Drop-in alternative to web fetch tools. Requires `OLOSTEP_API_KEY`. Usage after the change: ```python from camel.toolkits import SearchToolkit toolkit = SearchToolkit() # Web search results = toolkit.search_olostep("latest AI agent papers", num_results=5) # [{"url": "...", "title": "...", "description": "..."}, ...] # URL scraping page = toolkit.scrape_olostep("https://arxiv.org/abs/2303.17760") # {"url": "...", "markdown": "# CAMEL: ..."} # As agent tools tools = [ toolkit.search_olostep, toolkit.scrape_olostep, ] ``` ### Implementation plan - Add `search_olostep` method to `camel/toolkits/search_toolkit.py` - Calls `/v1/searches` via `requests.post` directly (the Olostep Python SDK has no `searches` namespace) - Decorated with `@api_keys_required([(None, 'OLOSTEP_API_KEY')])` - Returns `List[Dict[str, Any]]` - Add `scrape_olostep` method to `camel/toolkits/search_toolkit.py` - Uses `SyncOlostepClient` from the `olostep` pip package - Imports inside the method so `olostep` is an optional dependency - Returns `Dict[str, Any]` with `url` and `markdown` keys - Register both in `get_tools()` - Add unit tests with mocked HTTP/SDK calls ### No breaking changes All existing methods untouched. Both new methods are opt-in and only activate when `OLOSTEP_API_KEY` is set. ### References - Olostep docs: https://docs.olostep.com - Similar existing method: `search_bocha`, `search_brave` - I am a developer at Olostep and will maintain this integration. ### Solution _No response_ ### Alternatives _No response_ ### Additional context _No response_
关闭于 2026-05-25 1 条评论