ITADN

rattler.package_streaming efficiency

#2252OpenJeanChristopheMorinPerso 创建于 2026-03-19
### Checklist - [x] I added a descriptive title - [x] I searched open requests and couldn't find a duplicate ### What is the idea? Hello, I was looking at the new `rattler.package_streaming` module and it looks great. While testing it, I noticed that it differs from `conda-package-streaming` by not allowing to retrieve multiple files from the same stream. In conda-package-streaming, we can do stuff like ```python import contextlib from conda_package_streaming.url import conda_reader_for_url from conda_package_streaming.package_streaming import stream_conda_component filename, conda = conda_reader_for_url(url) data = {} with contextlib.closing(conda): for tar, member in stream_conda_component(filename, conda, component="pkg"): if member.name.endswith(".so"): data[member.name] = tar.extractfile(member).read() ``` which allows us to read multiple files from within with a single connection and I believe fewer HTTP calls (though I have no verified this). With rattler, I see the `fetch_raw_package_file_from_url` function and from what I see it works. But from what I see, it seems a little bit slower (which is hard to explain). And it is definitely slower if I try the equivalent, which look like: ```python import json import rattler import rattler.package_streaming client = rattler.Client.default_client() paths = json.loads(await rattler.package_streaming.fetch_raw_package_file_from_url(client, url, "info/paths.json")) data = {} for entry in paths: if entry["_path"].endswith(".so"): data[entry["_path"]] = await rattler.package_streaming.fetch_raw_package_file_from_url(client, url, entry["_path"]) ``` Should rattler provide a similar API to conda-package-streaming? ### Why is this needed? I think it could be more efficient to have an API like conda-package-streaming provides. But I have not fully verified this yet. ### What should happen? _No response_ ### Additional Context _No response_
2 条评论