Bug/semaphore lock
<!--
Congratulations! You've made it this far! Thanks for submitting a PR to Infinity!
## License & CLA
By submitting this PR, I confirm that my contribution is made under the terms of the [MIT license](https://github.com/michaelfeil/infinity/blob/main/LICENSE).
-->
## Related Issue
<!--
If you are sure that a related issue should be automatically closed, use the syntax to close issue 123.
closes #123
-->
## Checklist
- [x] I have read the [CONTRIBUTING](https://github.com/michaelfeil/infinity/tree/main?tab=readme-ov-file#contribute-and-develop) guidelines.
- [ ] I have added tests to cover my changes.
- [x] I have updated the documentation (docs folder) accordingly.
## Additional Notes
Hi!
When running multiple jobs asynchronously, a semaphore lock occurs. Multiple jobs will increment the semaphore counter to >3, and after the first job completes, the lock will remain.
I suggest using a mutex, so all new jobs will simply wait for it to be released and then proceed quietly.
Example code with locking:
```python
import uuid
import asyncio
from infinity_emb import AsyncEmbeddingEngine, EngineArgs
async def run_job_a(engine: AsyncEmbeddingEngine, texts: list[str]) -> list[list[float]]:
task_id = uuid.uuid4()
print(f"{task_id=} started")
async with engine:
result, usage = await engine.embed(sentences=texts)
print(f"{task_id=} completed")
return result
async def main() -> None:
args = EngineArgs(
model_name_or_path="/home/models/e5-small-v2",
)
engine = AsyncEmbeddingEngine.from_args(engine_args=args)
outputs = await asyncio.gather(
run_job_a(engine=engine, texts=["a", "b", "c", "d"]),
run_job_a(engine=engine, texts=["a", "b", "c", "d"]),
run_job_a(engine=engine, texts=["a", "b", "c", "d"])
)
if __name__ == "__main__":
asyncio.run(main())
```
I understand that the behavior with constantly turning on/off the engine is incorrect, but suddenly someone wants to do this)
合并状态:未合并 5 条评论