Limitation with registering 70+ periodic tasks
My team and I are using `procrastinate` to manage workflows centrally to be consumed by a fleet of workers running on EC2 instances.
As we've migrated all of our cron jobs over to `@app.periodic` we've started noticing that while all of our tasks are getting **registered** (as confirmed by the logs `Registering task <task_name>`), a variety of them never get **deferred**.
I have a task deployed to production that's cron schedule defines it to run once a day, at 1 AM. However checking the logs every morning, I never saw that task get deferred.
Running a few experiments locally confirmed that the number of periodic tasks is likely the issue here. After commenting out every other `@app.periodic` decorator, and changing the cron expression to run 2 minutes after I started my local worker, I saw that it was immediately deferred at the scheduled time.
Can you look into this scaling issue with multiple periodic tasks? Happy to provide more information if needed.
Documentation Referenced: [Launch a task periodically](https://procrastinate.readthedocs.io/en/stable/howto/advanced/cron.html#timestamp-argument)
0 条评论