`Bug` (kind of): ProxLB starts <parallel_jobs> Migrations, but then waits for all of them to finish before starting the next set.

## General ProxLB starts <parallel_jobs> Migrations, but then waits for all of them to finish before starting the next set. Before ProxLB we were using the "bulk migration" feature of PVE to vacate a host, which also can do migrations in parallel but it will run up to N parallel jobs at any given time - when one finishes and there are still VMs left to migrate, it'll just start the next migration. ## Weighting Score: 2-3 ## Config <details> <summary>Config file</summary> ``` proxmox_api: hosts: user: # API Token method token_id: token_secret: ssl_verification: True timeout: 10 proxmox_cluster: maintenance_nodes: ['host01'] ignore_nodes: [] overprovisioning: False balancing: enable: True enforce_affinity: False parallel: True # If running parallel job, you can define # the amount of prallel jobs (default: 5) parallel_jobs: 4 live: True with_local_disks: False # not supported on "older" (?! 8.4.14 is old?) PVE versions with_conntrack_state: False balance_types: ['vm', 'ct'] max_job_validation: 1800 balanciness: 10 method: memory mode: used service: daemon: True schedule: interval: 3 format: hours delay: enable: False time: 1 format: hours log_level: DEBUG ``` </details> ## Log <details> <summary>Shortened, annotated log showing the issue.</summary> ``` Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,001 - ProxLB - DEBUG - Balancing: Balancing for guest vm-1 of type VM started. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,001 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,001 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-1 from host01 to host02. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,078 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,078 - ProxLB - DEBUG - Balancing: Balancing for guest vm-2 of type VM started. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,078 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,078 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-2 from host01 to host02. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,131 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,131 - ProxLB - DEBUG - Balancing: Balancing for guest vm-3 of type VM started. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,131 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,131 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-3 from host01 to host03. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,182 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,183 - ProxLB - DEBUG - Balancing: Balancing for guest vm-4 of type VM started. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,183 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,183 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-4 from host01 to host03. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,239 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,239 - ProxLB - DEBUG - Starting: get_rebalancing_job_status. Jan 14 08:34:36 host-pbs ProxLB[3835]: 2026-01-14 08:34:36,267 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA3:35E44F93:69674712:qmigrate:122:proxlb@pve!proxlb: (guest: vm-1) is a standard migration job. Proceeding > Jan 14 08:34:46 host-pbs ProxLB[3835]: 2026-01-14 08:34:46,268 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA3:35E44F93:69674712:qmigrate:122:proxlb@pve!proxlb: (guest: vm-1) for migration is still running... (Run: > [... after about one minute, 3 migrations continue ...] Jan 14 08:35:06 host-pbs ProxLB[3835]: 2026-01-14 08:35:06,325 - ProxLB - DEBUG - Starting: get_rebalancing_job_status. Jan 14 08:35:06 host-pbs ProxLB[3835]: 2026-01-14 08:35:06,360 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA3:35E44F93:69674712:qmigrate:122:proxlb@pve!proxlb: (guest: vm-1) is a standard migration job. Proceeding > Jan 14 08:35:06 host-pbs ProxLB[3835]: 2026-01-14 08:35:06,360 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA3:35E44F93:69674712:qmigrate:122:proxlb@pve!proxlb: (guest: vm-1) was successfully. Jan 14 08:35:06 host-pbs ProxLB[3835]: 2026-01-14 08:35:06,361 - ProxLB - DEBUG - Finished: get_rebalancing_job_status. Jan 14 08:35:06 host-pbs ProxLB[3835]: 2026-01-14 08:35:06,361 - ProxLB - DEBUG - Starting: get_rebalancing_job_status. Jan 14 08:35:06 host-pbs ProxLB[3835]: 2026-01-14 08:35:06,381 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA5:35E44F99:69674712:qmigrate:121:proxlb@pve!proxlb: (guest: vm-2) is a standard migration job. Proceed> Jan 14 08:35:16 host-pbs ProxLB[3835]: 2026-01-14 08:35:16,385 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA5:35E44F99:69674712:qmigrate:121:proxlb@pve!proxlb: (guest: vm-2) for migration is still running... (R> [... after another minute only 1 migration continues ...] Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,507 - ProxLB - DEBUG - Starting: get_rebalancing_job_status. Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,541 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA5:35E44F99:69674712:qmigrate:121:proxlb@pve!proxlb: (guest: vm-2) is a standard migration job. Proceed> Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,541 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA5:35E44F99:69674712:qmigrate:121:proxlb@pve!proxlb: (guest: vm-2) was successfully. Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,541 - ProxLB - DEBUG - Finished: get_rebalancing_job_status. Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,541 - ProxLB - DEBUG - Starting: get_rebalancing_job_status. Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,569 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA6:35E44F9E:69674712:qmigrate:140:proxlb@pve!proxlb: (guest: vm-3) is a standard migration job. Proceedin> Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,569 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA6:35E44F9E:69674712:qmigrate:140:proxlb@pve!proxlb: (guest: vm-3) was successfully. Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,569 - ProxLB - DEBUG - Finished: get_rebalancing_job_status. Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,569 - ProxLB - DEBUG - Starting: get_rebalancing_job_status. Jan 14 08:35:56 host-pbs ProxLB[3835]: 2026-01-14 08:35:56,593 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA9:35E44FA4:69674712:qmigrate:113:proxlb@pve!proxlb: (guest: vm-4) is a standard migration job. Proceeding> Jan 14 08:36:06 host-pbs ProxLB[3835]: 2026-01-14 08:36:06,593 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA9:35E44FA4:69674712:qmigrate:113:proxlb@pve!proxlb: (guest: vm-4) for migration is still running... (Run:> [... almost 9 minutes later the last one is done ...] Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,079 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA9:35E44FA4:69674712:qmigrate:113:proxlb@pve!proxlb: (guest: vm-4) is a standard migration job. Proceeding> Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,079 - ProxLB - DEBUG - Balancing: Job ID UPID:host01:00164FA9:35E44FA4:69674712:qmigrate:113:proxlb@pve!proxlb: (guest: vm-4) was successfully. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,079 - ProxLB - DEBUG - Finished: get_rebalancing_job_status. [... and only now the next set of <parallel_jobs> VM migrations is started ...] Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,080 - ProxLB - DEBUG - Balancing: Balancing for guest vm-5 of type VM started. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,080 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,080 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-5 from host01 to host02. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,122 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,122 - ProxLB - DEBUG - Balancing: Balancing for guest vm-6 of type VM started. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,122 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,122 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-6 from host01 to host03. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,167 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,167 - ProxLB - DEBUG - Balancing: Balancing for guest vm-7 of type VM started. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,167 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,167 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-7 from host01 to host02. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,216 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,216 - ProxLB - DEBUG - Balancing: Balancing for guest vm-8 of type VM started. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,216 - ProxLB - DEBUG - Starting: exec_rebalancing_vm. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,216 - ProxLB - INFO - Balancing: Starting to migrate VM guest vm-8 from host01 to host02. Jan 14 08:44:58 host-pbs ProxLB[3835]: 2026-01-14 08:44:58,264 - ProxLB - DEBUG - Finished: exec_rebalancing_vm. ``` </details> ## Meta Version: ProxLB version: 1.1.11 Installed from: Repository Running as: dedicated, local on proxmox-backup-server host.