ITADN

parallel problem on SLURM system

#7Closedxdzhu 创建于 2023-12-03
X
xdzhucommented
Recently I found that there have been many updates, salute to the developer! I have been trying to perform NEB calculations using ATST-Tools v1.2.2 on the **SLURM job system**. This is my first attempt at parallel running since my **previous try on the PBS system failed**. When attempting to use 4 nodes (32 cores per node) and invoke GPAW for parallel computing, I found that although 4 directories were successfully created, the abacus program did not start on each node (**no abacus process was found when SSH to each node, and no output in abacus.out**). Then, when submitting with a single node, although it ran successfully, similar errors were reported. May I ask what might be the cause of this, and how can I successfully initiate image parallelization in my case? Thank you very much! 最近又看到更新了不少,给肝帝点赞!我使用1.2.2版本尝试在slurm作业系统上使用ATST-Tools进行NEB计算。在我尝试使用4节点(32核/节点)并调用GPAW进行并行计算时,发现可以成功生成4个文件夹,但在每个node上并没有成功跑起来abacus(ssh到每个节点上发现没有abacus进程,abacus.out无任何输出)。然后,使用单节点提交时,虽然成功运行,但也报有类似错误。请问这是什么原因d导致,在我这种情况下如何能够成功开启image并行?非常感谢! Calculation files: [neb-slurm-4node.zip](https://github.com/QuantumMisaka/ATST-Tools/files/13539066/neb-slurm-4node.zip) Environment: intel onepi 2021.3 gnu 11.2.1 ase 3.23b1 gpaw info: ![image](https://github.com/QuantumMisaka/ATST-Tools/assets/55775434/9730043b-b36a-4926-9422-850fd04b1ee0) 4 node running_neb.err: > srun: Job 9892350 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892350 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892350 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892350 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892350 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892350 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892350 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892350 step creation still disabled, retrying (Requested nodes are busy) 1 node running_neb.err: > srun: Job 9892355 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation temporarily disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Step created for job 9892355 > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Job 9892355 step creation still disabled, retrying (Requested nodes are busy) > srun: Step created for job 9892355
关闭于 2024-01-06 4 条评论