Duplicate Storage of Labeled Data During DPGEN2 iteration
I encountered the following issue while running DPGEN2. I used the `dpgen2 download input.json WFID -d collect-data/output/iter_data` code to download the generated training data, but I noticed that there are some duplicate data instances. For example, as highlighted in the red section of the diagram, under the same number of atoms, there are two set.00* folders, and the structural and energy information inside them is completely identical. Additionally, these duplicate data are also being used in subsequent training processes.
<img width="840" height="245" alt="Image" src="https://github.com/user-attachments/assets/78b22d13-8780-4dac-bfcf-af78f0bc45d4" />
Is there a way to avoid this issue?
关闭于 2025-09-08 17 条评论