Malt parallelization
enhancement
### Description of feature
Hi,
I am a bioinformatics student currently using nf-core/taxprofiler to profile fungal diversity in metagenomes. I have ~800 samples (one metagenome per sample), paired-end short reads, up to 200M reads per sample. I am using Kraken2, KrakenUniq and MALT.
I have trouble parallelizing MALT, as taxprofiler seems to force a single MALT job for all samples, unlike KrakenUniq where batch size can be defined (default = 20). At this scale, this becomes limiting.
If I understand correctly, one of the issues with MALT is that database loading can be resource-intensive.
I found two possible ways to work around this:
Splitting the samples into sublists and launching multiple taxprofiler runs (but it would mean launching TaxPasta manually).
Modifying the source code.
I am not very comfortable with the second option. From a discussion with DeepWiki, this would involve removing .groupTuple(by: [0, 2]) in profiling.nf (around line 130), replacing
def new_meta = db_meta + [id: db_meta.db_name]
with
def new_meta = meta + db_meta
(around line 120), and modifying maxForks in the config file.
I am therefore writing here to ask for advice, perhaps I have missed existing parameters?
Thank you.
关闭于 2026-03-09 7 条评论