ITADN

deploy local avian-flu with "nextstrain build . --configfile config/gisaid.yaml -f deploy_all" failed

#143ClosedNailouZhang 创建于 2025-02-26
bug
N
NailouZhangcommented
### Current Behavior When I want to deploy local avian-flu with `nextstrain build . --configfile config/gisaid.yaml -f deploy_all`, but some files seemed can't be download, the log as fellow: ### How to reproduce Steps to reproduce the current behavior: ```console $ nextstrain build . --configfile config/gisaid.yaml -f deploy_all Building DAG of jobs... Using shell: /bin/bash Provided cores: 48 Rules claiming more threads will be scaled down. Job counts: count jobs 2 add_h5_clade 48 align 48 ancestral 6 cleavage_site 1 deploy_all 1 download_s3_metadata 8 download_s3_sequences 48 export 48 filter 4 filter_metadata_by_subtype 32 filter_sequences_by_subtype 48 refine 48 rename_auspice_datasets 48 traits 48 translate 48 tree 486 [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_pb2.fasta jobid: 571 wildcards: input_name=gisaid, segment=pb2 aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/pb2/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_pb2.fasta [Wed Feb 26 04:57:42 2025] rule download_s3_metadata: output: data/gisaid/metadata.tsv jobid: 572 wildcards: input_name=gisaid aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/metadata.tsv.zst - | zstd -d > data/gisaid/metadata.tsv [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_na.fasta jobid: 578 wildcards: input_name=gisaid, segment=na aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/na/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_na.fasta [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_ha.fasta jobid: 576 wildcards: input_name=gisaid, segment=ha aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/ha/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_ha.fasta [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_ns.fasta jobid: 580 wildcards: input_name=gisaid, segment=ns aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/ns/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_ns.fasta [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_pa.fasta jobid: 575 wildcards: input_name=gisaid, segment=pa aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/pa/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_pa.fasta [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_pb1.fasta jobid: 574 wildcards: input_name=gisaid, segment=pb1 aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/pb1/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_pb1.fasta [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_np.fasta jobid: 577 wildcards: input_name=gisaid, segment=np aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/np/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_np.fasta [Wed Feb 26 04:57:42 2025] rule download_s3_sequences: output: data/gisaid/sequences_mp.fasta jobid: 579 wildcards: input_name=gisaid, segment=mp aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/mp/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_mp.fasta download failed: s3://nextstrain-data-private/files/workflows/avian-flu/pa/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/mp/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/ha/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/ns/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/pb2/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/na/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/pb1/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/np/sequences.fasta.zst to - Unable to locate credentials download failed: s3://nextstrain-data-private/files/workflows/avian-flu/metadata.tsv.zst to - Unable to locate credentials zstd: /*stdin*\: unexpected end of file zstd: /*stdin*\: unexpected end of file zstd: /*stdin*\: unexpected end of file [Wed Feb 26 04:57:50 2025] Error in rule download_s3_sequences: jobid: 580 [Wed Feb 26 04:57:50 2025] [Wed Feb 26 04:57:50 2025] output: data/gisaid/sequences_ns.fasta Error in rule download_s3_sequences: jobid: 577 shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/ns/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_ns.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_ns.fasta output: data/gisaid/sequences_np.fasta shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/np/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_np.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_np.fasta Error in rule download_s3_sequences: jobid: 571 output: data/gisaid/sequences_pb2.fasta shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/pb2/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_pb2.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_pb2.fasta zstd: /*stdin*\: unexpected end of file [Wed Feb 26 04:57:50 2025] Error in rule download_s3_sequences: jobid: 575 output: data/gisaid/sequences_pa.fasta shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/pa/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_pa.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_pa.fasta zstd: /*stdin*\: unexpected end of file zstd: /*stdin*\: unexpected end of file [Wed Feb 26 04:57:50 2025] [Wed Feb 26 04:57:50 2025] Error in rule download_s3_sequences: Error in rule download_s3_sequences: jobid: 576 jobid: 579 output: data/gisaid/sequences_ha.fasta shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/ha/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_ha.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_ha.fasta output: data/gisaid/sequences_mp.fasta shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/mp/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_mp.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_mp.fasta zstd: /*stdin*\: unexpected end of file zstd: /*stdin*\: unexpected end of file zstd: /*stdin*\: unexpected end of file [Wed Feb 26 04:57:50 2025] [Wed Feb 26 04:57:50 2025] Error in rule download_s3_sequences: Error in rule download_s3_metadata: jobid: 578 jobid: 572 output: data/gisaid/sequences_na.fasta [Wed Feb 26 04:57:50 2025] Error in rule download_s3_sequences: jobid: 574 output: data/gisaid/sequences_pb1.fasta shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/pb1/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_pb1.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) output: data/gisaid/metadata.tsv shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/metadata.tsv.zst - | zstd -d > data/gisaid/metadata.tsv (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_pb1.fasta shell: aws s3 cp s3://nextstrain-data-private/files/workflows/avian-flu/na/sequences.fasta.zst - | zstd -d > data/gisaid/sequences_na.fasta (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!) Removing output files of failed job download_s3_sequences since they might be corrupted: data/gisaid/sequences_na.fasta Removing output files of failed job download_s3_metadata since they might be corrupted: data/gisaid/metadata.tsv Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message Complete log: /nextstrain/build/.snakemake/log/2025-02-26T045741.789585.snakemake.log ``` ### Possible solution (optional) ### Your environment: if browsing Nextstrain online - Operating system: - Browser: ### Your environment: if running Nextstrain locally - Operating system: ubuntu 16.04 - Browser: Chrome - Version (e.g. `auspice 2.59.1`):
关闭于 2025-02-27 1 条评论