[log] germline: WGS + AWS: g4dn.xlarge (T4 + 4vCPU + 16GB)

[log] germline: WGS + AWS: g4dn.xlarge (T4 + 4vCPU + 16GB) === ###### tags: `Parabricks-v3.8` ###### tags: `基因體`, `NVIDIA`, `Clara`, `Parabricks`, `二級分析` :::danger :no_entry_sign: 此規格 **g4dn.xlarge (T4 + 4vCPU + 16GB)**，跑不起來 ::: [TOC] ## 硬體資源：EC2 / g4dn.xlarge > 查閱方式：[[HackMD] AWS / 容器執行個體(EC2) / EC2 / g4dn.xlarge](https://hackmd.io/QavypnMBRoW8P9aH4gw8FA#EC2--g4dnxlarge) [![](https://i.imgur.com/N6fauHL.png)](https://i.imgur.com/N6fauHL.png) ### GPU - GPU 0: Tesla T4 (15360MiB) ### CPU - Intel® Xeon® Platinum 8259CL CPU @ 2.50GHz - 1(插槽) x 2(核/插槽) x 2(超執行緒/核) = 4條執行緒 (4vCPU) ### RAM - 16GB <hr> ## script ```bash= # cmd: sh germline.sh 2>&1 | tee germline.sh.log & date && start_time=`date +%s` && echo "start_time:" $start_time pbrun germline \ --ref parabricks_sample/Ref/Homo_sapiens_assembly38.fasta \ --in-fq \ datasets/WGS-LIS-AI018A_R1.fastq.gz \ datasets/WGS-LIS-AI018A_R2.fastq.gz \ --knownSites parabricks_sample/Ref/Homo_sapiens_assembly38.known_indels.vcf.gz \ --out-bam output/output.bam \ --out-variants output/output.vcf \ --out-recal-file output/report.txt; \ echo "exit code:" $!; \ echo "Done!" date && end_time=`date +%s` && echo "end_time:" $end_time total=$((end_time - start_time)) time_h=$((total/3600)) time_m=$(( (total - time_h*3600) / 60 )) time_s=$(( total - time_h*3600 - time_m*60 )) echo "Total time: ${time_h}h ${time_m}m ${time_s}s" ``` <hr> ## Log > **g4dn.xlarge (T4 + 4vCPU + 16GB)** ``` $ Thu Mar 17 02:40:15 UTC 2022 start_time: 1647484815 Please visit https://docs.nvidia.com/clara/#parabricks for detailed documentation [Parabricks Options Mesg]: Automatically generating ID prefix [Parabricks Options Mesg]: Read group created for /workspace/datasets/WGS-LIS-AI018A_R1.fastq.gz and /workspace/datasets/WGS-LIS-AI018A_R2.fastq.gz [Parabricks Options Mesg]: @RG\tID:H3WFJDSXX.1\tLB:lib1\tPL:bar\tSM:sample\tPU:H3WFJDSXX.1 [Parabricks Options Mesg]: Checking argument compatibility [Parabricks Options Mesg]: Read group created for /workspace/datasets/WGS-LIS-AI018A_R1.fastq.gz and /workspace/datasets/WGS-LIS-AI018A_R2.fastq.gz [Parabricks Options Mesg]: @RG\tID:H3WFJDSXX.1\tLB:lib1\tPL:bar\tSM:sample\tPU:H3WFJDSXX.1 [PB Info 2022-Mar-17 02:40:15] Logger not initialized! [PB Info 2022-Mar-17 02:40:15] ------------------------------------------------------------------------------ [PB Info 2022-Mar-17 02:40:15] || Parabricks accelerated Genomics Pipeline || [PB Info 2022-Mar-17 02:40:15] || Version 3.7.0-1 || [PB Info 2022-Mar-17 02:40:15] || GPU-BWA mem, Sorting Phase-I || [PB Info 2022-Mar-17 02:40:15] || Contact: Parabricks-Support@nvidia.com || [PB Info 2022-Mar-17 02:40:15] ------------------------------------------------------------------------------ gpg: keyblock resource '/home/ubuntu/.gnupg/pubring.kbx': Permission denied [PB Info 2022-Mar-17 02:40:16] Logger already initialized, continuing with current settings. [M::bwa_idx_load_from_disk] read 0 ALT contigs [PB Warning 2022-Mar-17 02:41:00][ParaBricks/src/pbOpts.cu:316] WARNING The system has 15 GB, however recommended RAM with 1 GPU is 64 GB. The run might not finish or might have less than expected performance. [PB Warning 2022-Mar-17 02:41:00][ParaBricks/src/pbOpts.cu:325] WARNING The system has 4 threads, however recommended number of threads with 1 GPU is 16. The run might not finish or might have less than expected performance. [PB Info 2022-Mar-17 02:41:00] GPU-BWA mem [PB Info 2022-Mar-17 02:41:00] ProgressMeter Reads Base Pairs Aligned [PB Info 2022-Mar-17 02:43:34] 5033176 760000000 Killed For technical support visit https://docs.nvidia.com/clara/parabricks/3.7.0/index.html#how-to-get-help Exiting... Could not run fq2bam as part of germline pipeline Exiting pbrun ... Thu Mar 17 02:44:56 UTC 2022 end_time: 1647485096 Total time: 0h 4m 41s ``` :::warning :warning: **WARNING** The system has 15 GB, however recommended RAM with 1 GPU is 64 GB. The run might not finish or might have less than expected performance. ::: :::warning :warning: **WARNING** The system has 4 threads, however recommended number of threads with 1 GPU is 16. The run might not finish or might have less than expected performance. ::: ## 出現過的 exit code ``` Killed For technical support visit https://docs.nvidia.com/clara/parabricks/3.7.0/index.html#how-to-get-help Exiting... Could not run fq2bam as part of germline pipeline Exiting pbrun ... exit code: 2265 ``` ``` Killed For technical support visit https://docs.nvidia.com/clara/parabricks/3.7.0/index.html#how-to-get-help Exiting... Could not run fq2bam as part of germline pipeline Exiting pbrun ... exit code: 3479 ``` Ctrl+C 中斷： ``` ^CExiting...For technical support visit https://docs.nvidia.com/clara/parabricks/3.7.0/index.html#how-to-get-help [PB Error 2022-Mar-17 04:38:17][-unknown-:0] Received signal: 2 For technical support visit https://docs.nvidia.com/clara/parabricks/3.7.0/index.html#how-to-get-help[PB Error 2022-Mar-17 04:38:18][-unknown-:0] [PB Error 2022-Mar-17 04:38:20][-unknown-:0] Received signal: 2 For technical support visit https://docs.nvidia.com/clara/parabricks/3.7.0/index.html#how-to-get-helpReceived signal: 2 For technical support visit https://docs.nvidia.com/clara/parabricks/3.7.0/index.html#how-to-get-help, exiting. , exiting. , exiting. Traceback (most recent call last): File "/usr/local/parabricks/run_pipeline.py", line 7, in <module> sys.exit(PB.pb_main()) File "PB.pyx", line 2169, in PB.pb_main File "PB.pyx", line 394, in PB.runfq2bam File "PB.pyx", line 131, in PB.RunBinary File "/usr/lib/python3.7/subprocess.py", line 341, in call return p.wait(timeout=timeout) File "/usr/lib/python3.7/subprocess.py", line 1019, in wait return self._wait(timeout=timeout) File "/usr/lib/python3.7/subprocess.py", line 1653, in _wait (pid, sts) = self._try_wait(0) File "/usr/lib/python3.7/subprocess.py", line 1611, in _try_wait (pid, sts) = os.waitpid(self.pid, wait_flags) File "PB.pyx", line 54, in PB.signal_handler File "PB.pyx", line 50, in PB.exitFunction File "PB.pyx", line 54, in PB.signal_handler File "PB.pyx", line 49, in PB.exitFunction File "PB.pyx", line 54, in PB.signal_handler File "PB.pyx", line 49, in PB.exitFunction RuntimeError: reentrant call inside <_io.BufferedWriter name='<stdout>'> Exiting pbrun ... exit code: 3479 ```