[log] germline: WGS + NHRI: A100x4 === ###### tags: `Parabricks-v3.8` ###### tags: `基因體`, `NVIDIA`, `Clara`, `Parabricks`, `二級分析` <br> [TOC] <br> ## 硬體資源 ### Flavor - ### [246900] a100-parabricks-4v - **GPU:** a100-40gb x 4張 :::spoiler `$ nvidia-smi -L` ```= GPU 0: NVIDIA A100-SXM4-40GB (UUID: GPU-835407b2-058f-0ff9-938e-fca462f5beb7) GPU 1: NVIDIA A100-SXM4-40GB (UUID: GPU-fd747277-363e-0e90-1866-9512d2cc265e) GPU 2: NVIDIA A100-SXM4-40GB (UUID: GPU-48e6767e-5165-8dea-010c-56c016a9d6ad) GPU 3: NVIDIA A100-SXM4-40GB (UUID: GPU-c4595343-9ada-e53e-0032-ac1a3ec4ad4c) ``` ::: :::spoiler `$ lspci | grep -i nvida` ```= 07:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 0f:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 47:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 4e:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 87:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) 90:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) b7:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) bd:00.0 3D controller: NVIDIA Corporation Device 20b0 (rev a1) c4:00.0 Bridge: NVIDIA Corporation Device 1af1 (rev a1) c5:00.0 Bridge: NVIDIA Corporation Device 1af1 (rev a1) c6:00.0 Bridge: NVIDIA Corporation Device 1af1 (rev a1) c7:00.0 Bridge: NVIDIA Corporation Device 1af1 (rev a1) c8:00.0 Bridge: NVIDIA Corporation Device 1af1 (rev a1) c9:00.0 Bridge: NVIDIA Corporation Device 1af1 (rev a1) ``` ::: - **CPU:** 126 cores [全：256 = 2插槽 x 64(核/插槽) x 2(超執行緒/核)] :::spoiler `$ lscpu` ```= Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 256 On-line CPU(s) list: 0-255 Thread(s) per core: 2 Core(s) per socket: 64 Socket(s): 2 NUMA node(s): 8 Vendor ID: AuthenticAMD CPU family: 23 Model: 49 Model name: AMD EPYC 7742 64-Core Processor Stepping: 0 CPU MHz: 3392.706 CPU max MHz: 2250.0000 CPU min MHz: 1500.0000 BogoMIPS: 4491.41 Virtualization: AMD-V L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 16384K NUMA node0 CPU(s): 0-15,128-143 NUMA node1 CPU(s): 16-31,144-159 NUMA node2 CPU(s): 32-47,160-175 NUMA node3 CPU(s): 48-63,176-191 NUMA node4 CPU(s): 64-79,192-207 NUMA node5 CPU(s): 80-95,208-223 NUMA node6 CPU(s): 96-111,224-239 NUMA node7 CPU(s): 112-127,240-255 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate sme ssbd mba sev ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca ``` ::: - **RAM:** 480 GiB [全：1 TB] :::spoiler `$ lsmem` ```= RANGE SIZE STATE REMOVABLE BLOCK 0x0000000000000000-0x00000000afffffff 2.8G online yes 0-21 0x0000000100000000-0x000000e04fffffff 893.3G online yes 32-7177 0x0000010000000000-0x0000011fffffffff 128G online yes 8192-9215 Memory block size: 128M Total online memory: 1T Total offline memory: 0B ``` ::: :::spoiler `$ cat /proc/meminfo` ```= MemTotal: 1056634792 kB MemFree: 370156980 kB MemAvailable: 1021699836 kB Buffers: 6302856 kB Cached: 631340356 kB SwapCached: 0 kB Active: 124370696 kB Inactive: 525632312 kB Active(anon): 12042852 kB Inactive(anon): 70352 kB Active(file): 112327844 kB Inactive(file): 525561960 kB Unevictable: 19228 kB Mlocked: 19228 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 519576 kB Writeback: 121432 kB AnonPages: 12347456 kB Mapped: 4525364 kB Shmem: 134200 kB KReclaimable: 19075040 kB Slab: 25545236 kB SReclaimable: 19075040 kB SUnreclaim: 6470196 kB KernelStack: 92496 kB PageTables: 68424 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 528317396 kB Committed_AS: 49159716 kB VmallocTotal: 34359738367 kB VmallocUsed: 1672904 kB VmallocChunk: 0 kB Percpu: 1764352 kB HardwareCorrupted: 0 kB AnonHugePages: 47104 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB FileHugePages: 0 kB FilePmdMapped: 0 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 12491992 kB DirectMap2M: 232714240 kB DirectMap1G: 828375040 kB ``` ::: - **Network:** 100Gb ### Storage :::spoiler `$ fio` ``` $ fio \ --randrepeat=1 \ --ioengine=libaio \ --direct=1 \ --gtod_reduce=1 \ --name=fiotest \ --filename=testfio \ --bs=4k \ --iodepth=64 \ --size=8G \ --readwrite=randrw \ --rwmixread=75 ``` ::: - ### 實際效能測試1 | | **速度** | **IOPS** | |---|---------|----------| | **Read** | 144 MiB/s (151MB/s) | 36.9k | | **Write** | 48.1MiB/s (50.4MB/s) | 12.3k | - ### 實際效能測試2 (不同時間點) | | **速度** | **IOPS** | |---|---------|----------| | **Read** | 142MiB/s (149MB/s) | 36.4k | | **Write** | 47.5MiB/s (49.8MB/s) | 12.1k | <br> <hr> <br> ## 效能評估 | Programs | Round 1 | Round 2 | Round 3 | Round 4 | |----------|--------:|--------:|--------:|--------:| | GPU-BWA mem, Sorting Phase-I | 23m 07s | 23m 23s | 23m 19s | m s | | Sorting Phase-II | 1m 40s | 8m 10s | 9m 40s | m s | | Marking Dmuplicates, BQSR | 4m 51s | 17m 12s | 16m 51s | m s | | GPU-GATK4 HaplotypeCaller | 14m 03s | 15m 54s | 126m 54s | | | | | | **Total** | 43m 41s | 64m 39s | 2h 56m 44s | | **Actual** | 44m 20s | 65m 12s | 2h 57m 23s | - [[繁中] 官方效能](https://www.nvidia.com/zh-tw/clara/genomics/) [![](https://i.imgur.com/h5qNcsf.png)](https://i.imgur.com/h5qNcsf.png) - [Accelerated Whole Genome Sequencing Analysis](https://www.robusthpc.com/nvidia-clara-parabricks/) ![](https://i.imgur.com/0oplNc1.png) <br> <hr> <br> ## log :::spoiler 點此查看完整 log ```= # sh datasets/germline/WGS-LIS-AI018A_germline_pipeline.sh Tue May 24 18:02:55 CST 2022 start_time: 1653386575 Please visit https://docs.nvidia.com/clara/#parabricks for detailed documentation [Parabricks Options Mesg]: Automatically generating ID prefix [Parabricks Options Mesg]: Read group created for /workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz and /workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz [Parabricks Options Mesg]: @RG\tID:H3WFJDSXX.1\tLB:lib1\tPL:bar\tSM:sample\tPU:H3WFJDSXX.1 [Parabricks Options Mesg]: Checking argument compatibility [Parabricks Options Mesg]: Read group created for /workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz and /workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz [Parabricks Options Mesg]: @RG\tID:H3WFJDSXX.1\tLB:lib1\tPL:bar\tSM:sample\tPU:H3WFJDSXX.1 Namespace(append=None, bwa_options=None, fix_mate=None, fq_list=None, gpu_devices='all', in_fq=[['/workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz', '/workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz', '@RG\\tID:H3WFJDSXX.1\\tLB:lib1\\tPL:bar\\tSM:sample\\tPU:H3WFJDSXX.1']], in_fq_list=None, in_se_fq=None, interval=None, interval_file=None, interval_padding=None, keep_tmp=None, knownSites=['/workspace/datasets/ref/Homo_sapiens_assembly38.known_indels.vcf.gz'], license_file=None, logfile=None, markdups_assume_sortorder_queryname=None, markdups_picard_version_2182=None, memory_limit=503, no_markdups=None, no_seccomp_override=None, no_warnings=None, num_cpu_threads=0, num_gpus=4, optical_duplicate_pixel_distance=None, out_bam='/workspace/output/output.bam', out_duplicate_metrics=None, out_qc_metrics_dir=None, out_recal_file='/workspace/output/report.txt', read_group_id_prefix=None, read_group_lb=None, read_group_pl=None, read_group_sm=None, ref='/workspace/datasets/ref/Homo_sapiens_assembly38.fasta', standalone_bqsr=None, tmp_dir='/workspace/ZU287CHD', verbose=None, version=None, with_petagene_dir=None, x1=None, x2=None, x3=None, x4=None, x5=None) [PB Info 2022-May-24 18:02:56] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:02:56] || Parabricks accelerated Genomics Pipeline || [PB Info 2022-May-24 18:02:56] || Version 3.8.0-1.ampere || [PB Info 2022-May-24 18:02:56] || GPU-BWA mem, Sorting Phase-I || [PB Info 2022-May-24 18:02:56] || Contact: Parabricks-Support@nvidia.com || [PB Info 2022-May-24 18:02:56] ------------------------------------------------------------------------------ [M::bwa_idx_load_from_disk] read 0 ALT contigs [PB Info 2022-May-24 18:03:02] GPU-BWA mem [PB Info 2022-May-24 18:03:02] ProgressMeter Reads Base Pairs Aligned [PB Info 2022-May-24 18:03:17] 5033176 760000000 [PB Info 2022-May-24 18:03:24] 10066352 1510000000 [PB Info 2022-May-24 18:03:32] 15099528 2310000000 [PB Info 2022-May-24 18:03:40] 20132704 3040000000 [PB Info 2022-May-24 18:03:47] 25165880 3800000000 [PB Info 2022-May-24 18:03:54] 30199056 4580000000 [PB Info 2022-May-24 18:04:02] 35232232 5330000000 [PB Info 2022-May-24 18:04:09] 40265408 6070000000 [PB Info 2022-May-24 18:04:17] 45298584 6840000000 [PB Info 2022-May-24 18:04:24] 50331760 7610000000 [PB Info 2022-May-24 18:04:31] 55364936 8360000000 [PB Info 2022-May-24 18:04:38] 60398112 9110000000 [PB Info 2022-May-24 18:04:46] 65431288 9870000000 [PB Info 2022-May-24 18:04:53] 70464464 10670000000 [PB Info 2022-May-24 18:05:00] 75497640 11410000000 [PB Info 2022-May-24 18:05:08] 80530816 12150000000 [PB Info 2022-May-24 18:05:15] 85563992 12920000000 [PB Info 2022-May-24 18:05:22] 90597168 13700000000 [PB Info 2022-May-24 18:05:30] 95630344 14440000000 [PB Info 2022-May-24 18:05:37] 100663520 15210000000 [PB Info 2022-May-24 18:05:44] 105696696 15960000000 [PB Info 2022-May-24 18:05:51] 110729872 16710000000 [PB Info 2022-May-24 18:05:59] 115763048 17470000000 [PB Info 2022-May-24 18:06:06] 120796224 18230000000 [PB Info 2022-May-24 18:06:13] 125829400 19010000000 [PB Info 2022-May-24 18:06:21] 130862576 19750000000 [PB Info 2022-May-24 18:06:28] 135895752 20520000000 [PB Info 2022-May-24 18:06:36] 140928928 21290000000 [PB Info 2022-May-24 18:06:43] 145962104 22040000000 [PB Info 2022-May-24 18:06:50] 150995280 22810000000 [PB Info 2022-May-24 18:06:57] 156028456 23560000000 [PB Info 2022-May-24 18:07:05] 161061632 24320000000 [PB Info 2022-May-24 18:07:12] 166094808 25080000000 [PB Info 2022-May-24 18:07:19] 171127984 25840000000 [PB Info 2022-May-24 18:07:27] 176161160 26630000000 [PB Info 2022-May-24 18:07:35] 181194336 27370000000 [PB Info 2022-May-24 18:07:42] 186227512 28110000000 [PB Info 2022-May-24 18:07:49] 191260688 28900000000 [PB Info 2022-May-24 18:07:57] 196293864 29660000000 [PB Info 2022-May-24 18:08:04] 201327040 30400000000 [PB Info 2022-May-24 18:08:12] 206360216 31160000000 [PB Info 2022-May-24 18:08:19] 211393392 31930000000 [PB Info 2022-May-24 18:08:26] 216426568 32680000000 [PB Info 2022-May-24 18:08:33] 221459744 33440000000 [PB Info 2022-May-24 18:08:41] 226492920 34200000000 [PB Info 2022-May-24 18:08:48] 231526096 34960000000 [PB Info 2022-May-24 18:08:55] 236559272 35720000000 [PB Info 2022-May-24 18:09:02] 241592448 36470000000 [PB Info 2022-May-24 18:09:10] 246625624 37240000000 [PB Info 2022-May-24 18:09:17] 251658800 37980000000 [PB Info 2022-May-24 18:09:24] 256691976 38750000000 [PB Info 2022-May-24 18:09:31] 261725152 39510000000 [PB Info 2022-May-24 18:09:39] 266758328 40260000000 [PB Info 2022-May-24 18:09:46] 271791504 41040000000 [PB Info 2022-May-24 18:09:53] 276824680 41800000000 [PB Info 2022-May-24 18:10:01] 281857856 42560000000 [PB Info 2022-May-24 18:10:08] 286891032 43320000000 [PB Info 2022-May-24 18:10:15] 291924208 44100000000 [PB Info 2022-May-24 18:10:22] 296957384 44840000000 [PB Info 2022-May-24 18:10:29] 301990560 45620000000 [PB Info 2022-May-24 18:10:37] 307023736 46360000000 [PB Info 2022-May-24 18:10:44] 312056912 47130000000 [PB Info 2022-May-24 18:10:52] 317090088 47880000000 [PB Info 2022-May-24 18:10:59] 322123264 48640000000 [PB Info 2022-May-24 18:11:06] 327156440 49390000000 [PB Info 2022-May-24 18:11:13] 332189616 50170000000 [PB Info 2022-May-24 18:11:20] 337222792 50910000000 [PB Info 2022-May-24 18:11:28] 342255968 51660000000 [PB Info 2022-May-24 18:11:35] 347289144 52440000000 [PB Info 2022-May-24 18:11:43] 352322320 53170000000 [PB Info 2022-May-24 18:11:50] 357355496 53950000000 [PB Info 2022-May-24 18:11:57] 362388672 54710000000 [PB Info 2022-May-24 18:12:05] 367421848 55470000000 [PB Info 2022-May-24 18:12:12] 372455024 56230000000 [PB Info 2022-May-24 18:12:19] 377488200 57000000000 [PB Info 2022-May-24 18:12:27] 382521376 57780000000 [PB Info 2022-May-24 18:12:34] 387554552 58520000000 [PB Info 2022-May-24 18:12:41] 392587728 59280000000 [PB Info 2022-May-24 18:12:49] 397620904 60030000000 [PB Info 2022-May-24 18:12:56] 402654080 60820000000 [PB Info 2022-May-24 18:13:03] 407687256 61560000000 [PB Info 2022-May-24 18:13:10] 412720432 62290000000 [PB Info 2022-May-24 18:13:18] 417753608 63070000000 [PB Info 2022-May-24 18:13:25] 422786784 63840000000 [PB Info 2022-May-24 18:13:32] 427819960 64600000000 [PB Info 2022-May-24 18:13:40] 432853136 65360000000 [PB Info 2022-May-24 18:13:47] 437886312 66130000000 [PB Info 2022-May-24 18:13:54] 442919488 66890000000 [PB Info 2022-May-24 18:14:01] 447952664 67660000000 [PB Info 2022-May-24 18:14:08] 452985840 68400000000 [PB Info 2022-May-24 18:14:16] 458019016 69180000000 [PB Info 2022-May-24 18:14:23] 463052192 69920000000 [PB Info 2022-May-24 18:14:31] 468085368 70680000000 [PB Info 2022-May-24 18:14:38] 473118544 71440000000 [PB Info 2022-May-24 18:14:46] 478151720 72210000000 [PB Info 2022-May-24 18:14:53] 483184896 72960000000 [PB Info 2022-May-24 18:15:00] 488218072 73720000000 [PB Info 2022-May-24 18:15:08] 493251248 74500000000 [PB Info 2022-May-24 18:15:15] 498284424 75250000000 [PB Info 2022-May-24 18:15:22] 503317600 76010000000 [PB Info 2022-May-24 18:15:29] 508350776 76770000000 [PB Info 2022-May-24 18:15:36] 513383952 77530000000 [PB Info 2022-May-24 18:15:44] 518417128 78300000000 [PB Info 2022-May-24 18:15:51] 523450304 79040000000 [PB Info 2022-May-24 18:15:59] 528483480 79800000000 [PB Info 2022-May-24 18:16:06] 533516656 80570000000 [PB Info 2022-May-24 18:16:13] 538549832 81310000000 [PB Info 2022-May-24 18:16:21] 543583008 82070000000 [PB Info 2022-May-24 18:16:28] 548616184 82820000000 [PB Info 2022-May-24 18:16:35] 553649360 83600000000 [PB Info 2022-May-24 18:16:43] 558682536 84350000000 [PB Info 2022-May-24 18:16:50] 563715712 85120000000 [PB Info 2022-May-24 18:16:57] 568748888 85850000000 [PB Info 2022-May-24 18:17:05] 573782064 86640000000 [PB Info 2022-May-24 18:17:12] 578815240 87400000000 [PB Info 2022-May-24 18:17:20] 583848416 88170000000 [PB Info 2022-May-24 18:17:27] 588881592 88920000000 [PB Info 2022-May-24 18:17:34] 593914768 89700000000 [PB Info 2022-May-24 18:17:42] 598947944 90440000000 [PB Info 2022-May-24 18:17:49] 603981120 91200000000 [PB Info 2022-May-24 18:17:57] 609014296 91960000000 [PB Info 2022-May-24 18:18:04] 614047472 92730000000 [PB Info 2022-May-24 18:18:11] 619080648 93470000000 [PB Info 2022-May-24 18:18:19] 624113824 94250000000 [PB Info 2022-May-24 18:18:26] 629147000 95000000000 [PB Info 2022-May-24 18:18:33] 634180176 95750000000 [PB Info 2022-May-24 18:18:40] 639213352 96520000000 [PB Info 2022-May-24 18:18:47] 644246528 97280000000 [PB Info 2022-May-24 18:18:55] 649279704 98050000000 [PB Info 2022-May-24 18:19:02] 654312880 98800000000 [PB Info 2022-May-24 18:19:09] 659346056 99560000000 [PB Info 2022-May-24 18:19:16] 664379232 100300000000 [PB Info 2022-May-24 18:19:23] 669412408 101070000000 [PB Info 2022-May-24 18:19:31] 674445584 101840000000 [PB Info 2022-May-24 18:19:38] 679478760 102600000000 [PB Info 2022-May-24 18:19:45] 684511936 103350000000 [PB Info 2022-May-24 18:19:52] 689545112 104130000000 [PB Info 2022-May-24 18:20:00] 694578288 104880000000 [PB Info 2022-May-24 18:20:07] 699611464 105640000000 [PB Info 2022-May-24 18:20:15] 704644640 106400000000 [PB Info 2022-May-24 18:20:22] 709677816 107170000000 [PB Info 2022-May-24 18:20:29] 714710992 107920000000 [PB Info 2022-May-24 18:20:37] 719744168 108680000000 [PB Info 2022-May-24 18:20:44] 724777344 109440000000 [PB Info 2022-May-24 18:20:51] 729810520 110200000000 [PB Info 2022-May-24 18:20:58] 734843696 110970000000 [PB Info 2022-May-24 18:21:06] 739876872 111730000000 [PB Info 2022-May-24 18:21:13] 744910048 112460000000 [PB Info 2022-May-24 18:21:21] 749943224 113280000000 [PB Info 2022-May-24 18:21:28] 754976400 114000000000 [PB Info 2022-May-24 18:21:35] 760009576 114780000000 [PB Info 2022-May-24 18:21:42] 765042752 115530000000 [PB Info 2022-May-24 18:21:50] 770075928 116280000000 [PB Info 2022-May-24 18:21:57] 775109104 117040000000 [PB Info 2022-May-24 18:22:04] 780142280 117820000000 [PB Info 2022-May-24 18:22:12] 785175456 118560000000 [PB Info 2022-May-24 18:22:20] 790208632 119330000000 [PB Info 2022-May-24 18:22:27] 795241808 120110000000 [PB Info 2022-May-24 18:22:34] 800274984 120840000000 [PB Info 2022-May-24 18:22:42] 805308160 121600000000 [PB Info 2022-May-24 18:22:50] 810341336 122360000000 [PB Info 2022-May-24 18:22:57] 815374512 123120000000 [PB Info 2022-May-24 18:23:04] 820407688 123890000000 [PB Info 2022-May-24 18:23:12] 825440864 124640000000 [PB Info 2022-May-24 18:23:19] 830474040 125430000000 [PB Info 2022-May-24 18:23:26] 835507216 126160000000 [PB Info 2022-May-24 18:23:33] 840540392 126930000000 [PB Info 2022-May-24 18:23:41] 845573568 127670000000 [PB Info 2022-May-24 18:23:48] 850606744 128420000000 [PB Info 2022-May-24 18:23:55] 855639920 129200000000 [PB Info 2022-May-24 18:24:03] 860673096 129930000000 [PB Info 2022-May-24 18:24:10] 865706272 130730000000 [PB Info 2022-May-24 18:24:17] 870739448 131500000000 [PB Info 2022-May-24 18:24:24] 875772624 132230000000 [PB Info 2022-May-24 18:24:32] 880805800 133000000000 [PB Info 2022-May-24 18:24:39] 885838976 133740000000 [PB Info 2022-May-24 18:24:46] 890872152 134520000000 [PB Info 2022-May-24 18:24:54] 895905328 135270000000 [PB Info 2022-May-24 18:25:01] 900938504 136050000000 [PB Info 2022-May-24 18:25:08] 905971680 136800000000 [PB Info 2022-May-24 18:25:16] 911004856 137560000000 [PB Info 2022-May-24 18:25:23] 916038032 138330000000 [PB Info 2022-May-24 18:25:31] 921071208 139080000000 [PB Info 2022-May-24 18:25:38] 926104384 139850000000 [PB Info 2022-May-24 18:25:46] 931137560 140600000000 [PB Info 2022-May-24 18:26:03] GPU-BWA Mem time: 1380.876101 seconds [PB Info 2022-May-24 18:26:03] GPU-BWA Mem is finished. [main] CMD: PARABRICKS mem -Z ./pbOpts.txt /workspace/datasets/ref/Homo_sapiens_assembly38.fasta /workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz /workspace/datasets/germline/WGS-LIS-AI018A_R1.fastq.gz @RG\tID:H3WFJDSXX.1\tLB:lib1\tPL:bar\tSM:sample\tPU:H3WFJDSXX.1 [main] Real time: 1386.678 sec; CPU: 39483.336 sec [PB Info 2022-May-24 18:26:03] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:26:03] || Program: GPU-BWA mem, Sorting Phase-I || [PB Info 2022-May-24 18:26:03] || Version: 3.8.0-1.ampere || [PB Info 2022-May-24 18:26:03] || Start Time: Tue May 24 18:02:56 2022 || [PB Info 2022-May-24 18:26:03] || End Time: Tue May 24 18:26:03 2022 || [PB Info 2022-May-24 18:26:03] || Total Time: 23 minutes 7 seconds || [PB Info 2022-May-24 18:26:03] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:26:06] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:26:06] || Parabricks accelerated Genomics Pipeline || [PB Info 2022-May-24 18:26:06] || Version 3.8.0-1.ampere || [PB Info 2022-May-24 18:26:06] || Sorting Phase-II || [PB Info 2022-May-24 18:26:06] || Contact: Parabricks-Support@nvidia.com || [PB Info 2022-May-24 18:26:06] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:26:06] progressMeter - Percentage [PB Info 2022-May-24 18:26:06] 0.0 0.00 GB [PB Info 2022-May-24 18:26:16] 6.0 3.00 GB [PB Info 2022-May-24 18:26:26] 17.2 3.00 GB [PB Info 2022-May-24 18:26:36] 27.0 3.00 GB [PB Info 2022-May-24 18:26:46] 38.6 3.00 GB [PB Info 2022-May-24 18:26:56] 49.4 3.00 GB [PB Info 2022-May-24 18:27:06] 60.0 3.00 GB [PB Info 2022-May-24 18:27:16] 72.7 3.00 GB [PB Info 2022-May-24 18:27:26] 83.0 3.00 GB [PB Info 2022-May-24 18:27:36] 99.6 1.00 GB [PB Info 2022-May-24 18:27:46] Sorting and Marking: 100.004 seconds [PB Info 2022-May-24 18:27:46] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:27:46] || Program: Sorting Phase-II || [PB Info 2022-May-24 18:27:46] || Version: 3.8.0-1.ampere || [PB Info 2022-May-24 18:27:46] || Start Time: Tue May 24 18:26:06 2022 || [PB Info 2022-May-24 18:27:46] || End Time: Tue May 24 18:27:46 2022 || [PB Info 2022-May-24 18:27:46] || Total Time: 1 minute 40 seconds || [PB Info 2022-May-24 18:27:46] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:27:46] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:27:46] || Parabricks accelerated Genomics Pipeline || [PB Info 2022-May-24 18:27:46] || Version 3.8.0-1.ampere || [PB Info 2022-May-24 18:27:46] || Marking Duplicates, BQSR || [PB Info 2022-May-24 18:27:46] || Contact: Parabricks-Support@nvidia.com || [PB Info 2022-May-24 18:27:46] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:27:47] progressMeter - Percentage [PB Info 2022-May-24 18:27:57] 0.0 48.03 GB [PB Info 2022-May-24 18:28:07] 0.0 93.73 GB [PB Info 2022-May-24 18:28:17] 0.4 135.85 GB [PB Info 2022-May-24 18:28:27] 2.5 169.24 GB [PB Info 2022-May-24 18:28:37] 5.9 195.30 GB [PB Info 2022-May-24 18:28:47] 8.6 226.34 GB [PB Info 2022-May-24 18:28:57] 11.3 258.30 GB [PB Info 2022-May-24 18:29:07] 15.0 289.23 GB [PB Info 2022-May-24 18:29:17] 19.4 300.92 GB [PB Info 2022-May-24 18:29:27] 23.7 283.09 GB [PB Info 2022-May-24 18:29:37] 28.1 265.47 GB [PB Info 2022-May-24 18:29:47] 32.4 248.70 GB [PB Info 2022-May-24 18:29:57] 36.3 233.23 GB [PB Info 2022-May-24 18:30:07] 40.7 215.42 GB [PB Info 2022-May-24 18:30:17] 44.9 198.51 GB [PB Info 2022-May-24 18:30:27] 49.4 182.24 GB [PB Info 2022-May-24 18:30:37] 52.2 170.51 GB [PB Info 2022-May-24 18:30:47] 55.8 155.64 GB [PB Info 2022-May-24 18:30:57] 58.4 144.86 GB [PB Info 2022-May-24 18:31:07] 60.9 134.92 GB [PB Info 2022-May-24 18:31:17] 64.0 123.84 GB [PB Info 2022-May-24 18:31:27] 68.0 109.63 GB [PB Info 2022-May-24 18:31:37] 72.2 95.28 GB [PB Info 2022-May-24 18:31:47] 76.3 78.44 GB [PB Info 2022-May-24 18:31:57] 81.1 59.02 GB [PB Info 2022-May-24 18:32:07] 85.1 43.30 GB [PB Info 2022-May-24 18:32:17] 89.8 24.87 GB [PB Info 2022-May-24 18:32:27] 94.6 11.05 GB [PB Info 2022-May-24 18:32:37] 100.0 0.00 GB [PB Info 2022-May-24 18:32:37] BQSR and writing final BAM: 290.082 seconds [PB Info 2022-May-24 18:32:37] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:32:37] || Program: Marking Duplicates, BQSR || [PB Info 2022-May-24 18:32:37] || Version: 3.8.0-1.ampere || [PB Info 2022-May-24 18:32:37] || Start Time: Tue May 24 18:27:46 2022 || [PB Info 2022-May-24 18:32:37] || End Time: Tue May 24 18:32:37 2022 || [PB Info 2022-May-24 18:32:37] || Total Time: 4 minutes 51 seconds || [PB Info 2022-May-24 18:32:37] ------------------------------------------------------------------------------ /usr/local/parabricks/.pb/binaries//bin/htvc /workspace/datasets/ref/Homo_sapiens_assembly38.fasta /workspace/output/output.bam 4 -o /workspace/output/output.vcf -nt 5 -a /workspace/output/report.txt [PB Info 2022-May-24 18:33:11] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:33:11] || Parabricks accelerated Genomics Pipeline || [PB Info 2022-May-24 18:33:11] || Version 3.8.0-1.ampere || [PB Info 2022-May-24 18:33:11] || GPU-GATK4 HaplotypeCaller || [PB Info 2022-May-24 18:33:11] || Contact: Parabricks-Support@nvidia.com || [PB Info 2022-May-24 18:33:11] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:33:24] 0 /workspace/output/output.bam/workspace/output/output.vcf [PB Info 2022-May-24 18:33:24] ProgressMeter - Current-Locus Elapsed-MinutesRegions-Processed Regions/Minute [PB Info 2022-May-24 18:33:34] chr1:42652801 0.2 255182 1531074 [PB Info 2022-May-24 18:33:44] chr1:61718401 0.3 346556 1039668 [PB Info 2022-May-24 18:33:54] chr1:73795201 0.5 415832 831660 [PB Info 2022-May-24 18:34:04] chr1:88224001 0.7 497220 745830 [PB Info 2022-May-24 18:34:14] chr1:109180801 0.8 617064 740476 [PB Info 2022-May-24 18:34:24] chr1:167539201 1.0 826379 826379 [PB Info 2022-May-24 18:34:34] chr1:201307201 1.2 1013619 868816 [PB Info 2022-May-24 18:34:44] chr1:227640001 1.3 1165645 874233 [PB Info 2022-May-24 18:34:54] chr2:11337601 1.5 1347088 898058 [PB Info 2022-May-24 18:35:04] chr2:42672001 1.7 1526643 915985 [PB Info 2022-May-24 18:35:14] chr2:79939201 1.8 1739431 948780 [PB Info 2022-May-24 18:35:24] chr2:119448001 2.0 1935026 967513 [PB Info 2022-May-24 18:35:34] chr2:152414401 2.2 2121671 979232 [PB Info 2022-May-24 18:35:44] chr2:187540801 2.3 2319932 994256 [PB Info 2022-May-24 18:35:54] chr2:221404801 2.5 2509807 1003922 [PB Info 2022-May-24 18:36:04] chr3:12302401 2.7 2702866 1013574 [PB Info 2022-May-24 18:36:14] chr3:45072001 2.8 2884748 1018143 [PB Info 2022-May-24 18:36:24] chr3:79828801 3.0 3082745 1027581 [PB Info 2022-May-24 18:36:34] chr3:116976001 3.2 3287114 1038036 [PB Info 2022-May-24 18:36:44] chr3:150720001 3.3 3473801 1042140 [PB Info 2022-May-24 18:36:54] chr3:183139201 3.5 3648364 1042389 [PB Info 2022-May-24 18:37:04] chr4:18537601 3.7 3843360 1048189 [PB Info 2022-May-24 18:37:14] chr4:56472001 3.8 4048657 1056171 [PB Info 2022-May-24 18:37:24] chr4:90643201 4.0 4243640 1060910 [PB Info 2022-May-24 18:37:34] chr4:124478401 4.2 4433236 1063976 [PB Info 2022-May-24 18:37:44] chr4:156547201 4.3 4617614 1065603 [PB Info 2022-May-24 18:37:54] chr5:1272001 4.5 4818605 1070798 [PB Info 2022-May-24 18:38:04] chr5:36561601 4.7 5025918 1076982 [PB Info 2022-May-24 18:38:14] chr5:77596801 4.8 5226415 1081327 [PB Info 2022-May-24 18:38:24] chr5:113712001 5.0 5422373 1084474 [PB Info 2022-May-24 18:38:34] chr5:146683201 5.2 5603598 1084567 [PB Info 2022-May-24 18:38:44] chr6:796801 5.3 5802018 1087877 [PB Info 2022-May-24 18:38:54] chr6:40588801 5.5 6009406 1092619 [PB Info 2022-May-24 18:39:04] chr6:77836801 5.7 6213718 1096538 [PB Info 2022-May-24 18:39:14] chr6:112728001 5.8 6412760 1099330 [PB Info 2022-May-24 18:39:24] chr6:149001601 6.0 6609195 1101532 [PB Info 2022-May-24 18:39:34] chr7:13305601 6.2 6816555 1105387 [PB Info 2022-May-24 18:39:44] chr7:45940801 6.3 7004427 1105962 [PB Info 2022-May-24 18:39:54] chr7:87523201 6.5 7229211 1112186 [PB Info 2022-May-24 18:40:04] chr7:123369601 6.7 7424689 1113703 [PB Info 2022-May-24 18:40:14] chr8:3115201 6.8 7630785 1116700 [PB Info 2022-May-24 18:40:24] chr8:41212801 7.0 7859542 1122791 [PB Info 2022-May-24 18:40:34] chr8:80030401 7.2 8061798 1124902 [PB Info 2022-May-24 18:40:44] chr8:115958401 7.3 8255292 1125721 [PB Info 2022-May-24 18:40:54] chr9:4622401 7.5 8445700 1126092 [PB Info 2022-May-24 18:41:04] chr9:39806401 7.7 8655294 1128950 [PB Info 2022-May-24 18:41:14] chr9:100881601 7.8 8877581 1133308 [PB Info 2022-May-24 18:41:24] chr9:135667201 8.0 9073718 1134214 [PB Info 2022-May-24 18:41:34] chr10:31233601 8.2 9277419 1136010 [PB Info 2022-May-24 18:41:44] chr10:70396801 8.3 9492442 1139093 [PB Info 2022-May-24 18:41:54] chr10:104500801 8.5 9681305 1138977 [PB Info 2022-May-24 18:42:04] chr11:5937601 8.7 9884872 1140562 [PB Info 2022-May-24 18:42:14] chr11:41078401 8.8 10091265 1142407 [PB Info 2022-May-24 18:42:24] chr11:77760001 9.0 10290075 1143341 [PB Info 2022-May-24 18:42:34] chr11:114220801 9.2 10500771 1145538 [PB Info 2022-May-24 18:42:44] chr12:14136001 9.3 10699289 1146352 [PB Info 2022-May-24 18:42:54] chr12:51057601 9.5 10902904 1147674 [PB Info 2022-May-24 18:43:04] chr12:86851201 9.7 11106169 1148914 [PB Info 2022-May-24 18:43:14] chr12:122356801 9.8 11306209 1149783 [PB Info 2022-May-24 18:43:24] chr13:42590401 10.0 11525302 1152530 [PB Info 2022-May-24 18:43:34] chr13:79036801 10.2 11733750 1154139 [PB Info 2022-May-24 18:43:44] chr14:20491201 10.3 11949307 1156384 [PB Info 2022-May-24 18:43:54] chr14:55656001 10.5 12155722 1157687 [PB Info 2022-May-24 18:44:04] chr14:89059201 10.7 12344248 1157273 [PB Info 2022-May-24 18:44:14] chr15:45921601 10.8 12557598 1159162 [PB Info 2022-May-24 18:44:24] chr15:82084801 11.0 12763598 1160327 [PB Info 2022-May-24 18:44:34] chr16:20553601 11.2 12986350 1162956 [PB Info 2022-May-24 18:44:44] chr16:68524801 11.3 13187013 1163559 [PB Info 2022-May-24 18:44:54] chr17:13195201 11.5 13396048 1164873 [PB Info 2022-May-24 18:45:04] chr17:55915201 11.7 13595451 1165324 [PB Info 2022-May-24 18:45:14] chr18:5217601 11.8 13782533 1164721 [PB Info 2022-May-24 18:45:24] chr18:45648001 12.0 13990766 1165897 [PB Info 2022-May-24 18:45:34] chr19:2971201 12.2 14207126 1167708 [PB Info 2022-May-24 18:45:44] chr19:41169601 12.3 14409672 1168351 [PB Info 2022-May-24 18:45:54] chr20:19152001 12.5 14617645 1169410 [PB Info 2022-May-24 18:46:04] chr20:57388801 12.7 14820050 1170003 [PB Info 2022-May-24 18:46:14] chr21:41539201 12.8 15042581 1172149 [PB Info 2022-May-24 18:46:24] chr22:48916801 13.0 15260697 1173899 [PB Info 2022-May-24 18:46:34] chrX:35419201 13.2 15454760 1173778 [PB Info 2022-May-24 18:46:44] chrX:78806401 13.3 15650993 1173824 [PB Info 2022-May-24 18:46:54] chrX:118982401 13.5 15852480 1174257 [PB Info 2022-May-24 18:47:04] chrY:11289601 13.7 16088000 1177170 [PB Info 2022-May-24 18:47:14] Total time taken: 830.321212 [PB Info 2022-May-24 18:47:14] ------------------------------------------------------------------------------ [PB Info 2022-May-24 18:47:14] || Program: GPU-GATK4 HaplotypeCaller || [PB Info 2022-May-24 18:47:14] || Version: 3.8.0-1.ampere || [PB Info 2022-May-24 18:47:14] || Start Time: Tue May 24 18:33:11 2022 || [PB Info 2022-May-24 18:47:14] || End Time: Tue May 24 18:47:14 2022 || [PB Info 2022-May-24 18:47:14] || Total Time: 14 minutes 3 seconds || [PB Info 2022-May-24 18:47:14] ------------------------------------------------------------------------------ Tue May 24 18:47:15 CST 2022 end_time: 1653389235 output/ ├── [ 37G] output.bam ├── [9.0M] output.bam.bai ├── [885M] output.vcf ├── [ 90K] output_chrs.txt └── [112K] report.txt 0 directories, 5 files Total time: 0h 44m 20s ``` ::: - program1: - GPU: ![](https://i.imgur.com/2Ejul6r.png) - program4: GPU-GATK4 HaplotypeCaller - #### CPU: 利用率低 ![](https://i.imgur.com/02X8iY4.png) - #### GPU: 利用率非常低 ![](https://i.imgur.com/KunfInx.png) ![](https://i.imgur.com/UxgcdYt.png) ![](https://i.imgur.com/u3BXBtE.png) ![](https://i.imgur.com/ccKSqyd.png)