Try   HackMD

Taiwan Server1 update history

Update log

Install CentOS

Source

CentOS official download page
CentOS-8.5.2111-x86_64-dvd1.iso

Burn CentOS.iso into USB disk in Windows

USB disk should be formated in NTFS(Windows readable writeable, Linux readable)

balenaEtcher official download page
balenaEtcher-Portable-1.7.3

Partition assignment

Disk Parition MountPoint Size Descriptions
sda sda1 /boot/efi 200MB 開機磁區
sda sda2 /boot 500MB 開機磁區
sdc swap swap 128GB 記憶體交換區
sdc home /home 3TB 家目錄
sdc root / 2TB 根目錄
sdc pkg /pkg 2TB 公用程式
sdc work /work 32TB 工作目錄

  • /home

User specific space, which stores tools, references and etc. for their own use.

  • /pkg

Public tools share among users.
Always install packages by root.

  • /pkg/source

Downloaded packages.

  • /work

Working space, stores references and working directory for each projects seperately.

  • /work/reference

Stores references for projects.

Account and group Settings













adduser <New user name>
passwd <New user name>
groupadd MicroBioAd    #MicroBio Admin group with sudoers permission
groupadd MicroBio      #MicroBio group
visudo                 #Make MicroBioAd possesses sudoers permission
  -insert: %MicroBioAd     ALL=(ALL)       ALL
usermod -a -G MicroBioAd <New user name>
usermod -a -G MicroBio <New user name>
chown root:MicroBio /work
chown root:MicroBioAd /pkg
chmod 775 /work
chmod 775 /pkg

Environment settings

更改語系

系統安裝時選擇了中文，報錯時中文顯示不方便查詢問題，因此更改語系





locale -a|grep en_US    #check all en_US avaliable
vi /etc/locale.conf
    -replace: LANG="zh_TW.utf8" -> LANG="en_US.utf8"
source /etc/locale.conf

關閉Root登入

root作為系統最高權限，SSH 服務對外開放的情況下資安風險極大，透過網路就可以用暴力法嘗試破解 root 的密碼，因此關閉root改用個別帳號登入後用su指令登入root。




vi /etc/ssh/sshd_config
    -replace: PermitRootLogin yes -> PermitRootLogin no
systemctl restart sshd

防火牆設定











#開啟PORT
sudo firewall-cmd --zone=public --add-port=[PORT]/tcp --permanent
#開放HTTP連線
sudo firewall-cmd --zone=public --add-service=http --permanent
sudo firewall-cmd --reload
sudo firewall-cmd --zone=public --list-all  #check PORT
#需要時關閉PORT
sudo firewall-cmd --zone=public --permanent --remove-port=[PORT]/tcp --permanent 
#需要時關閉HTTP
sudo firewall-cmd --zone=public --remove-service=http --permanent

系統重啟

重啟後SSH無法連線，需檢查設定檔












#20220523歷程
#透過指令檢查SSH狀態
sshd -t
#循線查到/etc/ssh/sshd_config出現錯誤

#備份/etc/ssh/sshd_config
cp /etc/ssh/sshd_config /etc/ssh/sshd_config_bad1
#將預先準備好的檔案覆蓋回來
cp /etc/ssh/sshd_config_bak /etc/ssh/sshd_config
#重啟SSH服務
systemctl restart sshd

重啟Xampp，以啟動Apache MySQL ProFTPD等服務






#關閉原生系統HTTPD服務
systemctl disable httpd
systemctl stop httpd
#重啟Xampp
/opt/lampp/xampp restart

看見服務都啟動即完成

Restarting XAMPP for Linux 8.1.1-2…
XAMPP: Stopping Apache…not running.
XAMPP: Stopping MySQL…not running.
XAMPP: Stopping ProFTPD…ok.
XAMPP: Starting Apache…ok.
XAMPP: Starting MySQL…ok.
XAMPP: Starting ProFTPD…ok.

Install XAMPP

Source

XAMPP official

Commands






cd /pkg/source
wget https://downloadsapachefriends.global.ssl.fastly.net/8.1.6/xampp-linux-x64-8.1.6-0-installer.run
chmod +x xampp-linux-x64-8.1.6-0-installer.run
./xampp-linux-x64-8.1.6-0-installer.run
/opt/lampp/lampp start

Error: libnsl not found

XAMPP: SELinux is activated. Making XAMPP fit SELinux…
chcon: 無法存取 '/opt/lampp/lib/mysql/*.so': 沒有此一檔案或目錄
Starting XAMPP for Linux 8.1.1-2…
XAMPP: Starting Apache…fail.
httpd: Syntax error on line 522 of /opt/lampp/etc/httpd.conf: Syntax error on line 6 of /opt/lampp/etc/extra/httpd-xampp.conf: Cannot load modules/mod_perl.so into server: libnsl.so.1: cannot open shared object file: No such file or directory
XAMPP: Starting MySQL…ok.
XAMPP: Starting ProFTPD…ok.







## Do Repo update first
#Install libnsl
dnf upgrade --refresh
sudo dnf install libnsl
#Activate XAMPP again
/opt/lampp/lampp start

XAMPP: Starting Apache…ok.
XAMPP: Starting MySQL…already running.
XAMPP: Starting ProFTPD…already running.

Install Python

Source

Python official

Commands
















cd /pkg/source/
#Download python3.10
wget https://www.python.org/ftp/python/3.10.1/Python-3.10.1.tgz
tar -zxvf Python-3.10.1.tgz
mv ./Python-3.10.1 /pkg/
cd /pkg/Python-3.10.1
mkdir Build
cd Build
../configure
make -j 4 && make install
#compile finished

#Add path into $PATH
vi ~/.bash_profile
    -insert: PATH=$PATH:/pkg/Python-3.10.1/Build 在export PATH之前

Install Lynx

Source

Lynx official

Commands


















cd /pkg/source
wget http://invisible-island.net/datafiles/release/lynx-cur.tar.gz
tar zxvf lynx-cur.tar.gz
mv lynx2.9.0dev.10/ /pkg
cd /pkg/lynx2.9.0dev.10/
mkdir Build
cd Build
../configure

# Error massage:
# configure: error: No curses header-files found

yum install ncurses-devel
#ncurses-devel install successed, continue lynx install

../configure
make -j 4 && make install

Install Nodejs

Source

NodeJS official






cd /pkg/source
wget https://nodejs.org/dist/v16.13.1/node-v16.13.1-linux-x64.tar.xz
tar Jxvf node-v16.13.1-linux-x64.tar.xz
mv node-v16.13.1-linux-x64 /pkg
#Finished, no need compile

Install Miniconda & QIIME2

Source

Miniconda download page
QIIME2 official doc

Miniconda



















cd /pkg/source
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh
    - Do you accept the license terms? [yes|no]
        yes
    - Miniconda3 will now be installed into this location: /root/miniconda3
      - Press ENTER to confirm the location
      - Press CTRL-C to abort the installation
      - Or specify a different location below
        /pkg/miniconda3  #Change install path to /pkg/
    - Do you wish the installer to initialize Miniconda3 by running conda init? [yes|no] 
        no
vi ~/.bash_profile
    -insert: PATH=$PATH:/pkg/miniconda3/bin 在export PATH之前
# 不重新登入就使設定生效
source ~/.bash_profile
conda update conda -y
conda install wget -y

QIIME2

























cd /pkg/source

#安裝2022.8版本
wget https://data.qiime2.org/distro/core/qiime2-2022.8-py38-linux-conda.yml
conda env create -n qiime2-2022.8 --file qiime2-2022.8-py38-linux-conda.yml

#安裝2021.11版本
wget https://data.qiime2.org/distro/core/qiime2-2021.11-py38-linux-conda.yml
conda env create -n qiime2-2021.11 --file qiime2-2021.11-py38-linux-conda.yml

#安裝2019.10版本
wget https://data.qiime2.org/distro/core/qiime2-2019.10-py36-linux-conda.yml
conda env create -n qiime2-2019.10 --file qiime2-2019.10-py36-linux-conda.yml

# Initialize conda shell, activate when login
conda init bash
exit # Re-login
# Cancel activate when login
conda config --set auto_activate_base false
# Activate indicated environment when user login
vi ~/.bash_profile
    -insert: conda activate qiime2-2021.11
# 不重新登入就使設定生效
source ~/.bash_profile
QIIME2 Plugin install


# q2-picrust2
conda install q2-picrust2=2021.11 -c conda-forge -c bioconda -c gavinmdouglas

Python-MySQLdb






# miniconda
## "conda install mysql-python" #doesn't work
conda install -c bioconda mysqlclient # download from bioconda channel
# anaconda in windows
pip install mysqlclient

SSH Secure Shell - Algorithm negotiation failed

使用SSH Secure Shell連線時發生Algorithm negotiation failed的警告訊息
更新對應的演算法以解決金鑰產生的問題








vi /etc/ssh/sshd_config
    -insert: 
Ciphers aes128-cbc,aes192-cbc,aes256-cbc,aes128-ctr,aes192-ctr,aes256-ctr,3des-cbc,arcfour128,arcfour256,arcfour,blowfish-cbc,cast128-cbc
MACs hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-sha1-96,hmac-md5-96
KexAlgorithms diffie-hellman-group1-sha1,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group-exchange-sha256,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group1-sha1,curve25519-sha256@libssh.org

systemctl restart sshd

經測試依然無法解決SSH Secure Shell不能連線的問題

Repo update

安裝時出現錯誤
Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
因此修改兩個repo文件，引導linux自該位址下載








sudo vi /etc/yum.repos.d/CentOS-Linux-AppStream.repo
    -insert: baseurl=http://vault.centos.org/$contentdir/$releasever/AppStream/$basearch/os/

sudo vi /etc/yum.repos.d/CentOS-Linux-BaseOS.repo
    -insert: baseurl=http://vault.centos.org/$contentdir/$releasever/BaseOS/$basearch/os/
sudo yum install epel-release
sudo yum -y install htop

Install R

  1. 由於R安裝ggfx時出現錯誤，懷疑是miniconda的R造成的
    因此於系統另外安裝R
  2. yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
    yum install R
    使用上述指令安裝R時，出現錯誤
    Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
  3. 因此修改三個repo文件，引導linux自該位址下載




























vi /etc/yum.repos.d/CentOS-Linux-AppStream.repo
    -modify:
        註解此行
        mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=AppStream&infra=$infra
        
    -insert: 
        baseurl=http://vault.centos.org/$contentdir/$releasever/AppStream/$basearch/os/

vi /etc/yum.repos.d/CentOS-Linux-BaseOS.repo
    -modify:
        註解此行
        mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=BaseOS&infra=$infra
        
    -insert: 
        baseurl=http://vault.centos.org/$contentdir/$releasever/BaseOS/$basearch/os/

vi /etc/yum.repos.d/CentOS-Linux-PowerTools.repo
    -modify:
        註解此兩行
        baseurl=http://mirror.centos.org/$contentdir/$releasever/PowerTools/$basearch/os/
        mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=PowerTools&infra=$infra
        
    -insert: 
        baseurl=http://vault.centos.org/$contentdir/$releasever/PowerTools/$basearch/os/
        
        
yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm
yum install R #網路不好，會出現time out的訊息，這時不要放棄，多打幾次，多段下載才會完成

安裝特定R package時仍然會出現錯誤，如下install.packages('ggfx')

–––––––––––––- [ANTICONF] ––––––––––––––––
Configuration failed to find the harfbuzz freetype2 fribidi library. Try installing:

  • deb: libharfbuzz-dev libfribidi-dev (Debian, Ubuntu, etc)
  • rpm: harfbuzz-devel fribidi-devel (Fedora, EPEL)
  • csw: libharfbuzz_dev libfribidi_dev (Solaris)
  • brew: harfbuzz fribidi (OSX)
    If harfbuzz freetype2 fribidi is already installed, check that 'pkg-config' is in your
    PATH and PKG_CONFIG_PATH contains a harfbuzz freetype2 fribidi.pc file. If pkg-config
    is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
    R CMD INSTALL –configure-vars='INCLUDE_DIR=… LIB_DIR=…'
    ––––––––––––– [ERROR MESSAGE] –––––––––––––-
    <stdin>:1:10: fatal error: hb-ft.h: No such file or directory
    compilation terminated.

觀察錯誤訊息，提示安裝harfbuzz-devel fribidi-devel
發覺不是miniconda的R造成錯誤，改回miniconda的R去安裝

yum install harfbuzz-devel fribidi-devel

完成後再次於miniconda R輸入

install.packages('ggfx')

會再次報錯，繼續按提示安裝

yum install freetype-devel libpng-devel libtiff-devel libjpeg-turbo-devel

會再次報錯，繼續按提示安裝

yum install -y ImageMagick-c++-devel #沒有解決問題

再次進入R輸入install.packages('ggfx')會有重複的錯誤(此問題可能只存在conda)，改用

conda install -c conda-forge imagemagick

完成後再次於R輸入，可成功安裝

install.packages('ggfx')

Install R package

pairwiseAdonis




install.packages('devtools')
library(devtools)
install_github("pmartinezarbizu/pairwiseAdonis/pairwiseAdonis")

Install SRA Toolkit














#Download sratoolkit
wget https://ftp-trace.ncbi.nlm.nih.gov/sra/sdk/3.0.0/sratoolkit.3.0.0-centos_linux64.tar.gz
#extract sratoolkit
tar -zxvf sratoolkit.3.0.0-centos_linux64.tar.gz
# set sratoolkit.2.11.2-centos_linux64/bin/ to PATH
vi ~/.bash_profile
 -insert: 
    PATH=$PATH:/pkg/sratoolkit.2.11.2-centos_linux64/bin/
#configure
vdb-config -i
    CACHE>location of user-repository
    CACHE>process local-location
    save

Install STAR & Trimmomatic

conda install -c bioconda star
conda install -c bioconda trimmomatic

Install ghost-tree













# Download ghost-tree by git to /pkg/source
cd /pkg/source
sudo git clone https://github.com/JTFouquier/q2-ghost-tree.git
# Change permission group for MacGyver to install
sudo chown root:MicroBioAd q2-ghost-tree -R
sudo chmod 777 q2_ghost_tree.egg-info -R
cd ./q2-ghost-tree
# Change permission group of python packages for MacGyver to install ghost-tree
sudo chown MacGyver:MacGyver /pkg/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/ -R
# Install ghost-tree
pip install -e .

Install RIMA














mkdir RIMA
cd RIMA
#Downlad RMIA pipeline from github
git clone  https://github.com/liulab-dfci/RIMA_pipeline.git
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
#Insatll miniconda in RIMA folder
bash Miniconda3-latest-Linux-x86_64.sh
#Export Paths
export CONDA_ROOT=/work/RIMA/miniconda3
export PATH=/work/RIMA/miniconda3/bin:$PATH
#Select GCP as RIMA environment parameter
bash ./RIMA_environment.sh -p {platform --AWS, GCP}
source activate RIMA

Download pre-built references









cd /work/RIMA
wget http://cistrome.org/~lyang/ref.tar.gz
# unzip the reference 
tar -zxvf ref.tar.gz
cd /work/RIMA/RIMA_pipeline
# create ref_files link in RIMA_pipeline folder
ln -s /work/RIMA/ref_files

Update dplyr in conda environment stat_perl_r



















cd /work/RIMA/RIMA_pipeline
#Export Paths
export CONDA_ROOT=/work/RIMA/miniconda3
export PATH=/work/RIMA/miniconda3/bin:$PATH
source activate /work/RIMA/miniconda3/envs/stat_perl_r
> R
    update.packages(ask = FALSE)
    install.packages('ggpubr', dependencies = TRUE)

mkdir /work/RIMA/miniconda3/envs/stat_perl_r/R-3.6.3
cd /work/RIMA/miniconda3/envs/stat_perl_r/R-3.6.3
wget https://cran.r-project.org//src/base/R-3/R-3.6.3.tar.gz
tar -zxf R-3.6.3.tar.gz
cd R-3.6.3
./configure --with-readline=no --with-x=no
    
conda install -c conda-forge r-base=3.6.3
conda install -c conda-forge r-ggpubr

Run RIMA







cd /work/RIMA/RIMA_pipeline
#Export Paths
export CONDA_ROOT=/work/RIMA/miniconda3
export PATH=/work/RIMA/miniconda3/bin:$PATH
source activate RIMA
nohup time snakemake -p -s RIMA.snakefile -j 32 > RIMA.out &

Mount NTFS




















#NTFS3g
su
cd /pkg/source
wget https://tuxera.com/opensource/ntfs-3g_ntfsprogs-2017.3.23.tgz
tar -zxvf ntfs-3g_ntfsprogs-2017.3.23.tgz
cd ntfs-3g_ntfsprogs-2017.3.23/
./configure
make
make install

# EXFAT
yum -y install epel-release http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm
yum -y install fuse-exfat exfat-utils
#check dev
lsblk
    
mount -t ntfs-3g /dev/sdb /mnt/usb
mount -t exfat /dev/sdb /mnt/usb
umount /mnt/usb

Updata Databases for metaphlan







# Update bowtieDB for metaphlan (latest: vJun23)
metaphlan --install --bowtie2db /work/metaphlan/database/metaphlanDB
# Download CMH13v2 HumanGenome
wget https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/009/914/755/GCF_009914755.1_T2T-CHM13v2.0/GCF_009914755.1_T2T-CHM13v2.0_genomic.fna.gz
# Build Bowtie2 index
bowtie2-build GCF_009914755.1_T2T-CHM13v2.0_genomic.fna.gz CHM13v2 --threads 24

Update MataPhlAn HUMANn

Clone conda envronment and update MetaPhlAn HUMANn
Transfer new enviroment to other server















# Clone a new environment
conda create -n metaphlan4_20241224 --clone metaphlan4
conda activate metaphlan4_20241224
# Update mataphlan to indicated version
conda install -c bioconda metaphlan=4.0.6
# Update Humann to indicated version
conda install -c biobakery humann=3.7
# pack environment for other machine
conda env export --name metaphlan4_20241224 > metaphlan4_20241224.yml
# Transfer to target manchine
rsync -avP metaphlan4_20241224.yml Acc@192.168.0.0:/pkg/miniconda3/envs
rsync -avP /work/metaphlan/database/ Acc@192.168.0.0:/work/metaphlan/database/
 # On the target machine
 conda env create --file metaphlan4_20241224.yml

Install Package from github

remotes::install_github("kstagaman/phyloseqCompanion")

Install pyani







conda create --name pyani --yes python=3.7
conda activate pyani_env
#mummer is for accerating pyani
#conda install -c bioconda mummer
conda install --yes blast mummer
pip install pyani

Useage







#MUMmer4（最快）
average_nucleotide_identity.py -i genomes/ -o output_mummer --method ANIm
#BLAST(慢但是準確)
average_nucleotide_identity.py -i genomes/ -o output_blast --method ANIb
#Phylogenetic tree
pyani_graph.py -i output_mummer/ -o output_tree --formats newick,svg

Install ChroQueTas in Conda

A user-friendly software to perform fungicide resistance screening in fungal genomes






conda create --name ChroQueTas_v0.6 python=3.8
conda activate ChroQueTas_v0.6
conda config --add channels bioconda
conda config --add channels conda-forge
conda install -c nmquijada chroquetas=0.6

Useage






ChroQueTas.sh \
    -g Saccharomyces_cerevisiae_BCRC_20262.fasta \
    -s Saccharomyces_cerevisiae -c 1 \
    --min_id 75 --min_cov 75 -t 14 \
    -o Saccharomyces_cerevisiae

20220105重灌前

tags: Server
Last changed by 
 
中天生資
中天生資團隊
0
445

Read more

Taiwan Server2 update history

Install Miniconda & QIIME2

Nov 7, 2023
腸生態健康評估報告-DataBase

Change root password mysqladmin -u root [new password] Create database create database ReportSystem; GRANT ALL PRIVILEGES ON * . * TO &apos;[user]&apos;@&apos;localhost&apos;; FLUSH PRIVILEGES; ## Remove revoke All on *.* from [user]; ## Delete user

Mar 22, 2022
Taiwan server update history(before 20220105)

Change RHEL yum library repositories &#x554F;&#x984C; &#x5DF2;&#x8A3B;&#x518A;&#x7684;RHEL server&#xFF0C;&#x4F46;&#x76EE;&#x524D;&#x975E;&#x8A02;&#x95B1;&#x72C0;&#x614B; &#x5617;&#x8A66;&#x6B77;&#x7A0B; &#x66F4;&#x63DB;yum library repositories&#x8DEF;&#x5F91; &#x53C3;&#x7167;&#x7DB2;&#x5740; wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm sudo yum install ./epel-release-latest-*.noarch.rpm

Jan 14, 2022
Read more from 中天生資
Published on HackMD