# Running CYSTAL17 on Racham

Version: 2021.09.15

## Prerequisites
Make sure you have the following files in folder on Rackham
- `crystal17_v1_0_1_Linux-ifort17_XE_openmpi2.1_emt64.tar.gz` - You wull get access to installation files upon purchasing a license.
- `utils17.zip` - Can be downloaded from https://www.crystal.unito.it/documentation.php

## Installation

On Rackham, copy/paste the following script `install-rackham.sh` to the same folder which contains the files above. Edit `installDIR=${HOME}/bin/crystal17` to point to a project folder if necessary. Make it executable with `chmod +x install-rackham.sh` or execute the lines manually.

`install-rackham.sh`

#!/bin/bash

# Destination folder
# Default  installDIR=${HOME}/bin/crystal17
# To share between members of the same project installDIR=/proj/snic-xxx-xx/bin/crystal17

installDIR=${HOME}/bin/crystal17
mkdir -p ${installDIR}

tar xvf crystal17_v1_0_1_Linux-ifort17_XE_openmpi2.1_emt64.tar.gz -C ${installDIR}
unzip utils17.zip -d ${installDIR}/utils17

# Modifying files
# cry17.bashrc
cp ${installDIR}/utils17/cry17.bashrc ${installDIR}/utils17/cry17.bashrc.bak
sed -i -e "s#export CRY17_ROOT=\"/home/CRYSTAL17\"#export CRY17_ROOT=\"${installDIR}\"#"                   ${installDIR}/utils17/cry17.bashrc
sed -i -e "s#export CRY17_ARCH=\"Linux-ifort\"#export CRY17_ARCH=\"Linux-ifort17_XE_openmpi2.1.0_emt64\"#" ${installDIR}/utils17/cry17.bashrc
sed -i -e "s#export VERSION=\"std\"#export VERSION=\"v1.0.1\"#"                                            ${installDIR}/utils17/cry17.bashrc
sed -i -e "s#export CRY17_SCRDIR=\"\$HOME\/tmp\"#export CRY17_SCRDIR=\"\$SNIC_TMP\"#"                      ${installDIR}/utils17/cry17.bashrc

# runmpi17
cp ${installDIR}/utils17/runmpi17 ${installDIR}/utils17/runmpi17.bak
sed -i -e "s#set MPIDIR   = /opt/openmpi-1.4-icc/bin#set MPIDIR   = /opt/openmpi/2.1.0intel17.2/bin#"      ${installDIR}/utils17/runmpi17
sed -i -e "s#\${MPIDIR}/\${MPIBIN} -machinefile \$CRY17P_MACH/machines.LINUX -np \$NPROCS \$TMPDIR/Pcrystal < \$TMPDIR/INPUT >>\& \$OUTDIR/\$OUTFILE#\#\${MPIDIR}/\${MPIBIN} -machinefile \$CRY17P_MACH/machines.LINUX -np \$NPROCS \$TMPDIR/Pcrystal < \$TMPDIR/INPUT >>\& \$OUTDIR/\$OUTFILE\n \${MPIDIR}/\${MPIBIN} \$TMPDIR/Pcrystal < \$TMPDIR/INPUT >>\& \$OUTDIR/\$OUTFILE#"               ${installDIR}/utils17/runmpi17

## Submiting non-interactive job to the SLURM

Here is an example SLURM test submission script to the `devel` partition. Edit the parameters to match your project, number of CPUs, etc. This example will run on 2 devel nodes (40 CPUs) for up to 15 minutes.

`run-Pcrystal17.rackham`

#!/bin/bash -l
#SBATCH  -n 40
#SBATCH  -p devel
#SBATCH --exclusive 
##SBATCH --ntasks-per-node 20
#SBATCH -t 00:15:00
#SBATCH -J test
#SBATCH -A snic-xxx-xx

module load intel/17.2 openmpi/2.1.0
export OPAL_PREFIX=/opt/openmpi/2.1.0intel17.2

JOB="MgOH2"
DIR=$SLURM_SUBMIT_DIR
CRYSTAL17_root=${HOME}/bin/crystal17

source ${CRYSTAL17_root}/utils17/cry17.bashrc
export CRY17P_MACH=$(pwd)
env > env.txt
scontrol show hostnames $SLURM_JOB_NODELIST > nodes.par && awk '{print $1":"20}' nodes.par > machines.LINUX

${CRYSTAL17_root}/utils17/runmpi17 $SLURM_NTASKS "$JOB"

### Example test input
This input file runs full geometry optimization of MgOH~2~ crystal.

`MgOH2.d12`

MgOH2
CRYSTAL
1 0 0
P -3 M 1
3.179459082 4.763086088
3
12 0.0 0.0 0.
8 0.3333333333 0.66666666667 0.2193726043057150
1 0.333333333 0.666666667 0.4236468535645939
OPTGEOM
FULLOPTG
ENDOPT
END
12  4
0  0  8  2.  1.
 68371.875      0.0002226
  9699.34009    0.0018982
  2041.176786   0.0110451
   529.862906   0.0500627
   159.186000   0.169123
    54.6848     0.367031
    21.2357     0.400410
     8.74604    0.14987
0  1  5  8.  1.
   156.795     -0.00624    0.00772
    31.0339    -0.07882    0.06427
     9.6453    -0.07992    0.2104
     3.7109     0.29063    0.34314
     1.61164    0.57164    0.3735
0 1 1 0. 1.
    0.68     1.         1.
0 1 1 0. 1.
    0.28     1.         1.
    8  5
  0  0  8    2.  1.
 8.0200E+3  1.0800E-3
 1.3380E+3  8.0400E-3
 2.5540E+2  5.3240E-2
 6.9220E+1  1.6810E-1
 2.3900E+1  3.5810E-1
 9.2640E+0  3.8550E-1
 3.8510E+0  1.4680E-1
 1.2120E+0  7.2800E-2
  0  1  4    7.  1.
 4.9430E+1 -8.8300E-3  9.5800E-3
 1.0470E+1 -9.1500E-2  6.9600E-2
 3.2350E+0 -4.0200E-2  2.0650E-1
 1.2170E+0  3.7900E-1  3.4700E-1
  0  1  1    0.  1.
 4.7800E-1  1.0000E+0  1.0000E+0
  0  1  1    0.  1.
 1.7400E-1  1.0000E+0  1.0000E+0
  2  3  1    0.  1.
  1  3
  0  0  3    1.  1.
 19.238400  0.032828
  2.898720  0.231204
  0.653472  0.817226
  0  0  1    0.  1.
  0.163064  1.000000
  0  2  1    0.  1.
  1.000000  1.000000
99  0
PRINT
END
SCFDIR
DFT
B3LYP
XLGRID
END
BIPOSIZE
5000000
EXCHSIZE
5000000
TOLINTEG
8 8 8 8 16
TOLDEE
9
SHRINK
6 6
LEVSHIFT
4 1
FMIXING
40
END

To submit the job run : `$ sbatch run-Pcrystal17.rackham`

# Contacts:
- [Pavlin Mitev](https://katalog.uu.se/profile/?id=N3-1425)
- [UPPMAX](https://www.uppmax.uu.se/)
- [SNIC AE@UPPMAX - related documentation](/8sqXISVRRquPDSw9o1DizQ)


###### tags: `UPPMAX`, `SNIC`