# Run CESM (v1.2.1) on SNOW cluster (Feb, 2022) ### Log in to the SNOW cluster ``` ssh -X [yourNetID]@head.arcc.albany.edu ``` ### Setup environment variables Edit the `.bashrc` file in the home directory. * Set CESM root path - export the CESM(v1.2.1) built by Brian Rose. ``` export CCSMROOT="/network/rit/home/br546577/roselab_rit/cesm/cesm1_2_1" ``` * Set Intel compiler path ``` export PATH="$PATH:/network/rit/lab/snowclus/2101/bin:/network/daes/linuxapps/intel17/bin" export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/network/rit/lab/snowclus/2101/lib:/network/daes/linuxapps/intel17/lib/intel64" export LM_LICENSE_FILE="/network/daes/linuxapps/intel17/bin/licenses/intel.lic" ``` * Set NetCDF library and linker flags ``` export NETCDF="/network/rit/lab/snowclus/2101" export LDFLAGS="-L/network/rit/lab/snowclus/2101/lib" ``` ### Run CESM * Create Case (set case name, resolution, component sets) ``` /network/rit/lab/roselab_rit/cesm/cesm1_2_1/scripts/create_newcase -mach snow -res [resolution] -compset [compset_name] -case [casename] ``` * Modify `env_run.xml` if needed. For example, modify the run time to 3 years and do not archive the data output. ``` ./xmlchange STOP_OPTION=nyears ./xmlchange STOP_N=3 ./xmlchange DOUT_S=FALSE ``` * Setup ``` ./cesm_setup ``` * Build (on a compute node) ``` srun -p snow `pwd`/[casename].build ``` * Edit `[casename].run` For MPI4.0, an error (initialization failed) should be fixed. See https://www.open-mpi.org/faq/?category=openfabrics#ofa-device-error. In line #186, add `-mca btl ^openib -mca pml ucx` after `mpirun`. ``` #SBATCH --ntasks-per-node=32 #SBATCH --mem-per-cpu=3584M mpirun -mca btl ^openib -mca pml ucx ``` * Submit ``` ./[casename].submit ``` * Look up the log file ``` less +F cesm.log.xxxxxx ``` ### Slurm * See what is running ``` squeue sinfo ``` * See specific node info ``` scontrol show node [snow-xx] ``` * Kill the running case ``` scancel [JOBID] ```
Last changed by  
 
yuanjenlin
293
Published on HackMD