# OM4p5 JRA55do1.5
## Setup Gitlab repo
```
git clone https://gitlab.gfdl.noaa.gov/Jan-erik.Tesdal/xml.git om4_jra
cd om4_jra/
git remote add upstream https://gitlab.gfdl.noaa.gov/Brandon.Reichl/xml.git
git fetch upstream
```
```
git branch -a
```
Switch to Brandon's branch
```
git checkout Add_XML_OM4p25_JRA55do1p5_0netfw
```
## Test p25 xml
Open shell and load fre
```
tcsh
module load fre/bronx-19
```
Modify xml
```
vim OM4p25_JRA55do1.5_0netfw.xml
```
Change
> <property name="FRE_VERSION" value="bronx-19"
> <property name="PROD_SIMTIME" value="1"/>
```
frelist -x OM4p25_JRA55do1.5_0netfw.xml
fremake -p ncrc4.intel18 -t prod -x OM4p25_JRA55do1.5_0netfw.xml MOM6_SIS2_compile
sbatch /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223//MOM6_SIS2_compile/ncrc4.intel18-prod/exec/compile_MOM6_SIS2_compile.csh
```
```
frelist --directory=all --platform=ncrc4.intel18 --target=prod -x OM4p25_JRA55do1.5_0netfw.xml MOM6_SIS2_compile
```
* `/lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223//MOM6_SIS2_compile/ncrc4.intel18-prod/exec`
When you are all done, you should have a file in there called `fms_MOM6_SIS2_compile.x`
Run 1 year of OM4p25 JRA1.5
```
frerun -p ncrc4.intel18 -t prod -x OM4p25_JRA55do1.5_0netfw.xml OM4p25_JRA55do1.5_0netfw_cycle1
sbatch -t 20 -q debug /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p25_JRA55do1.5_0netfw_cycle1/ncrc4.intel18-prod/scripts/run/OM4p25_JRA55do1.5_0netfw_cycle1
sbatch /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p25_JRA55do1.5_0netfw_cycle1/ncrc4.intel18-prod/scripts/run/OM4p25_JRA55do1.5_0netfw_cycle1
```
```
frelist --directory=all --platform=ncrc4.intel18 --target=prod -x OM4p25_JRA55do1.5_0netfw.xml OM4p25_JRA55do1.5_0netfw_cycle1
```
* `/lustre/f2/scratch/gfdl/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p25_JRA55do1.5_0netfw_cycle1/ncrc4.intel18-prod/archive/ascii/19580101.ascii_out.tar/19580101.fms.out`
> MOM Date 1959/01/01 00:00:00 8760: En 2.255593E-03, MaxCFL 0.13386, Mass 1.381833347635E+21, Salt 34.72030187874, Temp 3.58886748162
> Total Energy: 43C5A0A53B9F59A6 3.1168542931305953E+18
> Total Mass: 1.3818333476352224E+21, Change: -3.9082447858311810E+11 Error: 1.78479E+02 ( 1.3E-19)
> Total Salt: 4.7977670976004710E+19, Change: -1.9541223997055433E+09 Error: -5.89756E+00 (-1.2E-19)
> Total Heat: 1.9797193331257074E+25, Change: 2.3926798001274605E+20 Error: 1.29432E+07 ( 6.5E-19)
> Total age: 1.3416794243561934E+21 yr kg
* `/archive/Brandon.Reichl/FMS2021.02_mom6_devgfdl_20221223/OM4p25_JRA55do1.5_0netfw_cycle1/gfdl.ncrc4-intel18-prod/ascii/19580101.ascii_out.tar/19580101.fms.out`
> MOM Date 1959/01/01 00:00:00 8760: En 2.255593E-03, MaxCFL 0.13386, Mass 1.381833347635E+21, Salt 34.72030187874, Temp 3.58886748162
> Total Energy: 43C5A0A53B9F59A6 3.1168542931305953E+18
> Total Mass: 1.3818333476352224E+21, Change: -3.9082447858311810E+11 Error: 1.78479E+02 ( 1.3E-19)
> Total Salt: 4.7977670976004710E+19, Change: -1.9541223997055433E+09 Error: -5.89756E+00 (-1.2E-19)
> Total Heat: 1.9797193331257074E+25, Change: 2.3926798001274605E+20 Error: 1.29432E+07 ( 6.5E-19)
> Total age: 1.3416794243561934E+21 yr kg
## OM4p5 xml
```
cp OM4p25_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw.xml
```
Edit OM4p5_JRA55do1.5_0netfw.xml
```
vim OM4p5_JRA55do1.5_0netfw.xml
```
```
frelist -x OM4p5_JRA55do1.5_0netfw.xml
```
```
frelist --directory=all --platform=ncrc4.intel18 --target=prod -x OM4p5_JRA55do1.5_0netfw.xml MOM6_SIS2_compile
frerun -p ncrc4.intel18 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
```
sbatch -t 20 -q debug /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/ncrc4.intel18-prod/scripts/run/OM4p5_JRA55do1.5_0netfw_cycle1
frelist --directory=all --platform=ncrc4.intel18 --target=prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
Standard output in
* `/lustre/f2/scratch/gfdl/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/ncrc4.intel18-prod/stdout/run`
```
frerun -o -p ncrc4.intel18 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
JRA river runoff
* `/lustre/f2/dev/Niki.Zadeh/archive/input/xanadu/Runoff_JRAv1.4_OM4p5`
## Generate river runoff for JRA 1.5 p5
Clone [this](https://github.com/raphaeldussin/OM4_025_runoff_JRA/tree/JRAv1.5.0.1) repo:
```
cd /archive/Jan-erik.Tesdal
git clone --recursive https://github.com/raphaeldussin/OM4_025_runoff_JRA.git
cd OM4_025_runoff_JRA
```
Branch **1.5.0** is for period 1958-2019 and **1.5.0.1** for period 2020 onwards. Start with 1.5.0.1 branch.
```
git checkout JRAv1.5.0.1
```
Simply copy the `ocean_hgrid` and `ocean_mask` for OM4p5. You can find `ocean_hgrid.nc` and `ocean_mask.nc` for OM4p5 in `/archive/gold/datasets/OM4_05/mosaic_ocean.v20180227.unpacked`
```
cp /archive/gold/datasets/OM4_05/mosaic_ocean.v20180227.unpacked/ocean_hgrid.nc .
cp /archive/gold/datasets/OM4_05/mosaic_ocean.v20180227.unpacked/ocean_mask.nc .
```
Start `tmux` session:
```
csh
conda activate base
tmux
```
Create a conda env with the provided env file, and run make
```bash
conda create --name pad_jra --file conda_env_runoff
conda activate pad_jra
```
Before running the script you want to do a `dmget` of the `friver_*` and `licalv_*` files.
> JRA_DIR = /archive/gold/datasets/reanalysis/JRA55-do/v1.5.0.1/padded
> JRA_FILES = $(wildcard $(JRA_DIR)/friver_*.nc $(JRA_DIR)/licalvf_*.nc)
```
dmget /archive/gold/datasets/reanalysis/JRA55-do/v1.5.0.1/padded/friver_*.nc
dmget /archive/gold/datasets/reanalysis/JRA55-do/v1.5.0.1/padded/licalvf_*.nc
```
Generate JRA runoff files
```
make
```
Update checksums
```
md5sum *compressed.nc > hash.md5
```
Tar up the files
```
mkdir input
cp *.compressed.nc input/
tar -chvzf input.tar.gz input/*
```
On GAEA create a new directory and transfer the files
```
cd /lustre/f2/dev/gfdl/Jan-erik.Tesdal/input
mkdir Runoff_JRAv1.5_OM4p5
cd Runoff_JRAv1.5_OM4p5
module load gcp
gcp -v gfdl:/archive/Jan-erik.Tesdal/OM4_025_runoff_JRA/input.tar.gz .
tar -xvzf input.tar.gz
mv input/* .
rm -fr input
rm input.tar.gz
```
Fork Raf's repo and set your fork as the origin. On PP/AN:
```
git remote set-url origin https://github.com/jetesdal/OM4_05_runoff_JRA.git
git remote add upstream https://github.com/raphaeldussin/OM4_025_runoff_JRA.git
```
Update repo:
```
rm -fr input
rm input.tar.gz
git add hash.md5
git commit -m "Updated hash.md5"
git push origin JRAv1.5.0.1
```
Switch to branch 1.5.0
```
git checkout JRAv1.5
```
Edit `Makefile` to change `JRA_DIR` to
> JRA_DIR = /archive/gold/datasets/reanalysis/JRA55-do/v1.5.0_no_null_char/padded
Before running `make` you want to do a `dmget` of the `friver_*` and `licalv_*` files.
```
dmget /archive/gold/datasets/reanalysis/JRA55-do/v1.5.0_no_null_char/padded/friver_*.nc
dmget /archive/gold/datasets/reanalysis/JRA55-do/v1.5.0_no_null_char/padded/licalvf_*.nc
```
Tar up the files
```
mkdir input
cp *.compressed.nc input/
tar -chvzf input.tar.gz input/*
```
On GAEA transfer the files
```
cd /lustre/f2/dev/gfdl/Jan-erik.Tesdal/input/Runoff_JRAv1.5_OM4p5
module load gcp
gcp -v gfdl:/archive/Jan-erik.Tesdal/OM4_025_runoff_JRA/input.tar.gz .
tar -xvzf input.tar.gz
mv input/* .
rm -fr input
rm input.tar.gz
```
Update directory in `OM4p5_JRA55do1.5_0netfw.xml`:
> <property name="JRA_dataset_p5" value="/lustre/f2/dev/gfdl/Jan-erik.Tesdal/input/Runoff_JRAv1.5_OM4p5"/>
Regenerate runscript:
```
cd /ncrc/home1/Jan-erik.Tesdal/om4_jra
frerun -o -p ncrc4.intel18 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
Check output:
```
frelist --directory=all --platform=ncrc4.intel18 --target=prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
On GAEA Model output is in
```
/lustre/f2/scratch/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/ncrc4.intel18-prod/archive
```
On PP/AN model output is
```
/archive/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc4-intel18-prod
```
## Update rho2 coords in xml
### (1) MOM override section
> #override NUM_DIAG_COORDS = 3
> #override DIAG_COORDS = "z Z ZSTAR", "rho2 RHO2 RHO", "rho2l35 RHO2L35 RHO"
> #override DIAG_COORD_DEF_Z = "WOA09"
> #override DIAG_COORD_DEF_RHO2 = "FILE:diag_rho2_74layer.nc,interfaces=rho2"
> #override DIAG_COORD_DEF_RHO2L35 = "RFNC1:35,999.5,1028,1028.5,8.,1038.,0.0078125"
### (2) Add file and definition for rho2 coords
<dataFile label="input" target="INPUT/" chksum="" size="" timestamp="">
<dataSource platform="$(platform)">/lustre/f2/dev/Raphael.Dussin/input/coords/diag_rho2_74layer.nc</dataSource>
</dataFile>
### (3) Add diagnostics in diagTable append:
> "ocean_annual_rho2l35", 12, "months", 1, "days", "time"
> "ocean_model_rho2l35", "volcello", "volcello", "ocean_annual_rho2l35", "all", "mean", "none",2
> "ocean_model_rho2l35", "umo", "umo", "ocean_annual_rho2l35", "all", "mean", "none",2
> "ocean_model_rho2l35", "vmo", "vmo", "ocean_annual_rho2l35", "all", "mean", "none",2
### (4) Add postprocessing for `ocean_annual_rho2l35` in `awg_include/xml_building_blocks/om4p25_postprocess.xml`
```xml
<component type="ocean_annual_rho2l35" start="$(PP_START_YEAR)" source="ocean_annual_rho2l35" sourceGrid="ocean-tripolar">
<timeSeries freq="annual" chunkLength="$(CHUNK_LENGTH_A)" source="ocean_annual_rho2l35"/>
<timeAverage source="annual" interval="$(CHUNK_LENGTH_A)"/>
</component>
```
## Switch to C5
To update to c5 use `fre/bronx-20`.
Note, when using `bronx-20`, you have to comment out (or remove) all the the `gnu` based compiler options in the main XML, otherwise you will get an error like
```
*************************** 1 XML validation errors ****************************
awg_include/xml_building_blocks/platforms.xml:
Line 150 - Element 'compiler', attribute 'type': 'gnu' is not a valid value of the atomic type 'compilerType'.
Line 150 - Element 'compiler', attribute 'type': [facet 'enumeration'] The value 'gnu' is not an element of the set {'intel', 'intel-classic', 'intel-oneapi', 'pgi', 'gcc', 'cce', 'nvhpc'}.
********************************************************************************
*FATAL*: The XML file '/ncrc/home1/Jan-erik.Tesdal/om4_jra/OM4p25_JRA55do1.5_0netfw.xml' is not valid
*FATAL*: The XML file '/ncrc/home1/Jan-erik.Tesdal/om4_jra/OM4p25_JRA55do1.5_0netfw.xml' can't be parsed
```
Edit `OM4p5_JRA55do1.5_0netfw.xml`, in the <setup> block add:
```xml
<xi:include href="awg_include/xml_building_blocks/platforms.xml" xpointer="xpointer(//freInclude/platform[@name='ncrc5.intel22'])"/>
<xi:include href="awg_include/xml_building_blocks/platforms.xml" xpointer="xpointer(//freInclude/platform[@name='gfdl.ncrc5-intel22'])"/>
```
In `awg_include/xml_building_blocks/platforms.xml`, add:
```xml
<platform name="ncrc5.intel22">
<freVersion>$(FRE_VERSION)</freVersion>
<compiler type="intel-classic" version="2022.0.2"/>
<project>$(GFDL_GROUP)</project>
<directory stem="$(FRE_STEM)">
<root>$DEV/$USER/$(FRE_STEM)</root>
<scripts>$DEV/$USER/$(FRE_STEM)$(DEBUGLEVEL)/$(name)/$(platform)-$(target)/scripts</scripts>
<state>$DEV/$USER/$(FRE_STEM)$(DEBUGLEVEL)/$(name)/$(platform)-$(target)/state</state>
<src>$DEV/$USER/$(FRE_STEM)/$(BUILD_DATE)/$(name)/src</src>
<exec>$DEV/$USER/$(FRE_STEM)/$(BUILD_DATE)/$(name)/$(platform)-$(target)/exec</exec>
<archive>$SCRATCH/$USER/$(stem)$(DEBUGLEVEL)/$(name)/$(platform)-$(target)/archive</archive>
<stdout>$SCRATCH/$USER/$(stem)$(DEBUGLEVEL)/$(name)/$(platform)-$(target)/stdout</stdout>
<work>$SCRATCH/$USER/work/$(stem)/$FRE_JOBID</work>
</directory>
<property name="EXEC_ROOT" value="$DEV/$USER/$(stem)/$(BUILD_DATE)"/>
<property name="F2003_FLAGS" value=" -DINTERNAL_FILE_NML -g "/>
<property name="AWG_INPUT_HOME" value="$(AWG_INPUT_HOME_GAEA)"/>
<property name="NB_ROOT" value="/nbhome/$USER/$(FRE_STEM)$(PLATINFO)/$(name)"/>
<property name="ARCHIVE_NNZ" value="/lustre/f2/dev/Niki.Zadeh/archive"/>
<property name="REFERENCE" value="/lustre/f2/dev/Niki.Zadeh/archive/answers/$(reference_tag)/$(name)/$(platform)-$(target)"/>
<property name="FMS_ARCHIVE_ROOT" value="$PDATA/gfdl/fms"/>
<property name="CMIP6_ARCHIVE_ROOT" value="$PDATA/gfdl/cmip6/datasets"/>
<property name="NCORE_PER_NODE" value="36"/>
<property name="NPES_IO_LAND" value="18"/>
<csh><![CDATA[
module load git
module unload cray-libsci
module unload darshan
$(append_to_setup_csh)
]]></csh>
</platform>
</freInclude>
```
Also in the same file, add:
```xml
<platform name="gfdl.ncrc5-intel22">
<xi:include xpointer="xpointer(//freInclude/platform[@name='gfdl.default']/node())" />
</platform>
```
Recompile model with `--plattform/-p ncrc5.intel22` on a C5 login node (`gaea5*`).
```bash
# Run fremake
fremake -p ncrc5.intel22 -t prod -x OM4p5_JRA55do1.5_0netfw.xml MOM6_SIS2_compile
# Run compile script
sbatch /ncrc/home1/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223//MOM6_SIS2_compile/ncrc5.intel22-prod/exec/compile_MOM6_SIS2_compile.csh
```
Look at directories
```
frelist --directory=all --platform=ncrc5.intel22 --target=prod -x OM4p5_JRA55do1.5_0netfw.xml MOM6_SIS2_compile
```
If the compiling successfully completed, you should see the file `fms_MOM6_SIS2_compile.x`
```
ls -lah /ncrc/home1/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/MOM6_SIS2_compile/ncrc5.intel22-prod/exec/fms_MOM6_SIS2_compile.x
```
Next we generate the runscript
```
frerun -p ncrc5.intel22 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
1-year test run
```
sbatch /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/ncrc5.intel22-prod/scripts/run/OM4p5_JRA55do1.5_0netfw_cycle1
```
Check output:
```
frelist --directory=all --platform=ncrc5.intel22 --target=prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
On GAEA Model output is in
```
/lustre/f2/scratch/gfdl/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/ncrc5.intel22-prod/archive
```
On PP/AN model output is
```
/archive/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc5-intel22-prod
```
Compare checksums between C4 and C5:
* `gfdl.ncrc5-intel22-prod`
> MOM Date 1958/12/27 00:00:00
> Total Energy: 43BCD5DFB40A3BC2
> Total Mass: 1.3815353609147614E+21
> Total Salt: 4.7967439731286311E+19
> Total Heat: 1.9798415209849202E+25
* `gfdl.ncrc4-intel18-prod`
> MOM Date 1958/12/27 00:00:00
> Total Energy: 43BCE8227C8E2DAA
> Total Mass: 1.3815353603132348E+21
> Total Salt: 4.7967439728278675E+19
> Total Heat: 1.9798573745570062E+25
## Run complete simulation
JRA 1.5 goes to 2019 and we have the 1.5.0.1 extension for the most recent years (2020-2022, 2023 partial at this time). Thus, each cycle should be 65 years long (1958-2022) or 62 years long (1958-2019).
> <property name="PROD_SIMTIME" value="62"/>
Regenerate the runscript
```
frerun -e -p ncrc5.intel22 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
Submit run
```
sbatch /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/ncrc5.intel22-prod/scripts/run/OM4p5_JRA55do1.5_0netfw_cycle1
```
### Last three years (2020-2022)
Note: You need to rename the files in
```
/lustre/f2/dev/gfdl/Jan-erik.Tesdal/input/Runoff_JRAv1.5_OM4p5
```
```
vas_input4MIPs_atmosphericState_OMIP_MRI-JRA55-do-1-5-0-1_gr_202001010000-202012312100.padded.nc
vas_input4MIPs_atmosphericState_OMIP_MRI-JRA55-do-1-5-0-1_gr_202101010000-202112312100.padded.nc
vas_input4MIPs_atmosphericState_OMIP_MRI-JRA55-do-1-5-0-1_gr_202201010000-202212312100.padded.nc
```
Change
`1-5-0-1` --> `1-5-0`
Renamed files
```
vas_input4MIPs_atmosphericState_OMIP_MRI-JRA55-do-1-5-0_gr_202001010000-202012312100.padded.nc
vas_input4MIPs_atmosphericState_OMIP_MRI-JRA55-do-1-5-0_gr_202101010000-202112312100.padded.nc
vas_input4MIPs_atmosphericState_OMIP_MRI-JRA55-do-1-5-0_gr_202201010000-202212312100.padded.nc
```
In the xml change
> <property name="JRA_version" value="1-5-0"/>
full run
> <property name="PROD_SIMTIME" value="65"/>
Regenerate the runscript
```
frerun -e -p ncrc5.intel22 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
## Postprocessing
On PP/AN, model output is located in
```
/archive/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc5-intel22-prod/
```
Global mean metrics and postprocessing
```
/home/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc5-intel22-prod
```
### Redo postprocessing
Remove state files
```
rm -f /home/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc5-intel22-prod/state/postProcess/ocean_*
rm -f /home/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc5-intel22-prod/state/postProcess/ice*
```
Remove postprocessing script
```
rm -f /home/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc5-intel22-prod/scripts/postProcess/OM4p5_JRA55do1.5_0netfw_cycle1_*
```
Generate postprocessing scripts for the first 5-year chunk
```
# Load FRE
module load fre/bronx-20
# Run frepp
frepp -s -c split -P gfdl.ncrc5-intel22 -T prod -t 19620101 -d /archive/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/gfdl.ncrc5-intel22-prod/history -x /home/Jan-erik.Tesdal/ncrc/om4_jra/OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle1
```
<p style='color:red; font-size:20px; font-weight:bold;'>Section transports are missing:</p>
> ERROR: No history files matching $hDate*/*.ocean_Davis_Strait.nc
> ERROR: No history files matching $hDate*/*.ocean_Indonesian_Throughflow.nc
# OM4p5_JRA55do1.5_0netfw_cycle2
- Copy restart file
```bash
cd /lustre/f2/dev/gfdl/Jan-erik.Tesdal/ICgen
cp /lustre/f2/scratch/gfdl/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle1/ncrc5.intel22-prod/archive/restart/20230101.tar .
mv 20230101.tar OM4p5_JRA55do1.5_0netfw_cycle1.20230101.tar
```
- Update path in the `initCond` statement:
```bash
cd ~/om4_jra
vim OM4p5_JRA55do1.5_0netfw.xml
```
> <dataFile label="initCond" target="INPUT/" chksum="" size="" timestamp="">
> <dataSource platform="$(platform)">/lustre/f2/dev/gfdl/Jan-erik.Tesdal/ICgen/OM4p5_JRA55do1.5_0netfw_cycle1.20230101.tar</dataSource>
> </dataFile>
- Make sure that you have `input_filename = 'r'` such that restart is used even for first job:
> <namelist name="MOM_input_nml" >
> output_directory = './',
> input_filename = 'r'
> restart_input_dir = 'INPUT/',
> restart_output_dir = 'RESTART/',
> parameter_filename = 'INPUT/MOM_input','INPUT/MOM_layout','INPUT/MOM_override','INPUT/MOM_saltrestore'
> </namelist>
> <namelist name="SIS_input_nml" >
> output_directory = './',
> input_filename = 'r'
> restart_input_dir = 'INPUT/',
> restart_output_dir = 'RESTART/',
> parameter_filename = 'INPUT/SIS_input','INPUT/SIS_layout','INPUT/SIS_override'
> </namelist>
- Make sure the xml includes statement to delete `coupler.res`
> if ( $currentSeg == 1 ) then
> #Take current date from coupler_nml
> if(-e $work/INPUT/coupler.res ) \rm -rf $work/INPUT/coupler.res
> endif
- Change to 1-year run
> <property name="PROD_SIMTIME" value="1"/>
- Generate runscript
```bash
frerun -p ncrc5.intel22 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle2
```
- Submit runscript
```bash
sbatch /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle2/ncrc5.intel22-prod/scripts/run/OM4p5_JRA55do1.5_0netfw_cycle2
```
First, try with `sbatch -t 20 -q debug`
Compare checksums between cycle 1 and 2:
* cycle 1
> MOM Date 1958/12/27 00:00:00
> Total Energy: 43BCD5DFB40A3BC2
* cycle 2
> MOM Date 1958/12/27 00:00:00
> Total Energy: 43BF7C6202ACAC02
* cycle 3
> MOM Date 1958/12/27 00:00:00
> Total Energy: 43BEF137E25DFECA
## Run complete simulation
> <property name="PROD_SIMTIME" value="62"/>
Regenerate the runscript
```
frerun -e -p ncrc5.intel22 -t prod -x OM4p5_JRA55do1.5_0netfw.xml OM4p5_JRA55do1.5_0netfw_cycle2
```
Submit run
```
sbatch /lustre/f2/dev/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle2/ncrc5.intel22-prod/scripts/run/OM4p5_JRA55do1.5_0netfw_cycle2
```
## Postprocessing
On PP/AN, model output is located in
```
/archive/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle2/gfdl.ncrc5-intel22-prod/
```
Global mean metrics and postprocessing
```
/home/Jan-erik.Tesdal/FMS2021.02_mom6_devgfdl_20221223/OM4p5_JRA55do1.5_0netfw_cycle2/gfdl.ncrc5-intel22-prod/
```
<p style='color:red; font-size:20px; font-weight:bold;'>Diagnotics are missing:</p>
* `zos`, `tos` and `sos` in `ocean_annual`
Sign in with Wallet
Connect another wallet