The following steps have been tested on `iceberg03.fnal.gov` under user `dunesw`.

## Install dependencies

Dependencies are installed via spack. Note that during the installation, I had to change the ownership of `/var/log/nvidia` to `dunesw` (or made it writable to `dunesw`) to allow `spack install cuda` to proceed correctly.

### Install spack

git clone --depth=100 --branch=releases/v0.20 https://github.com/spack/spack.git ~/dingpf/spack

### Apply changes to spack configuration files

diff --git a/etc/spack/defaults/config.yaml b/etc/spack/defaults/config.yaml
index 43f8a98d..30c579ec 100644
--- a/etc/spack/defaults/config.yaml
+++ b/etc/spack/defaults/config.yaml
@@ -93,7 +93,7 @@ config:
   # Timeout in seconds used for downloading sources etc. This only applies
   # to the connection phase and can be increased for slow connections or
   # servers. 0 means no timeout.
-  connect_timeout: 10
+  connect_timeout: 100


   # If this is false, tools like curl that use SSL will not verify
@@ -203,7 +203,7 @@ config:
     # dependencies. Their type can either be "rpath" or "runpath". For glibc, rpath is
     # inherited and has precedence over LD_LIBRARY_PATH; runpath is not inherited
     # and of lower precedence. DO NOT MIX these within the same install tree.
-    type: rpath
+    type: runpath


     # (Experimental) Embed absolute paths of dependent libraries directly in ELF
diff --git a/etc/spack/defaults/packages.yaml b/etc/spack/defaults/packages.yaml
index 22b76626..23875c65 100644
--- a/etc/spack/defaults/packages.yaml
+++ b/etc/spack/defaults/packages.yaml
@@ -62,3 +62,13 @@ packages:
     permissions:
       read: world
       write: user
+  libxpm:
+    externals:
+    - spec: libxpm@3.5.12%gcc@11.3.1
+      prefix: /usr
+    buildable: False
+  python:
+    externals:
+    - spec: python@3.9.16%gcc@11.3.1
+      prefix: /usr
+    buildable: False

### Install dependencies

source ~/dingpf/spack/share/spack/setup-env.sh
spack install --reuse root@6.28.04%gcc@11.3.1^python@3.9.16 \ 
py-numpy@1.24.3^python@3.9.16 \
opencv@4.6.0+imgcodecs+cuda+python3+ml+cudnn+highgui+imgproc^python@3.9.16
 

## Build `larcv2`

### Obtain `larcv2` source

cd ~/dingpf
git clone https://github.com/DeepLearnPhysics/larcv2.git
# Current head of develop points to tag: v2_1_2
cd larcv2

### Apply patches to `larcv2` source

diff --git a/larcv/core/CVUtil/CVUtil.cxx b/larcv/core/CVUtil/CVUtil.cxx
index 75161fe..c88b886 100644
--- a/larcv/core/CVUtil/CVUtil.cxx
+++ b/larcv/core/CVUtil/CVUtil.cxx
@@ -123,7 +123,7 @@ namespace larcv {
   Image2D imread(const std::string file_name)
   {
     ::cv::Mat image;
-    image = ::cv::imread(file_name.c_str(), CV_LOAD_IMAGE_COLOR);
+    image = ::cv::imread(file_name.c_str(), cv::IMREAD_COLOR);

     ImageMeta meta(image.cols,image.rows,image.cols, image.rows, 0., 0.);
     Image2D larcv_img(meta);
@@ -147,7 +147,7 @@ namespace larcv {
   Image2D imread_gray(const std::string file_name)
   {
     ::cv::Mat image;
-    image = ::cv::imread(file_name.c_str(), CV_LOAD_IMAGE_GRAYSCALE);
+    image = ::cv::imread(file_name.c_str(), cv::IMREAD_GRAYSCALE);

     ImageMeta meta(image.cols,image.rows,image.cols, image.rows, 0., 0.);
     Image2D larcv_img(meta);

### Build `larcv2`

#### Find out the path to the `opencv` installation

[dunesw@iceberg03 dingpf]$ spack find -p opencv
-- linux-almalinux9-icelake / gcc@11.3.1 ------------------------
opencv@4.6.0  /home/dunesw/dingpf/spack/opt/spack/linux-almalinux9-icelake/gcc-11.3.1/opencv-4.6.0-tkgbpxskrub4v2xzxscru3soiopot3x3
==> 1 installed package

#### Load dependencies and set ENVs required for building `larcv2`

spack load root
spack load opencv

export OPENCV_LIBDIR=/home/dunesw/dingpf/spack/opt/spack/linux-almalinux9-icelake/gcc-11.3.1/opencv-4.6.0-tkgbpxskrub4v2xzxscru3soiopot3x3/lib64
export OPENCV_INCDIR=/home/dunesw/dingpf/spack/opt/spack/linux-almalinux9-icelake/gcc-11.3.1/opencv-4.6.0-tkgbpxskrub4v2xzxscru3soiopot3x3/include/opencv4

#### Build `larcv2`

cd ~/dingpf/larcv2
source configure.sh
make -j 50

## Next steps

In `larcv2` source package, there is `ups/create_ups_release.sh` which creates a UPS package for the version built above.

The script creates a skeleton UPS package first, and then copy `bin, lib, include` from the build directory (`larcv2/build`) into the UPS package.

It should be straight-forward to write a spack recipe file (`package.py`) for `larcv2`, which does the following things:
1. sets up dependencies for `larcv2`;
2. set environment variables for `opencv`;
3. run the build steps;
4. install the `bin, lib, include` directories.

With that recipe file in place, installing and using `larcv2` can be as simple as `spack install larcv2@<version>` and `spack load larcv2`.