---
# System prepended metadata

title: Using Spack Infrastructure
tags: [DUNE-DAQ, spack]

---

# Using Spack Infrastructure

###### tags: `DUNE-DAQ` `spack`

## Convert existing UPS products area

### Test setup

* Using UPS products put into the nighly docker image used by CI jobs. The nightly docker image is: `dunedaq/sl7-minimal-nightly:N21-12-26`;
* This image is based on `dunedaq/sl7-minimal:latest` with addtional cvmfs mirrors containing UPS products and the latest DAQ development release;
* The tests in this note will be all done with running `dingpf/sl7` ~~(may be `dunedaq/sl7-minimal:latest`?)~~ docker container with the same cvmfs mirror found in the nightly image, running on a host machine with `Ubuntu 20.04.1 LTS`.  

### Details of the test

* The test is done by following [this documentation](https://cdcvs.fnal.gov/redmine/projects/spack-infrastructure/wiki/Converting);
* Note: there is a difference `spack infrastructure` repo in the official spack org; the equivelant one to that in Fermilab Redmine is under Marc Mengel's personal Github account [here](https://github.com/marcmengel/spack-infrastructure);
* Another note: the latest tagged version is `v2_16_01_01`, where `v2_16_01` is the tag of `spack` the package targets towards.
* tested with `dunedaq/sl7-minimal:latest`, but got prompt that the following RPMs are needed when installing `bzip2`?

```
==> patchelf: Successfully installed patchelf-0.9-3kdvjmz4caxq7g4wwkecdyp6gsqo32ti
  Fetch: 1.01s.  Build: 6.99s.  Total: 7.99s.
[+] /cvmfs/dunedaq.opensciencegrid.org/products/patchelf/0.9/linux-scientific7-x86_64-gcc-8.2.0-3kdvjmz4caxq7g4wwkecdyp6gsqo32ti
==> Warning: No manifest file in tarball for spec bzip2/rbgyavo
==> Error: No compilers for operating system scientific7 satisfy spec gcc@4.8.5
make_packages_yaml: INFO: creating /cvmfs/dunedaq.opensciencegrid.org/products/spack/v0.16.1.fermi/NULL/etc/spack/scientific7/packages.yaml
Need to install rpm: expat-devel
Need to install rpm: gdbm-devel
Need to install rpm: gettext-devel
Need to install rpm: libdrm-devel
Need to install rpm: libedit-devel
Need to install rpm: libffi-devel
Need to install rpm: libgcrypt-devel
Notice: no version found for package libjpeg-turbo
Need to install rpm: libpciaccess-devel
Need to install rpm: libpng-devel
Need to install rpm: lz4-devel
Need to install rpm: ncurses-devel
Need to install rpm: pcre2-devel
Need to install rpm: readline-devel
[root@280b06af7721 scratch]#

```

#### Step 1

Start the docker container `dingpf/sl7` with cvmfs mirrors found in `dunedaq/sl7-minimal-nightly:N21-12-26`.

#### Step 2

Obtain `spack_infrastructure` by copying over `spack_infrastructure` UPS products from `/cvmfs/fermilab.opensciencegrid.org/products/` to the local products area.

#### Step 3

Put a spcak instance into UPS products area.

```
P1=/cvmfs/dunedaq.opensciencegrid.org/products; 
source $P1/setup
setup spack_infrastructure
setup gcc v8_2_0
make_spack -u $P1
```

Failed with the following errors:

```
==> Error: invalid syntax in /cvmfs/dunedaq.opensciencegrid.org/products/spack/v0.16.1.fermi/NULL/var/spack/repos/fnal_art/packages/gm2pip/package.py, line 96
^Bmake_packages_yaml: INFO: creating /cvmfs/dunedaq.opensciencegrid.org/products/spack/v0.16.1.fermi/NULL/etc/spack/scientific7/packages.yaml
Need to install rpm: gettext-devel
Need to install rpm: libedit-devel
Need to install rpm: libgcrypt-devel
Need to install rpm: libpciaccess-devel
Need to install rpm: lz4-devel
Need to install rpm: pcre2-devel
```

Installed these RPMs with `yum install -y gettext-devel libedit-devel libgcrypt-devel libpciaccess-devel lz4-devel pcre2-devel`
        
Rerun `make_spack -u $P1`

Note: although I did `setup gcc v8_2_0`, make_spack still installs things for the system `gcc 4.8.5` and `python 2.7`.

Might worth trying with setting up both gcc and python UPS products...

Got error
```
==> Error: invalid syntax in /cvmfs/dunedaq.opensciencegrid.org/products/spack/v0.16.1.fermi/NULL/var/spack/repos/fnal_art/packages/gm2pip/package.py, line 96
make_packages_yaml: INFO: creating /cvmfs/dunedaq.opensciencegrid.org/products/spack/v0.16.1.fermi/NULL/etc/spack/scientific7/packages.yaml

```

That line seems to be just comments, removed the whole comments section, and rerun `make_spack -u $P1`.


#### Step 4

Add Spack declarations for UPS products
```
P1=/cvmfs/dunedaq.opensciencegrid.org/products; 
source $P1/setup
source $P1/setup-env.sh 
ups_to_spack -a 
spack reindex
```

`spack find` shows the following

```
[root@e1515878b09d scratch]# spack find
==> 87 installed packages
-- linux-scientific7-x86_64 / gcc@4.8.5 -------------------------
bzip2@1.0.8             openssl@1.1.1c
bzip2@1.0.8             patchelf@0.9
clang-runtime@7.0.0rc3  patchelf@0.9
cmake@3.17.2            python@2.7.16
expat@2.2.5             readline@7.0
expat@2.2.5             readline@7.0
gcc-runtime@9.3.0       spack-infrastructure@1.0
gdb@9.2                 spack-infrastructure@1.2
gdbm@1.18.1             spack-infrastructure@1.4
gdbm@1.18.1             spack-infrastructure@1.5
gettext@0.19.8.1        spack-infrastructure@1.6
gettext@0.19.8.1        spack-infrastructure@1.7
libbsd@0.9.1            spack-infrastructure@2.16.01
libbsd@0.9.1            spack-infrastructure@2.16.01.01
libffi@3.2.1            sqlite@3.29.0
libffi@3.2.1            sqlite@3.29.0
libiconv@1.15           tar@1.31
libiconv@1.15           tar@1.31
libxml2@2.9.9           trace@3.16.02
libxml2@2.9.9           ups@6.0.8
ncurses@6.1             xz@5.2.4
ncurses@6.1             xz@5.2.4
ninja@1.10.0            zlib@1.2.11
openssl@1.1.1c          zlib@1.2.11

-- linux-scientific7-x86_64 / gcc@8.2.0 -------------------------
boost@1.73.0             fmt@6.2.1                msgpack-c@3.3.0
boost@1.73.0             folly@2021.12.13         nlohmann-json@3.9.0c
cetlib@3.11.01           gcc-runtime@8.2.0        pistache@2020.10.07
cetlib@3.11.01           glog@0.4.0               pistache@2020.10.07
cetlib-except@1.05.00    hdf5@1.12.0a             protobuf@3.14.0
cetlib-except@1.05.00    hdf5@1.12.0a             pugixml@1.11
cetpkgsupport@1.14.01    hep-concurrency@1.05.00  pybind11@2.6.2
cppunit@1.15.1a          hep-concurrency@1.05.00  python@3.8.3b
cppunit@1.15.1a          highfive@2.3.1           sqlite@3.32.03.00
cppzmq@4.3.0             intel-tbb@2020.2a        uhal@2.8.1
cpr@1.5.2                intel-tbb@2020.2a        yaml-cpp@0.6.3
double-conversion@3.1.5  libevent@2.1.8           zmq@4.3.1c
felix@1.2.0              librdkafka@1.7.0         zstd@1.5.0
```

This is the list of UPS products.

```
[root@e1515878b09d scratch]# ups list -aK+
"boost" "v1_73_0" "Linux64bit+3.10-2.17" "e19:prof" ""
"boost" "v1_73_0" "Linux64bit+3.10-2.17" "debug:e19" ""
"bzip2" "1.0.8" "Linux64bit+3.10-2.17" "" ""
"cetlib" "v3_11_01" "Linux64bit+3.10-2.17" "e19:prof" ""
"cetlib" "v3_11_01" "Linux64bit+3.10-2.17" "debug:e19" ""
"cetlib_except" "v1_05_00" "Linux64bit+3.10-2.17" "e19:prof" ""
"cetlib_except" "v1_05_00" "Linux64bit+3.10-2.17" "debug:e19" ""
"cetpkgsupport" "v1_14_01" "NULL" "" "current"
"clang" "v7_0_0rc3" "Linux64bit+3.10-2.17" "" ""
"cmake" "v3_17_2" "Linux64bit+3.10-2.17" "" ""
"cppunit" "v1_15_1a" "Linux64bit+3.10-2.17" "e19:prof" ""
"cppunit" "v1_15_1a" "Linux64bit+3.10-2.17" "debug:e19" ""
"cppzmq" "v4_3_0" "Linux64bit+3.10-2.17" "e19:prof" ""
"cpr" "v1_5_2" "Linux64bit+3.10-2.17" "e19:prof" ""
"double_conversion" "v3_1_5" "Linux64bit+3.10-2.17" "e19:prof" ""
"expat" "2.2.5" "Linux64bit+3.10-2.17" "" ""
"felix" "v1_2_0" "Linux64bit+3.10-2.17" "e19:prof" ""
"fmt" "v6_2_1" "Linux64bit+3.10-2.17" "e19:prof" ""
"folly" "v2021_12_13" "Linux64bit+3.10-2.17" "e19:prof" ""
"gcc" "v8_2_0" "Linux64bit+3.10-2.17" "" ""
"gcc" "v9_3_0" "Linux64bit+3.10-2.17" "" ""
"gdb" "v9_2" "Linux64bit+3.10-2.17" "" "current"
"gdbm" "1.18.1" "Linux64bit+3.10-2.17" "" ""
"gettext" "0.19.8.1" "Linux64bit+3.10-2.17" "" ""
"glog" "v0_4_0" "Linux64bit+3.10-2.17" "e19:prof" ""
"hdf5" "v1_12_0a" "Linux64bit+3.10-2.17" "e19:prof" ""
"hdf5" "v1_12_0a" "Linux64bit+3.10-2.17" "debug:e19" ""
"hep_concurrency" "v1_05_00" "Linux64bit+3.10-2.17" "e19:prof" ""
"hep_concurrency" "v1_05_00" "Linux64bit+3.10-2.17" "debug:e19" ""
"highfive" "v2_3_1" "Linux64bit+3.10-2.17" "e19:prof" ""
"libbsd" "0.9.1" "Linux64bit+3.10-2.17" "" ""
"libevent" "v2_1_8" "Linux64bit+3.10-2.17" "e19:prof" ""
"libffi" "3.2.1" "Linux64bit+3.10-2.17" "" ""
"libiconv" "1.15" "Linux64bit+3.10-2.17" "" ""
"librdkafka" "v1_7_0" "Linux64bit+3.10-2.17" "e19:prof" ""
"libxml2" "2.9.9" "Linux64bit+3.10-2.17" "" ""
"msgpack_c" "v3_3_0" "Linux64bit+3.10-2.17" "e19:prof" ""
"ncurses" "6.1" "Linux64bit+3.10-2.17" "" ""
"ninja" "v1_10_0" "Linux64bit+3.10-2.17" "" "current"
"nlohmann_json" "v3_9_0c" "Linux64bit+3.10-2.17" "e19:prof" ""
"openssl" "1.1.1c" "Linux64bit+3.10-2.17" "" ""
"patchelf" "0.9" "Linux64bit+3.10-2.17" "" ""
"pistache" "v2020_10_07" "Linux64bit+3.10-2.17" "e19:prof" ""
"pistache" "v2020_10_07" "Linux64bit+3.10-2.17" "debug:e19" ""
"protobuf" "v3_14_0" "Linux64bit+3.10-2.17" "e19" ""
"pugixml" "v1_11" "Linux64bit+3.10-2.17" "e19:prof" ""
"pybind11" "v2_6_2" "Linux64bit+3.10-2.17" "e19:prof" ""
"python" "v3_8_3b" "Linux64bit+3.10-2.17" "" "current"
"readline" "7.0" "Linux64bit+3.10-2.17" "" ""
"spack_infrastructure" "v1_0" "NULL" "" ""
"spack_infrastructure" "v2_16_01" "NULL" "" ""
"spack_infrastructure" "v1_6" "NULL" "" ""
"spack_infrastructure" "v2_16_01_01" "NULL" "" "current"
"spack_infrastructure" "v1_4" "NULL" "" ""
"spack_infrastructure" "v1_2" "NULL" "" ""
"spack_infrastructure" "v1_5" "NULL" "" ""
"spack_infrastructure" "v1_7" "NULL" "" ""
"sqlite" "3.29.0" "Linux64bit+3.10-2.17" "" ""
"sqlite" "v3_32_03_00" "Linux64bit+3.10-2.17" "" ""
"tar" "1.31" "Linux64bit+3.10-2.17" "" ""
"tbb" "v2020_2a" "Linux64bit+3.10-2.17" "e19:prof" ""
"tbb" "v2020_2a" "Linux64bit+3.10-2.17" "debug:e19" ""
"TRACE" "v3_16_02" "Linux64bit+3.10-2.17" "" ""
"uhal" "v2_8_1" "Linux64bit+3.10-2.17" "e19:prof" ""
"ups" "v6_0_8" "Linux64bit+3.10-2.17" "" "current"
"xz" "5.2.4" "Linux64bit+3.10-2.17" "" ""
"yaml_cpp" "v0_6_3" "Linux64bit+3.10-2.17" "e19:prof" ""
"zlib" "1.2.11" "Linux64bit+3.10-2.17" "" ""
"zmq" "v4_3_1c" "Linux64bit+3.10-2.17" "e19:prof" ""
"zstd" "v1_5_0" "Linux64bit+3.10-2.17" "e19:prof" ""
```

I saw no spack module for `gcc` was generated, but `gcc-runtime@8.2.0`. When I tried to load any module depending on `gcc 8.2.0`, I got:

```
[root@e1515878b09d scratch]# spack load zmq@4.3.1c
==> Error: No compilers for operating system scientific7 satisfy spec gcc@8.2.0
```

Doing `spack compilers` gives me `gcc@4.8.5` only.

Doing `spack install gcc@8.2.0` gives me:

```
[root@e1515878b09d scratch]# spack install gcc@8.2.0
==> Error: trying to set variants "shared, pic, or optimize" in package "zlib", but the package has no such variants [happened during concretization of zlib@1.2.7%gcc@4.8.5+optimize+pic+shared arch=linux-scientific7-None]
```

After consulting Marc, I was able to fix the issue with:

1. setting up `gcc v8_2_0` UPS product;
2. do `spack compiler find --scope=site` to make spack aware of the compiler. Spack will put the info into `/cvmfs/dunedaq.opensciencegrid.org/products/spack/v0.16.1.fermi/NULL/etc/spack/compilers.yaml`


```
compilers:
- compiler:
    paths:
      cc: /cvmfs/dunedaq.opensciencegrid.org/products/gcc/v8_2_0/Linux64bit+3.10-2.17/bin/gcc
      cxx: /cvmfs/dunedaq.opensciencegrid.org/products/gcc/v8_2_0/Linux64bit+3.10-2.17/bin/g++
      f77: /cvmfs/dunedaq.opensciencegrid.org/products/gcc/v8_2_0/Linux64bit+3.10-2.17/bin/gfortran
      fc: /cvmfs/dunedaq.opensciencegrid.org/products/gcc/v8_2_0/Linux64bit+3.10-2.17/bin/gfortran
    operating_system: scientific7
    target: x86_64
    modules: []
    environment: {}
    extra_rpaths: []
    flags: {}
    spec: gcc@8.2.0
- compiler:
    paths:
      cc: /usr/bin/gcc
      cxx: /usr/bin/g++
      f77:
      fc:
    operating_system: scientific7
    target: x86_64
    modules: []
    environment: {}
    extra_rpaths: []
    flags: {}
    spec: gcc@4.8.5
```

NOTE: when loading a package where two variants exist with only difference being the hash, one can specify the variant by `spack load package/hash`.

## Using chained spack instances to build (`subspack`)

## Using external UPS products installed by Spack

