# cvmfs repo layout (reorganization)

## Usage case and requirements

### Externals directory

* A built release uses a single set of externals. Externals are not overlaied. Although technically that's doable;
* A set of externals can be used by multiple releases. It does not need to be rebuilt/reinstalled from scratch for each frozen release.
* A new set of externals is prepared when:
1. Moving to a newer spack release to pick up new versions of external packages from spack's built-in repository of `package.py`s; the upgrade is worth doing when updating more than 10 packages (including all dependencies), otherwise, we can add the new `package.py` files to DUNE DAQ's external repo;
2. Moving to a newer compiler version;
3. The previous set of external packages contain many deprecated versions of external packages, and creating a new set without those will significantly reduce the total size of the set or the total number of versions/pkgs included in the set.

### Releases directory

* Multiple releases distinguished by release names should co-exist in the same directory, so that `daq-builtools` can list them quickly;
* For a given release, multipe builds can co-exist. These different builds can use same spack and/or gcc version or different ones;
* For each build of release, only one set of externals package are used. This is handbled by the spack configuration inside each build.

## Proposed solutions

### Solution 1

#### `externals`

* Inside `externals` subdirectory, directories with names like `<rX-spack-X.Y.Z-gcc-X.Y.Z>`, denoting the release (`<rX>`), spack (`spack-X.Y.Z`) and gcc (`gcc-X.Y.Z`) versions;
* A `current` symbolic link links to the set of externals used by the latest frozen release and nightly releases;

#### `releases`

* Each release directory contains directories with names like `<vX-spack-X.Y.Z-gcc-X.Y.Z>`, where `vX` denotes the build, `spack-X.Y.Z` and `gcc-X.Y.Z` denotes the spack and gcc version;
* Each release directory contains a `default` symbolic link to the recommended/default build for the release.
* Under `releases` directory, a `current` symbolic link links to the current frozen release used in operation or active development/testing.

### Solution 2

The `releases` directory structure is the same as Solution 1. The only difference in this solution is one additional layer of directory in the `externals` subdirectory.

Instead of using `<rX-spack-X.Y.Z-gcc-X.Y.Z>` directlry under `externals`, it uses a subdirectory `<rX>` with `<spack-X.Y.Z-gcc-X.Y.Z>` inside it.

├── solution_1
│   └── spack
│       ├── externals
│       │   ├── current -> r2-spack-0.18.1-gcc-12.1.0
│       │   ├── r1-spack-0.18.1-gcc-12.1.0
│       │   ├── r2-spack-0.18.1-gcc-12.1.0
│       │   ├── r3-spack-0.18.1-gcc-13.0.0
│       │   └── r4-spack-0.19.1-gcc-13.0.0
│       └── releases
│           ├── current -> dunedaq-v3.3.0
│           ├── dunedaq-v3.1.1
│           │   ├── default -> v2-spack-0.18.1-gcc-12.1.0
│           │   ├── v1-spack-0.18.1-gcc-12.1.0
│           │   ├── v2-spack-0.18.1-gcc-12.1.0
│           │   └── v3-spack-0.19.1-gcc-13.0.0
│           ├── dunedaq-v3.2.0
│           │   ├── default -> v1-spack-0.18.1-gcc-12.1.0
│           │   ├── v1-spack-0.18.1-gcc-12.1.0
│           │   └── v2-spack-0.19.1-gcc-12.1.0
│           └── dunedaq-v3.3.0
│               ├── default -> v1-spack-0.19.1-gcc-12.1.0
│               └── v1-spack-0.19.1-gcc-12.1.0
└── solution_2
    └── spack
        ├── externals
        │   ├── current -> r2
        │   ├── r1
        │   │   └── spack-0.18.1-gcc-12.1.0
        │   ├── r2
        │   │   └── spack-0.18.1-gcc-12.1.0
        │   ├── r3
        │   │   └── spack-0.18.1-gcc-13.0.0
        │   └── r4
        │       └── spack-0.19.1-gcc-13.0.0
        └── releases
            ├── current -> dunedaq-v3.3.0        
            ├── dunedaq-v3.1.1
            │   ├── default -> v2-spack-0.18.1-gcc-12.1.0
            │   ├── v1-spack-0.18.1-gcc-12.1.0
            │   ├── v2-spack-0.18.1-gcc-12.1.0
            │   └── v3-spack-0.19.1-gcc-13.0.0
            ├── dunedaq-v3.2.0
            │   ├── default -> v1-spack-0.18.1-gcc-12.1.0
            │   ├── v1-spack-0.18.1-gcc-12.1.0
            │   └── v2-spack-0.19.1-gcc-12.1.0
            └── dunedaq-v3.3.0
                ├── default -> v1-spack-0.19.1-gcc-12.1.0
                └── v1-spack-0.19.1-gcc-12.1.0

## Chosen solution

./solution_chosen/
└── spack
    ├── externals
    │   ├── beta -> ext-v3.0
    │   ├── ext-v1.0
    │   │   └── spack-0.18.1-gcc-12.1.0
    │   ├── ext-v1.1
    │   │   └── spack-0.18.1-gcc-12.1.0
    │   ├── ext-v2.0
    │   │   └── spack-0.18.1-gcc-13.0.0
    │   ├── ext-v3.0
    │   │   └── spack-0.19.1-gcc-13.0.0
    │   └── stable -> ext-v2.0
    └── releases
        ├── current -> dunedaq-v3.3.0    
        ├── dunedaq-v3.1.1
        │   ├── default -> spack-0.18.1-gcc-12.1.0-b1
        │   ├── spack-0.18.1-gcc-12.1.0-b1
        │   ├── spack-0.18.1-gcc-12.1.0-b2
        │   └── spack-0.19.1-gcc-13.0.0-b3
        ├── dunedaq-v3.2.0
        │   ├── default -> spack-0.19.1-gcc-12.1.0-b2
        │   ├── spack-0.18.1-gcc-12.1.0-b1
        │   └── spack-0.19.1-gcc-12.1.0-b2
        └── dunedaq-v3.3.0
            ├── default -> spack-0.19.1-gcc-12.1.0-b1
            └── spack-0.19.1-gcc-12.1.0-b1