# What would make or break EESSI for your site? - commercial compilers / software - just EESSI won't be enough - for CUDA: we think we have a solution for that - ship only CUDA libraries - install compat layer to work away diff with older GPU drivers - AMD compiler (Clang-based) - most changes are upstreamed, so less of an issue? - also applies to Intel oneAPI - Agner Fog: "The LLVM-based Intel compiler is a forking of the Clang compiler. The behavior is almost identical to Clang. Intel claim that it optimizes better than the plain Clang compiler (link), but this is not seen in my tests." - https://www.agner.org/forum/viewtopic.php?f=1&t=88&sid=3620055b93f32c14f44f25a5f27433d9 - Intel has more aggressive optimizations by default - diff is getting smaller for C++, Fortran is a different story - Clang compiler toolchains have alway been a 2nd class citizen in EasyBuild (partially due to Fortran situation?) - still a problem for ISV software like MATLAB, Fluent - at ComputeCanada/the-Alliance: separate key-protected repo, also because of user groups - even more problematic is other software with strict licenses (Mathematica, or starts with V ends with ASP) - patchelf for proprietary applications is technically forbidden by the license - being able to extend EESSI or install stuff on top/on the side will need to be possible anyway - shadowing stuff in EESSI is easy to do, but anything we ship in EESSI is RPATH'ed (so replacing dependencies is significantly more difficult to do, but not impossible) - cfr. the approach we're taking to allow injecting vendor-provided MPI (for example via MPItrampoline) - the alternative approach that makes RPATH obsolete looks really promising - cfr. https://stoppels.ch/2022/08/04/stop-searching-for-shared-libraries.html + https://fzakaria.com/2022/09/12/making-runpath-redundant-for-nix.html - this should also make the searching of libraries significantly less severe - availability - sysadmins are often very reluctant to rely on anything outside their control - can be largely mitigated by having own Stratum-1 - lots of sites are already using CVMFS - how long can a Stratum-1 be disconnected from Stratum-0 while still being usable - some sites may worry about loosing control w.r.t. what is provided in EESSI repository - can CVMFS be configured to not provide certain paths? - big benefit of EESSI: more tested than what you would install yourself - w.r.t. archiving: is creating tarball the most efficient way of archving (w.r.t. deduplication) - archiving via alien cache may be better? - w.r.t. disk space: ~600GB in 2019, ~1.2TB in 2022 @ ComputeCanada - see https://indico.cern.ch/event/1079490/contributions/4939532/attachments/2507162/4308239/Canada%20CVMFS%202022.pdf - more or less linear growth - every MATLAB installation is ~600k files - interproscan data is split from the install - impact of data needed for running tests - probably a necessary evil if we're serious about testing software - how does this affect CVMFS cache? - seems like you can opt-out of a shared CVMFS cache, cfr. https://cvmfs.readthedocs.io/en/stable/cpt-configure.html#cache-settings ("CVMFS_SHARED_CACHE") - should ask CVMFS devs - what about visualisation software? - could EESSI also be a solution for this? - JSC does all its graphical stuff via EB, can pick up GL from the host - if components like VirtuaGL are provided in EESSI, it should work - ComputeCanada includes stuff Xvnc, X libs, Mesa, glvnd in compat layer (which increased it a lot) - VirtuaGL 3.0 & up can work directly on EGL, which simplifies things a lot (no need to run 3DX server anymore) - relies on host for OpenGL libraries - can have glvnd also in software layer with GPU support - what about the driver aspect - any issues there? - need to keep an eye on user experience with EESSI - module tree is a big part of that - sites/groups should be able to build their own module tree - "eb --module-only" is currently far from perfect (like PythonPackage, cfr. https://github.com/easybuilders/easybuild-easyblocks/issues/2745) - we should probably ship both a flat & hierarchical module naming scheme - we can filter the module tree via properties (like GPU) based on whether or not the software is actually supported on that system - we should also provide a way to still shows everything that is available (for example on the login nodes) - we should highlight the impact of building software specifically for the hardware - public dashboard with performance results, incl. deliberately running wrong binaries and showing the impact of that - war story: a QuantumESRESSO built for x86_64/generic was significantly faster than running an optimized binaries - EESSI could offer sites with less experience a gateway to getting help