# ADMIRE Review

###### tags: `EUROHPC`

> 100 Pms

Archive with I/O traces

Also address AI and deep learning workloads.

--> create I/O stack

- intelligent global coordination
- schedule of storage and compute resources for any application.

--> take into account the overall jobs running.

+ power efficient but mostly through reducing data movement.

collaborate with pother projects through WP5 and WP7.

- Open source & libraries. available through ADMIRE webpage

Open research datasets. One community in Zenodo

Published in datasets -> zenodo community.

Collaboration with other projects.

IO-SEA, DEEP-SEA, RED-SEA. TIME-X, TEXTARROSA.

## WP1

no major deviaiton.
organized in a professional manner. No specific issues.

cross-testing of software / products.

UC3M e-archive --> but anyway all publications are linked on the web. archive for publications.

admire-eurohpc zenodo community.

Cooperation with ADMIRe & RED-SEA not with IO-SEA?

Open cooperation to enlarge with external users.

different people when same partners.

## WP2

Deployed at partners' sites. In practice what does it mean?

GekkoFS. filesystem.

Currently testing with applications.

dataClay (support Object storage).

Hercules

Are these deployed at partners' sites.

Goal is to merge? What does it mean?

all ad-hoc storage support the same APIs.

all ad-hoc storage are connected to I/O scheduler.

Application to Streamiling Deep LEarning with ad-hoc storage.

Direct Lustre -ad-hoc storage
can be done with GeekoFS abut with Expand, etc.

indepdenent of ad-hoc storage system.

Collecting traces.
FAIR--? Increase the FAIRness but not FAIR...

joint metadata schema --> developed by who?

Only for Deep learning applications?

Performance tools and tuning.

--> Is it something you will describe somewhere?

Go out of your bubble and sustainability

--> different steps and the benefit of each step.

Having the overall framework is the best but steps by steps
can be useful.

## WP3

Malleability Management.

Enabling malleability in some of C-based use cases of WP7

Will support Fortran so can be extended tp other use case.

Task: extend FlexMPI to Fortran and Python.

Elastisim to support energy consumption.

integrating the schedulers and malleabilitu support to Slurm

Collaboration with DEEP-SEA or IO-SEA on paper review.

Hamid Fard.

Move to PMIx

## WP4

IO Scheduler

Design SLURM CLI extensions

is it per file?

Different scope: per file, per application, etc.

Directives in SLURM not so useful for end-users.

## WP5

Sensing and profiling.

Prometheus

Per jobid

per user

per node

exporting database to be implemented.
Isn't dependend on the infrastructure.

so it can be reused in other sites.

predictive performance.

IO-SEA --> IO trace repository

Darshan can use old info in ADMIRE framework.

predicitve performance for applications.

per Dataset

Metrics refinements

--> choose the best metrics

do you need as many applications as possible

## WP6 Intelligent controller

From scratch.

Design APIs in collaboration with other WPs.

Scalability: how do you ensure/test it?

get feedback from applications.

monitoring per node --> collected to get monitoring per application.

with flexMPI (demo) how many applications can use flexMPI?

6.2 --> prediction.

based on machine learning. Need a lot of data, right?

IO bound versus compute bound.

6.3 improve intelligent controller

decentralize intelligent controller

distribute IO scheduling

other criteria: energy efficiency.

REDIS is already a distributed database.

WaComm+++ (one of the application)

Manage application workflows.

deployment on one site only? or on several sites?

probably both sites.

## WP7 - Application co-design

support applications developers in use case analysis and test deployments.

Where you have safe reconfiguraitons in the applications.

co-design lifecycle process

WaComM++

integration with Intelligent controller

could increase or decrease the number of particles

Quantum expresso.

Integration with FlexMPI what does it mean for application developers?

NEK5000
CFD
computational intensive.

Can be reconfigured.

Remote sensing.
Distirbuted Deep learning and AI at scale for remote sensing applications.

checkpoint informaiton in D7.2 but we don't have it yet.

Life Science application.
Same as previous.

Software Heritage and Analytics

Some applications do not contribute to the project objectives?

4 applications contribute to the objective?

profiling, etc.

Can it be "easily" applied to other applications?

- port software + ADMIRE components.

Tracing recipe development.

How difficult is it to do for other applications

WP8 Dissemination and Exploitation

Spack --> for installation

different from other projects such as DEEP-SEA

Standardization. Common with other EuroHPC projects.

PMIx increasing usage because of MPI Session.

Hackathons? for onboarding.

Also summer schools. 1 week.

Demos:

Do you have the links of the videos somewhere?

There are on track.

timeline