[link to the 2024 telecons](https://hackmd.io/ZX4WVjldSUOxfLO73E6ZoA)

# CU8 Validation Database Telecons 2023

# Minutes from the telecon on November, 29, 2023, at 11h00
###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/10/24, 10:00 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Alejandro Lorca] (AL)
- [name=Frederic Pailler] (FP)
- [name=Ludovic Delchambre] (LD)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| [ApsisVal4.3b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-161) | [C8OPS-183](https://issues.cosmos.esa.int/gaia/browse/C8OPS-183)| val43b | 21.0.30| 21.0.30b | 21.1.0 | R_21.1.1 | 1.2.0 | August 2023 |
| [ApsisVal4.3a](https://issues.cosmos.esa.int/gaia/browse/C8OPS-158) | [C8OPS-166](https://issues.cosmos.esa.int/gaia/browse/C8OPS-166)| val43a | 21.0.30| 21.0.30b | 21.1.0 | R_21.1.1 | 1.2.0 | August 2023 |
| [ApsisVal4.2d](https://issues.cosmos.esa.int/gaia/browse/C8OPS-171) |[C8OPS-172](https://issues.cosmos.esa.int/gaia/browse/C8OPS-172) | val42d | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2a](https://issues.cosmos.esa.int/gaia/browse/C8OPS-134) |[C8OPS-174](https://issues.cosmos.esa.int/gaia/browse/C8OPS-174) | val42a | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-148) |[C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) | val42b | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | May 2023 |
| [ApsisVal4.1f](https://issues.cosmos.esa.int/gaia/browse/C8OPS-128) |[C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)|val41f | 21.0.27 | 21.0.27b | 21.0.1 | R_21.0.1 | 1.1.1 | Feb 2023 |
| [ApsisOps4.2c](https://issues.cosmos.esa.int/gaia/browse/C8OPS-126) |[C8OPS-125](https://issues.cosmos.esa.int/gaia/browse/C8OPS-125)|ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://issues.cosmos.esa.int/gaia/browse/C8OPS-107) |[C8OPS-115](https://issues.cosmos.esa.int/gaia/browse/C8OPS-115)|val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) | [C8OPS-94](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) |val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

All RUNs for scientific validation : [C8MNG-210](https://issues.cosmos.esa.int/gaia/browse/C8MNG-210)

### Status DONE :

* Add a **new external catalog** containing **Gaia Benchmark Stars** to the VDB : [C8OPS-192](https://issues.cosmos.esa.int/gaia/browse/C8OPS-192)
* :a: **Alejandro** : **New DAF DM** and **the conversion code** for Apsis **Val4.4a** are **ready**.
* :a: **Inna** : to close the corresponding C8DM jiras. - DONE

<!-- * [Wiki page](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_validation_cycle3plus#CU8_Validation_database) is updated -->

### ONGOING & ACTIONS :

1. **Make available the Apsis Val 4.4a in the VDB** [C8OPS-179](https://issues.cosmos.esa.int/gaia/browse/C8OPS-179)

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| ApsisVal4.4a | [C8OPS-179](https://issues.cosmos.esa.int/gaia/browse/C8OPS-179)| val44a | 21.0.30| 21.0.30b | 21.2.0 | R_21.2.0| 1.2.0 | November 2023 |

**Remarks, observations and actions concerning the run :**
* **ESPCS** had still the issue with too long values for **flags** parameter in Apsis Val4.3a. But the issue is resolved in val4.4a ([C8CAL-881](https://issues.cosmos.esa.int/gaia/browse/C8CAL-881))
* [C8DM-409](https://issues.cosmos.esa.int/gaia/browse/C8DM-409) : NaN values are expected for val4.4a.
* **Data delivery** from DPCC to DPCE is planned for **November 29**.
* :a: **Inna** : to test new val4.4a catalogs when they are ready.
* Alejandro will be on holidays during two first weeks of December.

2. **Add new *external* catalogs to the VDB** (top-level jira [C8TVCD-116](https://issues.cosmos.esa.int/gaia/browse/C8TVCD-116)) :

* **NearInfraRedAstroSource** [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210) :
* **in progress** : Sara will create a file.

3. Alejandro and Enrique with a team from Google test some capabilities of the **Google cloud** platform to host and handle large datasets from Gaia. - in progress.

4. **General questions & planning:**
* Preparing **val4.5a** :
* :a: **Inna** : to prepare the Jiras for **val4.5a** :
* general one
* top-level jiras for new DAF DM and conversion code
* remark 1 : Jira "Create and ingest apsis val4.5a" [C8OPS-194](https://issues.cosmos.esa.int/gaia/browse/C8OPS-194) already exists
* remark 2 : New ticket : Create UCA validation tables [C8DM-411](https://issues.cosmos.esa.int/gaia/browse/C8DM-411) (is taken into account in val4.4a) + [C8T-246](https://issues.cosmos.esa.int/gaia/browse/C8T-246) (to take into account in val4.5a)
* New DAF DM - end of December.
* :a: **Orlagh** : send this information to all the CU8 team - DONE
* :a: **Inna** : add this information to the Wiki VDB page;
* During the next telecon : to discuss the **schedule for future actions** (ingesting val4.5a etc)
* Will we need dr3 - dr4 sourceIds in VDB ? For VST ?

### Postponed tasks

1. :a: **Enrique** : Enforcing DM units and boundary checks [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192). - **in progress**, waiting for new version of Dictionary Tool; comment from David "Tentatively rescheduled for OCT23NOV23".

2. Some WP managers want to see the **same parameter names in VDB and GACS**
* :a: **Alejandro** : we need to convert camelCase names (Dictionary Tool) into **names_with_underscore** (e.g. [C8DM-354](https://issues.cosmos.esa.int/gaia/browse/C8DM-354) for GSP-Phot) : [C8T-193](https://issues.cosmos.esa.int/gaia/browse/C8T-193) - **Implimentation expected with the new version of Dictionary Tool (release 22.O)**.

3. It is now impossible do **download huge csv files** (> 100Mb). Alejandro will look at this if he has some time; **not urgent**.

4. GSP-SPEC : It would be good to replace old names (caFe1 by caFe etc) in conversion code. We need MDB updates first [MDBICD-2620](https://issues.cosmos.esa.int/gaia/browse/MDBICD-2620) - **POSTPONED for cycle 5**.

## :calendar: Next telecon

January 16, at 11h00

************

# Minutes from the telecon on October, 24, 2023, at 10h00
###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/10/24, 10:00 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Alejandro Lorca] (AL)
- [name=Frederic Pailler] (FP)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| [ApsisVal4.3b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-161) | [C8OPS-183](https://issues.cosmos.esa.int/gaia/browse/C8OPS-183)| val43b | 21.0.30| 21.0.30b | 21.1.0 | R_21.1.1 | 1.2.0 | August 2023 |
| [ApsisVal4.3a](https://issues.cosmos.esa.int/gaia/browse/C8OPS-158) | [C8OPS-166](https://issues.cosmos.esa.int/gaia/browse/C8OPS-166)| val43a | 21.0.30| 21.0.30b | 21.1.0 | R_21.1.1 | 1.2.0 | August 2023 |
| [ApsisVal4.2d](https://issues.cosmos.esa.int/gaia/browse/C8OPS-171) |[C8OPS-172](https://issues.cosmos.esa.int/gaia/browse/C8OPS-172) | val42d | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2a](https://issues.cosmos.esa.int/gaia/browse/C8OPS-134) |[C8OPS-174](https://issues.cosmos.esa.int/gaia/browse/C8OPS-174) | val42a | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-148) |[C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) | val42b | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | May 2023 |
| [ApsisVal4.1f](https://issues.cosmos.esa.int/gaia/browse/C8OPS-128) |[C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)|val41f | 21.0.27 | 21.0.27b | 21.0.1 | R_21.0.1 | 1.1.1 | Feb 2023 |
| [ApsisOps4.2c](https://issues.cosmos.esa.int/gaia/browse/C8OPS-126) |[C8OPS-125](https://issues.cosmos.esa.int/gaia/browse/C8OPS-125)|ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://issues.cosmos.esa.int/gaia/browse/C8OPS-107) |[C8OPS-115](https://issues.cosmos.esa.int/gaia/browse/C8OPS-115)|val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) | [C8OPS-94](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) |val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

All RUNs for scientific validation : [C8MNG-210](https://issues.cosmos.esa.int/gaia/browse/C8MNG-210)

### Status DONE :

**New DM for Apsis Val 4.4a is ready** : [C8OPS-168](https://issues.cosmos.esa.int/gaia/browse/C8OPS-168).

<!-- * [Wiki page](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_validation_cycle3plus#CU8_Validation_database) is updated -->

### ONGOING & ACTIONS :

1. **Make available the Apsis Val 4.4a in the VDB** [C8OPS-179](https://issues.cosmos.esa.int/gaia/browse/C8OPS-179)

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| ApsisVal4.4a | [C8OPS-179](https://issues.cosmos.esa.int/gaia/browse/C8OPS-179)| val44a | 21.0.30| 21.0.30b | 21.2.0 | R_21.2.0| 1.2.0 | November 2023 |

**Questions :**
* **All modules ?** - modules which have updated thier DMs : GSP-PHOT, FLAME, DSC, ASP + probably someone else.
* **Expected size ?** - VST size

**Remarks, observations and actions concerning the run :**
* **ESPCS** has still the issue with too long values for **flags** parameter; Alessandro is informed [C8OTH-294](https://issues.cosmos.esa.int/gaia/browse/C8OTH-294). :a: **Inna** : ask Alessandro if it is fixed. - DONE
* :a: **Inna** : to close the corresponding C8DM jiras. If impossible, contact Morgan. - Waiting for creation of new DAF DM version (asked during closing)
* :a: **Alejandro** : to publish new DAF DM and to prepare the new conversion code.
* [C8DM-409](https://issues.cosmos.esa.int/gaia/browse/C8DM-409) : NaN values are expected for val4.4a.
* Data are expected during the last week of November.
* Alejandro will be probably on holidays during two first weeks of December (TBC).

2. **Add new *external* catalogs to the VDB** (top-level jira [C8TVCD-116](https://issues.cosmos.esa.int/gaia/browse/C8TVCD-116)) :

* **NearInfraRedAstroSource** [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210) :
* **in progress** : Sara will create a file.

* **Gaia Benchmark Stars V3** [C8OPS-192](https://issues.cosmos.esa.int/gaia/browse/C8OPS-192): ~200 stars with fundamental values of Teff, luminosity, radius, logg and ~40 stars with seismic mass, radius, logg (request from Caroline Soubiran) :
* :a: **Orlagh** : when the list of sources is ready, Orlagh will provide a list of correspondance *sourceid_dr3 - sourceid_dr4* for them.

3. Alejandro and Enrique with a team from Google test some capabilities of the **Google cloud** platform to host and handle large datasets from Gaia.

### Postponed tasks

1. :a: **Enrique** : Enforcing DM units and boundary checks [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192). - **in progress**, waiting for new version of Dictionary Tool; comment from David "Tentatively rescheduled for OCT23NOV23".

2. Some WP managers want to see the **same parameter names in VDB and GACS**
* :a: **Alejandro** : we need to convert camelCase names (Dictionary Tool) into **names_with_underscore** (e.g. [C8DM-354](https://issues.cosmos.esa.int/gaia/browse/C8DM-354) for GSP-Phot) : [C8T-193](https://issues.cosmos.esa.int/gaia/browse/C8T-193) - **Implimentation expected with the new version of Dictionary Tool (release 22.O)**.

3. It is now impossible do **download huge csv files** (> 100Mb). Alejandro will look at this if he has some time; **not urgent**.

4. GSP-SPEC : It would be good to replace old names (caFe1 by caFe etc) in conversion code. We need MDB updates first [MDBICD-2620](https://issues.cosmos.esa.int/gaia/browse/MDBICD-2620) - **POSTPONED for cycle 5**.

## :calendar: Next telecon

November 29, at 11h00

****************

# Minutes from the telecon on July, 20, 2023, at 15h00

###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/07/20, 15:00 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Alejandro Lorca] (AL)
- [name=Frederic Pailler] (FP)
- [name=David Teyssier] (DT)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| [ApsisVal4.2d](https://issues.cosmos.esa.int/gaia/browse/C8OPS-171) |[C8OPS-172](https://issues.cosmos.esa.int/gaia/browse/C8OPS-172) | val42d | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2a](https://issues.cosmos.esa.int/gaia/browse/C8OPS-134) |[C8OPS-174](https://issues.cosmos.esa.int/gaia/browse/C8OPS-174) | val42a | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-148) |[C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) | val42b | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | May 2023 |
| [ApsisVal4.1f](https://issues.cosmos.esa.int/gaia/browse/C8OPS-128) |[C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)|val41f | 21.0.27 | 21.0.27b | 21.0.1 | R_21.0.1 | 1.1.1 | Feb 2023 |
| [ApsisOps4.2c](https://issues.cosmos.esa.int/gaia/browse/C8OPS-126) |[C8OPS-125](https://issues.cosmos.esa.int/gaia/browse/C8OPS-125)|ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://issues.cosmos.esa.int/gaia/browse/C8OPS-107) |[C8OPS-115](https://issues.cosmos.esa.int/gaia/browse/C8OPS-115)|val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) | [C8OPS-94](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) |val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

All RUNs for scientific validation : [C8MNG-210](https://issues.cosmos.esa.int/gaia/browse/C8MNG-210)

### Status DONE :

* Release **DAF version 1.2.0** with new images in pre-production and production
[C8OPS-180](https://issues.cosmos.esa.int/gaia/browse/C8OPS-180) :a: **Inna** : validation
* **ESP-CS : some values of the parameter flags are still very long** [C8T-227](https://issues.cosmos.esa.int/gaia/browse/C8T-227) + [C8CAL-887](https://issues.cosmos.esa.int/gaia/browse/C8CAL-887)
* **Information from Frédéric and Orlagh** : it is possible to **send the data to DPCE module by module** to accelerate the ingestion into VDB. This poses no problem to DPCC to do it in small
batches (module by module) or one large batch.
**For ApsisVal4.3a they will most likely send everything together.**
* [Wiki page](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_validation_cycle3plus#CU8_Validation_database) is updated

### QUESTION :

*Which changes for the conversion code will be made for ApsisVal4.3a and which ones will be posponed for the next run ?*

**Decision :**

Top-level Jira for conversion code for **ApsisVal4.3a** : [C8OPS-169](https://issues.cosmos.esa.int/gaia/browse/C8OPS-169)
* DSC : update one parameter (**classlabeldscjoint**) [C8T-220](https://issues.cosmos.esa.int/gaia/browse/C8T-220)
* OA : Add the parameter **solutionid** to the conversion code for OA module [C8T-228](https://issues.cosmos.esa.int/gaia/browse/C8T-228)
* ASP & OA : Check **deprecated methods** for val43a software mapping [C8T-230](https://issues.cosmos.esa.int/gaia/browse/C8T-230)

Postoponed for **ApsisVal4.4a (Oct-Nov 2023) :** top-level jira for conversion code [C8OPS-182](https://issues.cosmos.esa.int/gaia/browse/C8OPS-182)

* Delete or comment the **deprecated parameters** [C8OTH-273](https://issues.cosmos.esa.int/gaia/browse/C8OTH-273), [C8OTH-283](https://issues.cosmos.esa.int/gaia/browse/C8OTH-283)
* Delete or comment the **empty parameters in ASP** [C8OTH-283](https://issues.cosmos.esa.int/gaia/browse/C8OTH-283) : waiting for [C8CAL-889](https://issues.cosmos.esa.int/gaia/browse/C8CAL-889)

### ONGOING & ACTIONS :

1. Make available apsis **ApsisVal4.3a** in the VDB [C8OPS-166](https://issues.cosmos.esa.int/gaia/browse/C8OPS-166) :
* Frédéric : the run is finished for almost all the modules except OA and DIBSPEC
* Plan : the next week (after July 24), DPCC will start to send the data to DPCE even if DIBSPEC and OA will not be finished yet.

2. Make available **ApsisVal4.3b** in the VDB for **DSC** [C8OPS-183](https://issues.cosmos.esa.int/gaia/browse/C8OPS-183) - run is planned for the next week (after July 24)

3. **Add new *external* catalogs to the VDB** (top-level jira [C8TVCD-116](https://issues.cosmos.esa.int/gaia/browse/C8TVCD-116)) :

* **NearInfraRedAstroSource** : :a: **Morgan** : Add the table NearInfraRedAstroSource as an external catalog to VDB [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210) - **in progress** : waiting for implementation of [C8OTH-252](https://issues.cosmos.esa.int/gaia/browse/C8OTH-252).

*********************

# Minutes from the telecon on June, 15, 2023, at 16h30

###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/06/15, 16:30 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Alejandro Lorca] (AL)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| [ApsisVal4.2d](https://issues.cosmos.esa.int/gaia/browse/C8OPS-171) |[C8OPS-172](https://issues.cosmos.esa.int/gaia/browse/C8OPS-172) | val42d | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2a](https://issues.cosmos.esa.int/gaia/browse/C8OPS-134) |[C8OPS-174](https://issues.cosmos.esa.int/gaia/browse/C8OPS-174) | val42a | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | June 2023 |
| [ApsisVal4.2b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-148) |[C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) | val42b | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | May 2023 |
| [ApsisVal4.1f](https://issues.cosmos.esa.int/gaia/browse/C8OPS-128) |[C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)|val41f | 21.0.27 | 21.0.27b | 21.0.1 | R_21.0.1 | 1.1.1 | Feb 2023 |
| [ApsisOps4.2c](https://issues.cosmos.esa.int/gaia/browse/C8OPS-126) |[C8OPS-125](https://issues.cosmos.esa.int/gaia/browse/C8OPS-125)|ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://issues.cosmos.esa.int/gaia/browse/C8OPS-107) |[C8OPS-115](https://issues.cosmos.esa.int/gaia/browse/C8OPS-115)|val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) | [C8OPS-94](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) |val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

All RUNs for scientific validation : [C8MNG-210](https://issues.cosmos.esa.int/gaia/browse/C8MNG-210)

### Status DONE :

* Apsis Val4.2b is published
* Apsis Val4.2a is published
* Apsis Val4.2d is published
* [Wiki page](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_validation_cycle3plus#CU8_Validation_database) is updated

### QUESTIONS :

1. Updating DAF conversion code is actually planned for September-October. **Is it possible to do it earlier ?** - **DECISION : Alejandro** : Yes if we have enough time taking into account summer holidays (first two weeks for Alejandro). Otherwise, Enrique will probably put the date into VDB with old conversion code. We will stay in touch to decide how to proceed ==>
:a: **Orlagh** : ask Frédéric if it is possible to send the data to DPCE module by module to accelerate the ingestion into VDB
:a: **Inna** : create a top-level Jira C8T for conversion code R_21.1.1 :
* DSC : update one parameter (classlabeldscjoint) [C8T-220](https://issues.cosmos.esa.int/gaia/browse/C8T-220)
* OA : add one parameter in OA module (solutionId) [C8OTH-276](https://issues.cosmos.esa.int/gaia/browse/C8OTH-276)
* Delete or comment the deprecated parameters ? [C8OTH-273](https://issues.cosmos.esa.int/gaia/browse/C8OTH-273), [C8OTH-283](https://issues.cosmos.esa.int/gaia/browse/C8OTH-283)
* Delete or comment the empty parameters in ASP ? [C8OTH-283](https://issues.cosmos.esa.int/gaia/browse/C8OTH-283)

2. **Empty parameters in ASP :** what will do with them? **DECISION :** :a: **Orlagh** will present the table below to Frédéric Pailler and also during CU8OPS meeting on June 23 to see with all the colleagues why the parameters are empty and who needs them :

Mapping:
(AstroSourceProb) input = data.getInput();
cu3 = input.getAs().getCu3();
calSpec = input.getAs().getCu5().getCalSpecSource();
calPhot = input.getAs().getCu5().getCalPhotSource();
bpSpec = calSpec.getBpSpec();
rpSpec = calSpec.getRpSpec();
normSpectrum = input.getAs().getCu6().getNormalizedCalibratedSpectrum();
meanSpec = input.getAs().getCu5().getMeanSpec();

WP = "-" means that this parameter was already empty in VAL4.1f and that it was already mentionned in my e-mail on April 4. Nobody answered that they needs these parameters.

| n | parameter | values | mapping from conversion code and spreadsheet | CU8 DAF DM status | WP |
|----| -------- | -------- | -------- |-------- | -------- |
| 1 | astrometricgmag | 0 | cu3.getMeanOnBoardGMag() | OK | - |
| 2 | rvsflux | NaN | calSpec.getRvsFlux() | OK | - |
| 3 | rvsfluxerror | NaN | calSpec.getRvsFluxError() | OK | - |
| 4 | nbpspecs | NaN | calSpec.getBpNumOfTransits() | OK | - |
| 5 | nrpspecs | NaN | calSpec.getRpNumOfTransits() | OK | - |
| 6 | nbpcoeffs | NaN | bpSpec.getSolution().getParameters().length | OK | - |
| 7 | nrpcoeffs | NaN | rpSpec.getSolution().getParameters().length | OK | - |
| 8 | rvslogflux | NaN | Math.log10(calSpec.getRvsFlux()) | OK | - |
| 9 | basisdefbp | NaN | bpSpec.getBasisFunctionSetDefId() | OK | - |
| 10 | basisdefrp | NaN | rpSpec.getBasisFunctionSetDefId() | OK | - |
| 11 | nrvscombccd | NaN | normSpectrum.getNbCombinedCcd() | OK | - |
| 12 | nrvsdebccd | NaN | normSpectrum.getNbDeblendedCcd() | OK | - |
| 13 | rvsstatus | NaN | normSpectrum.getStatus() | OK | - |
| 14 | xpspecnorm | NaN | meanSpec.getNormalisationFactor() | OK | - |
| 15 | xpspecsnr | NaN | meanSpec.getSignalToNoiseRatio() | OK | - |
| 16 | ipdgofharmonicamplitude | 0 | cu3.getIpdGofHarmonicAmplitude() | OK | - |
| 17 | ipdgofharmonicphase | 0 | cu3.getIpdGofHarmonicPhase() | OK | - |
| 18 | photbprpexcessfactor | 0 | calPhot.getCorrectedFluxExcess() (conv.code) **BUT** calPhot.getMeanFluxExcess() (spreadsheet) | OK | - |
| 19 | paramssolved |NaN | | deprecated | |
| 20 | paramsexported | 0 | cu3.getParamsExported() **BUT** absent in spreadsheet | OK | |

2. **What will we do with deprecated parameters ?** Should we create a new jira to delete deprecated parameters ? **Do we really need this status ?** Would it be better to delete them immediately in order to avoid their appearance with all NaN values ? The situation is already complicated between different versions of MDB and VDB DMs. - **DECISION : We will delete existing deprecated values. And we will not use this status anymore : parameter will be either present or absent.**

3. **Is there one responsible for ASP catalogue ?** - No. This table contains parameters from other CUs. Several CU8 colleagues use different parameters from this table.

### ONGOING & ACTIONS :

1. :a: **Enrique** : Enforcing DM units and boundary checks [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192). - **in progress** :a: Alejandro will ask Enrique about news.

2. **Add new *external* catalogs to the VDB** (top-level jira [C8TVCD-116](https://issues.cosmos.esa.int/gaia/browse/C8TVCD-116)) :

* **NearInfraRedAstroSource** : :a: **Morgan** : Add the table NearInfraRedAstroSource as an external catalog to VDB [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210) - **in progress** : waiting for implementation of [C8OTH-252](https://issues.cosmos.esa.int/gaia/browse/C8OTH-252).

3. Make available apsis **ApsisVal4.3a** in the VDB [C8OPS-166](https://issues.cosmos.esa.int/gaia/browse/C8OPS-166) : should be started the week of June 19 (TBC).
* to confirm the versions of MDB DM and of the conversion code.

<!--
4. [Validation Runs] Run ApsisVal4.3b using nearIR data
https://issues.cosmos.esa.int/gaia/browse/C8OPS-161 Will we need it in VDB ?-->

### Postponed tasks

1. Make available the **Apsis Val 4.4a** in the VDB (planned for September-October) [C8OPS-179](https://issues.cosmos.esa.int/gaia/browse/C8OPS-179)

2. Some WP managers want to see the **same parameter names in VDB and GACS**
* :a: **Alejandro** : we need to convert camelCase names (Dictionary Tool) into **names_with_underscore** (e.g. [C8DM-354](https://issues.cosmos.esa.int/gaia/browse/C8DM-354) for GSP-Phot) : [C8T-193](https://issues.cosmos.esa.int/gaia/browse/C8T-193) - **Implimentation expected with the new Main Data Model 22.0**.

3. :a: **Alejandro** will upgrade the DAF infrastructure (new versions of Jupyter, containers, add some new features etc.). We will need to check if all is OK in the Pre-Production before upgrating the Production.

4. **ESP-CS : some values of the parameter flags are still very long** [C8T-227](https://issues.cosmos.esa.int/gaia/browse/). :a: **Orlagh** will create C8CAL jira for Alessandro Lanzafame.

5. GSP-SPEC : It would be good to replace old names (caFe1 by caFe etc) in conversion code. We need MDB updates first [MDBICD-2620](https://issues.cosmos.esa.int/gaia/browse/MDBICD-2620) - **POSTPONED for cycle 5**.

## :calendar: Next telecon

June 19, at 10h00

# Minutes from the telecon on May, 10, 2023, at 11h00

###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/05/10, 11:00 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Alejandro Lorca] (AL)
- [name=Frédéric Pailler] (FP)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| [ApsisVal4.2b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-148) |[C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) | val42b | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1 | May 2023 |
| [ApsisVal4.1f](https://issues.cosmos.esa.int/gaia/browse/C8OPS-128) |[C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)|val41f | 21.0.27 | 21.0.27b | 21.0.1 | R_21.0.1 | 1.1.1 | Feb 2023 |
| [ApsisOps4.2c](https://issues.cosmos.esa.int/gaia/browse/C8OPS-126) |[C8OPS-125](https://issues.cosmos.esa.int/gaia/browse/C8OPS-125)|ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://issues.cosmos.esa.int/gaia/browse/C8OPS-107) |[C8OPS-115](https://issues.cosmos.esa.int/gaia/browse/C8OPS-115)|val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) | [C8OPS-94](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) |val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

All RUNs for scientific validation : [C8MNG-210](https://issues.cosmos.esa.int/gaia/browse/C8MNG-210)

### Status DONE :

1. **Add the DIBSpec param table to the VDB** (https://wiki.cosmos.esa.int/gaia-dpac/index.php?title=CU8:Processing_telecon_20230303) - :a: **Inna** : remind to Mathias and Christophe to create corresponding Jiras. - **DONE** : table DIBSPEC is created in DAF DM ([C8DM-397](https://issues.cosmos.esa.int/gaia/browse/C8DM-397), [C8T-218](https://issues.cosmos.esa.int/gaia/browse/C8T-218))

2. **Make available apsis val42b in the VDB - part 1** [C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) :
* :a: **Inna** : remind to Yves to update VDB mapping for new fields in VDB/asp in Jira [C8T-188](https://issues.cosmos.esa.int/gaia/browse/C8T-188) - **DONE** comment is put in corresponding DM Jira [C8DM-377](https://issues.cosmos.esa.int/gaia/browse/C8DM-377) + an e-mail is sent on April 5.
* :a: **Inna** : Send a reminder to WP managers :
* **asking if someone needs the parameters which are actually empty in the ASP catalogue** ([C8OTH-260](https://issues.cosmos.esa.int/gaia/browse/C8OTH-260)). - **DONE** - en e-mail is sent on April 4 - No comments from WP managers - **Decision : we will keep the empty parameters this time, and then depricate them.**
* asking to add/check boundary values for their parameters in the Dictionary Tool. It will allow us an automatic check of the data during their ingestion into the VDB : [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192). - **DONE** - en e-mail is sent on April 4.
* :a: **Inna** : Approve C8DM jiras and assign them back to reporters. - **DONE**

3. **Question : move the DM changes to the next version of DAF DM ?** : [C8OPS-168](https://issues.cosmos.esa.int/gaia/browse/C8OPS-168) & [C8OPS-169](https://issues.cosmos.esa.int/gaia/browse/C8OPS-169) - Yes, these changes will be implemented for DAF DM 21.1.1 (September - October 2023).

4. :a: **Orlagh** : **2023 operations plan**. (added by Orlagh) This concerns the clarification of the work in the VDB for 2023. - **DONE:** [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

5. :a: **Inna** : Update Wiki VDB page, section News - **DONE**.

### ONGOING & ACTIONS :

1. **Make available apsis val42b in the VDB - part 2** [C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) :

* :a: **Enrique** : Enforcing DM units and boundary checks [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192). - **in progress**.

2. **Add new *external* catalogs to the VDB** (top-level jira [C8TVCD-116](https://issues.cosmos.esa.int/gaia/browse/C8TVCD-116)) :

* **NearInfraRedAstroSource** : :a: **Morgan** : Add the table NearInfraRedAstroSource as an external catalog to VDB [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210) - **Work in progress**; :a: **Inna** : ask Morgan and Rene about the news. **Done, waiting for an answer.**

3. :a: **Orlagh** : to follow up with DSC/GSPPhot/UCD concerning sending data with IR runs to the VDB. - **in progress**

4. **Make available the ApsisVal4.2d run for 5 modules and asp in the VDB** [C8OPS-172](https://issues.cosmos.esa.int/gaia/browse/C8OPS-172)

5. **Make available ApsisVal4.2a in the VDB** [C8OPS-174](https://issues.cosmos.esa.int/gaia/browse/C8OPS-174)

### Postponed tasks

1. Make available apsis **ApsisVal4.3a** in the VDB [C8OPS-166](https://issues.cosmos.esa.int/gaia/browse/C8OPS-166)

2. :a: **Orlagh** : [C8DM-396](https://issues.cosmos.esa.int/gaia/browse/C8DM-396) - modify the type of the parameter ngoodmodels (int or float) - postponed for the next DM/conversion software updates in September - october 2023.

3. Some WP managers want to see the **same parameter names in VDB and GACS**
* :a: **Alejandro** : we need to convert camelCase names (Dictionary Tool) into **names_with_underscore** (e.g. [C8DM-354](https://issues.cosmos.esa.int/gaia/browse/C8DM-354) for GSP-Phot) : [C8T-193](https://issues.cosmos.esa.int/gaia/browse/C8T-193) - **Implimentation expected with the new Main Data Model 22.0**.

4. :a: **Alejandro** will upgrade the DAF infrastructure (new versions of Jupyter, containers, add some new features etc.). We will need to check if all is OK in the Pre-Production before upgrating the Production.

5. GSP-SPEC : It would be good to replace old names (caFe1 by caFe etc) in conversion code. We need MDB updates first [MDBICD-2620](https://issues.cosmos.esa.int/gaia/browse/MDBICD-2620) - **POSTPONED for cycle 5**.

## :calendar: Next telecon

June 15, at 16h30

# Minutes from the telecon on April, 4, 2023, at 11h15

###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/04/04, 11:15 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Alejandro Lorca] (AL)
- [name=Frédéric Pailler] (FP)
- [name=Alessandro Lanzafame] (AL)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment

| Apsis run | VDB ticket| VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- |-------- | -------- | -------- |
| [ApsisVal4.2b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-148) |[C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) | val42b | 21.0.29 | 21.0.29b | 21.1.0 | R_21.1.0 | 1.1.1? | April 2023 |
| [ApsisVal4.1f](https://issues.cosmos.esa.int/gaia/browse/C8OPS-128) |[C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)|val41f | 21.0.27 | 21.0.27b | 21.0.1 | R_21.0.1 | 1.1.1 | Feb 2023 |
| [ApsisOps4.2c](https://issues.cosmos.esa.int/gaia/browse/C8OPS-126) |[C8OPS-125](https://issues.cosmos.esa.int/gaia/browse/C8OPS-125)|ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://issues.cosmos.esa.int/gaia/browse/C8OPS-107) |[C8OPS-115](https://issues.cosmos.esa.int/gaia/browse/C8OPS-115)|val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) | [C8OPS-94](https://issues.cosmos.esa.int/gaia/browse/C8OPS-94) |val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

Future RUNs for scientific validation : [C8MNG-210](https://issues.cosmos.esa.int/gaia/browse/C8MNG-210)

### Status DONE :

1. Access to VDB is open to six CU9 colleagues.

### ONGOING & ACTIONS :

1. **Add the DIBSpec param table to the VDB** (https://wiki.cosmos.esa.int/gaia-dpac/index.php?title=CU8:Processing_telecon_20230303) - :a: **Inna** : remind to Mathias and Christophe to create corresponding Jiras.

2. **Make available apsis val42b in the VDB** [C8OPS-165](https://issues.cosmos.esa.int/gaia/browse/C8OPS-165) :
* :a: **Inna** : remind to Yves to update VDB mapping for new fields in VDB/asp in Jira [C8T-188](https://issues.cosmos.esa.int/gaia/browse/C8T-188)
* :a: **Enrique** : Enforcing DM units and boundary checks [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192).
* :a: **Inna** : Send a reminder to WP managers :
* asking if someone needs the parameters which are actually empty in the ASP catalogue ([C8OTH-260](https://issues.cosmos.esa.int/gaia/browse/C8OTH-260)). Otherwise, these parameters will be deleted. Deadline April 10.
* asking to add/check boundary values for their parameters in the Dictionary Tool. It will allow us an automatic check of the data during their ingestion into the VDB : [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192).
* :a: **Inna** : Approve C8DM jiras and assign them back to reporters.

2. **Add new *external* catalogs to the VDB** :

* **NearInfraRedAstroSource** : :a: **Morgan** : Add the table NearInfraRedAstroSource as an external catalog to VDB [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210) - **Work in progress**

4. :a: **Orlagh** : **2023 operations plan**. (added by Orlagh) This concerns the clarification of the work in the VDB for 2023.

5. :a: **Orlagh** : to follow up with DSC/GSPPhot/UCD concerning sending data with IR runs to the VDB.

### Postponed tasks

1. :a: **Alejandro** will upgrade the DAF infrastructure (new versions of Jupyter, containers etc.). We will need to check if all is OK in the Pre-Production before upgrating the Production. - **postponed for June if needed**.

2. Some WP managers want to see the **same parameter names in VDB and GACS**
* :a: **Alejandro** : we need to convert camelCase names (Dictionary Tool) into **names_with_underscore** (e.g. [C8DM-354](https://issues.cosmos.esa.int/gaia/browse/C8DM-354) for GSP-Phot) : [C8T-193](https://issues.cosmos.esa.int/gaia/browse/C8T-193) - **Implimentation expected with the new Main Data Model 22.0**.

3. Make available apsis val43a in the VDB [C8OPS-166](https://issues.cosmos.esa.int/gaia/browse/C8OPS-166) - infos sur DM versions, number of sources etc to be confirmed

4. GSP-SPEC : It would be good to replace old names (caFe1 by caFe etc) in conversion code. We need MDB updates first [MDBICD-2620](https://issues.cosmos.esa.int/gaia/browse/MDBICD-2620) - **POSTPONED for cycle 5**.

## :calendar: Next telecon

May 3, at 15h30

# Minutes from the telecon on February 28, 2023, at 10h00

###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/02/28, 10:00 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Alejandro Lorca] (AL)
- [name=Frédéric Pailler] (FP)
- [name=David Teyssier] (DT)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment

| Apsis run | VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| [ApsisVal4.1f](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:Apsis_runs_ApsisVal4.1f) | val41f | 21.0.27 | 21.0.27b | 21.0.1 | R_21.0.1 | 1.1.1 | Feb 2023 |
| [ApsisOps4.2c](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8_APSIS_4.2_Kick_Off_Meeting) | ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:Apsis_runs_ApsisVal4.1e) | val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:Apsis_runs_ApsisVal4.1b) | val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [Schedule](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Schedule)

### Status DONE :

1. **ApsisVal4.1f : Test the new post-processing to strip large fields from ASP before delivery to DPCE** : [C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144). ApsisVal4.1f ASP catalog is successfully ingested into the VDB.

2. Inna has **updated the VDB WIKI page** :
* information about the VDB webinar link : if this link does not work anymore (expired), users should contact Alejandro who will renew the link.
* information about two new catalogs (val41f, ext_dr16q_v4),
* future changes in DMs.

### ONGOING & ACTIONS :

1. **Add new *external* catalogs to the VDB** :

* **NearInfraRedAstroSource** : :a: **Morgan** : Add the table NearInfraRedAstroSource as an external catalog to VDB [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210) - **Work in progress**

2. **Enforcing DM units and boundary checks** [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192). :a: **Enrique** : **work in progress**.

3. **Should we open VDB to CU9** to receive their feedback earlier ? - **YES, we should !**
Orlagh has asked the opinion of CU9 members - 6 colleagues from CU9 are interested in. Orlagh has sent an e-mail to them. **We will have more details in a month.**

4. :a: **Inna** : During the next CU8OPS meeting (March, 3), share the following informations to all WP managers :
* ask if someone needs the parameters which are actually empty in the ASP catalogue ([C8OTH-260](https://issues.cosmos.esa.int/gaia/browse/C8OTH-260));
* In order to accelerate and facilitate the validation of the data, WP managers should check and indicate boundary values for their parameters in the Dictionary Tool. It will allow an automatic check of the data during their ingestion into the VDB : [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192).

5. :a: **Alejandro** will upgrade the DAF infrastructure (new versions of Jupyter, containers etc.). We will need to check if all is OK in the Pre-Production before upgrating the Production.

### Postponed tasks

1. **Future changes in VDB for the ApsisVal4.2b (for end of March 2023)**:
* Top-level Jira for changes in DAF DM (for version DAF_21.0.2) : [C8OPS-118](https://issues.cosmos.esa.int/gaia/browse/C8OPS-118).
* Corresponding top-level Jira for DAF conversion code (for version DMC_21.0.1) [C8OPS-119](https://issues.cosmos.esa.int/gaia/browse/C8OPS-119)
* Reminder for CU8 WP managers : Apsis code changes will be needed to do a scientific test of the mapping updates [C8CAL-779](https://issues.cosmos.esa.int/gaia/browse/C8CAL-779) (**April 2023**)
* :a: **Inna** : create new top-level jira for this run (clone [C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144))

2. Some WP managers want to see the **same parameter names in VDB and GACS**
* :a: **Alejandro** : we need to convert camelCase names (Dictionary Tool) into **names_with_underscore** (e.g. [C8DM-354](https://issues.cosmos.esa.int/gaia/browse/C8DM-354) for GSP-Phot) : [C8T-193](https://issues.cosmos.esa.int/gaia/browse/C8T-193)
* See also [DICT-590](https://issues.cosmos.esa.int/gaia/browse/DICT-590)
* Implimentation expected in the **end of March 2023** with the new Data Model.

3. GSP-SPEC : It would be good to replace old names (caFe1 by caFe etc) in conversion code. We need MDB updates first [MDBICD-2620](https://issues.cosmos.esa.int/gaia/browse/MDBICD-2620) - **POSTPONED for cycle 5**.

4. Proposition to add the DIBSpec param table to the VDB (https://wiki.cosmos.esa.int/gaia-dpac/index.php?title=CU8:Processing_telecon_20230303)

## :calendar: Next telecon

April 4, at 11h15

# Minutes from the telecon on January 24, 2023, at 10h00

###### tags: `meeting`, `database`, `validation`, `cu8`, `dpac`, `gaia`
:::info
- **Location:** Zoom Meeting
- [Zoom link](https://cnrs.zoom.us/j/97306209507?pwd=cy83UXozSk5kc2lDc3E0ejJhR3VrQT09)
- **Date:** 2023/01/24, 10:00 CEST
- **Duration:** 60 min
- **Agenda**
1. Updates
2. Actions until the next telecon
3. Next telecon plan
- **Participants:**
- [name=Orlagh Creevey] (OC)
- [name=Ludovic Delchambre] (LD)
- [name=Alejandro Lorca] (AL)
- [name=Morgan Fouesneau] (MFX)
- [name=Inna Slezak Oreshina] (ISO)

- **Contact:** <[e-mail]>
- **Host:** Inna Slezak
:::

## :notebook: Status & Updates

### Environment (*from* [C8T-179](https://issues.cosmos.esa.int/gaia/browse/C8T-179))

| Apsis run | VDB tag | MDB DM | DPCC CU8 DM | CU8 DAF DM | CU8 DAF Mapping | DAF infrastructure | Ingestion dates |
| -------- | -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| [ApsisOps4.2c](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8_APSIS_4.2_Kick_Off_Meeting) | ops42c | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Dec 2022 |
| [ApsisVal4.1e](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:Apsis_runs_ApsisVal4.1e) | val41e | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.1 | 1.1.1 | Nov 2022 |
| [ApsisVal4.1b](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:Apsis_runs_ApsisVal4.1b) | val41b | 21.0.26 | 21.0.26a | 21.0.1 | R_21.0.0 | 1.0.1 | July 2022 |

We are working according to the **[Cycle04] Plan Apsis Development and Validation activities (2023/2024)** : [C8MNG-212](https://issues.cosmos.esa.int/gaia/browse/C8MNG-212)

### Status DONE :

1. **SESSION 2 of the WEBINAR on 16/12/2022** :

* [C8OPS-133](https://issues.cosmos.esa.int/gaia/browse/C8OPS-133) : Preparation for DAF hands-on.
* [C8OPS-138](https://issues.cosmos.esa.int/gaia/browse/C8OPS-138) : Execution of DAF hands-on

2. **Make ApsisOps4.2c (ESPCS scientific run) available in the VDB** [C8OPS-125](https://issues.cosmos.esa.int/gaia/browse/C8OPS-125):
* **DONE** on 22 December.
* **issue** with **flags in ESP-CS** is reported in [C8T-200](https://issues.cosmos.esa.int/gaia/browse/C8T-200) : values of the parameter flags can not be ingested into VDB

3. :a: **Inna** : "Update wiki FAQ with proposition for next VDB updates" - to announce the **end of March** as the deadline for the next VDB updates. - DONE [WIKI : News of VDB](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_validation_cycle3plus#News) and [Wiki : Datamodel](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_Astrophysical_Parameters#Datamodel)

4. **Add new *external* catalog to the VDB** :
* **SDSS Quasar Catalog** : [C8OPS-124](https://issues.cosmos.esa.int/gaia/browse/C8OPS-124)

5. :a: **Inna** will create a top-level Jira to ingest the ASP data into the VDB. - **DONE** [C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)
**Questions** :
DAF DM Fix Version DAF21.0.1 (?) - **YES**
DMC Fix Software Conversion Version DMC21.0.1 (?) - **YES**

### ONGOING & ACTIONS :

1. **ApsisVal4.1f : Test the new post-processing to strip large fields from ASP before delivery to DPCE** : [C8OPS-144](https://issues.cosmos.esa.int/gaia/browse/C8OPS-144)

2. **Add new *external* catalogs to the VDB** :

* **NearInfraRedAstroSource** : :a: **Morgan** : Add the table NearInfraRedAstroSource as an external catalog to VDB [C8T-210](https://issues.cosmos.esa.int/gaia/browse/C8T-210)

3. **Enforcing DM units and boundary checks** [C8T-192](https://issues.cosmos.esa.int/gaia/browse/C8T-192). :a: **Enrique** : work in progress.

4. :a: **Inna** : add to the VDB WIKI the information about the VDB webinar link : if this link does not work anymore (expired), users should contact Alejandro who will renew the link.

5. **Should we open VDB to CU9** to receive their feedback earlier ? - **YES, we should !**
Orlagh has asked the opinion of CU9 members - 6 colleagues from CU9 are interested in. Orlagh has sent an e-mail to them. We will have more details in a month.

### Other tasks

**future changes in VDB for the ApsisVal4.2b (for end of March 2023)**:
* Top-level Jira for changes in DAF DM (for version DAF_21.0.2) : [C8OPS-118](https://issues.cosmos.esa.int/gaia/browse/C8OPS-118).
* Corresponding top-level Jira for DAF conversion code (for version DMC_21.0.1) [C8OPS-119](https://issues.cosmos.esa.int/gaia/browse/C8OPS-119)
* Reminder for CU8 WP managers : Apsis code changes will be needed to do a scientific test of the mapping updates [C8CAL-779](https://issues.cosmos.esa.int/gaia/browse/C8CAL-779) (April 2023)
* :a: **Inna** : add these three Jiras to VDB Wiki page.

### Postponed tasks

1. Some WP managers want to see the same parameter names in VDB and GACS
* :a: **Alejandro** : we need to convert camelCase names (Dictionary Tool) into **names_with_underscore** (e.g. [C8DM-354](https://issues.cosmos.esa.int/gaia/browse/C8DM-354) for GSP-Phot) : [C8T-193](https://issues.cosmos.esa.int/gaia/browse/C8T-193)
* See also [DICT-590](https://issues.cosmos.esa.int/gaia/browse/DICT-590)
* Implimentation expected in the **end of March 2023** with the new Data Model.

2. GSP-SPEC : It would be good to replace old names (caFe1 by caFe etc) in conversion code. We need MDB updates first [MDBICD-2620](https://issues.cosmos.esa.int/gaia/browse/MDBICD-2620) - **POSTPONED for cycle 5**.

## :calendar: Next telecon

February 28, at 10h

## Previous archived telecons (2019 - 2022)
* [2022](https://hackmd.io/a7_3rMUKRpyZp_QTsIaF2w)
* [2021, part 2](https://hackmd.io/b4srOkL2TG25GmoY49zmXQ?both) (September - December)
* [2021, part 1](https://hackmd.io/xrg45HnHTTmYa_aChkuQvw) (January - July)
* [2020](https://hackmd.io/@mfouesneau/rkTdiTLx_)
* [2019](https://hackmd.io/@mfouesneau/HyxD3a8xO)

previous telecons on SVN and wiki :
* [svn:DPAC/CU8/docs/validation/valdb](https://gaia.esac.esa.int/dpacsvn/DPAC/CU8/docs/validation/valdb/)
* [wiki:CU8:_validation_cycle3plus#Telecons](https://wiki.cosmos.esa.int/gaia-dpac/index.php/CU8:_validation_cycle3plus#Telecons)

---