# VirES - future plans

### Datasets:
- AUX_OBS:
- minute :heavy_check_mark:
- second :heavy_check_mark:
- hour :clock2: *CDF files now ready*
- Use of `ObsIndex` variable
- `Quality` are currently all `Q`, to be modified in future release
- Locations (`Latitude`, `Longitude`, `Radius`) given are not identical to the minute/second ones (fix not expected)
- GVO :clock2: (waiting on project conclusion)
- Preliminary information and products at:
https://www.space.dtu.dk/english/research/projects/project-descriptions/geomagnetic-virtual-observatories
- PRISM :clock2: (waiting on project conclusion)
- https://www.gfz-potsdam.de/en/section/geomagnetism/projects/prism-plasmapause-related-boundaries-in-the-topside-ionosphere-as-derived-from-swarm-measurements/
- Swarm Echo (e-POP / Cassiope) MGF
- test data [from U Iowa](https://space.physics.uiowa.edu/swarm/epop/data/2019/)
- LR (1Hz?) & HR (160Hz?)
- Problem with too large file size (daily ~GB) of 160Hz data
- Swarm TII cross-track
- 2Hz & 16Hz on [swarm-diss](https://swarm-diss.eo.esa.int/#swarm/Advanced/Plasma_Data)

### VirES/VRE extensions
- SIFACIT outputs: notebooks at https://github.com/Swarm-DISC/FAC_exploration
- Load into VRE with nbgitpuller link
- Add a list of nbgitpuller links to the VRE launcher?
- AC-B counterrotating orbits and conjunction points
- More generally an arbitrary conjunction-finding tool? Is [SPICE](https://www.cosmos.esa.int/web/spice) part of the solution?

### Testing & maintenance
- Need for systematic automatic testing of VirES API as new data comes in
- Requires a reliable database of Swarm metadata (i.e. the contents of files at `swarm-diss`) to verify against
- See *SPASE* below for the wider solution

### Interoperability
- **HAPI** is a long term standard for accessing a wide array of **heliophysics** data:
https://hapi-server.github.io/
https://drive.google.com/file/d/1PE5XJaOYJlR_gZsC34nilBLnKbeCSRC8/view?usp=sharing
- Cluster (an ESA mission) is available through HAPI [(example)](http://hapi-server.org/servers/#server=SSCWeb&dataset=cluster1&parameters=GSE_Magnetic_Vector_X&start=2000-08-22T00:02:30.000Z&stop=2000-08-24T00:02:30.000Z&return=image&format=png)
- Can we add a HAPI-compliant interface to VirES?
- It probably needs to be open access (without a token), so perhaps a separate server with some VirES features disabled?
- Using HAPI would allow Swarm data to automatically plug into other tools
- *Can HAPI outputs include ISTP-labelled CDF?*

- **SPASE** is a metadata system connected with HAPI (both endorsed by [International Heliophysics Data Environment Alliance](https://ihdea.net/))
- [The SPASE Data Model: A Metadata Standard for Registering, Finding, Accessing, and Using Heliophysics Data Obtained From Observations and Modeling](https://doi.org/10.1029/2018SW002038)
- Swarm dataset descriptions should be added to SPASE at the [hpde/ESA](https://github.com/hpde/ESA) repo
- SPASE describes the data and where to access it; HAPI provides the access mechanism; together they enable tools like https://heliophysicsdata.gsfc.nasa.gov
- SPASE records can be used for DOI minting [(further intiatives are expected here)](https://hpde.gsfc.nasa.gov/DOI_Initiative_October_2020.pdf)

- Use of SPASE & HAPI will embed Swarm within the heliophysics data environment:
VirES would be the engine working underneath to provide the HAPI interface
Could find similar pathways to mesh with **other geo-focussed data environments**

- **ISTP CDF conventions** are a set of rules for how to label data in CDF files:
https://spdf.gsfc.nasa.gov/sp_use_of_cdf.html
- These rules enable tools to automatically interpret the data to correctly make plots etc.
- The rules prescribe variable names and variable attributes
- Swarm products do not follow these
- **Should we make Swarm products compatible?**
*Do the rules require that Swarm product variables be renamed?*
*or do we only need to add extra attributes?*
*> What changes to Swarm data is necessary to be compliant?*
*> Should this be done in the source files (adjusting all processing chains)? or it could be done to VirES outputs (requiring to define the mapping)?* or not at all?
- **VirES should be able to interpret uploaded ISTP-labelled data**
*What should be added to VirES to enable proper interpretation of uploaded ISTP-compliant data?*