---
title: VRE Webinar Slides
tags: vre-webinar
description: View the slide with "Slide Mode".
slideOptions:
theme: white
transition: slide
# parallaxBackgroundImage: 'https://s3.amazonaws.com/hakim-static/reveal-js/reveal-parallax-1.jpg'
# allottedMinutes: 5
# spotlight:
# enabled: true
---

<style>
.reveal section p {
display: inline-block;
font-size: 0.6em;
line-height: 1.2em;
vertical-align: top;
}
.reveal section ul {
font-size: 0.6em;
}
</style>

<!-- TIPS
https://hackmd.io/s/how-to-create-slide-deck
https://hackmd.io/slide-example
https://revealjs.com/demo/
https://revealjs.com/themes/

Just paste images in - it autouploads to imgur

Images without borders:
<img style="border:none; box-shadow:none" src="" width="100">

-->

<a href="https://hackmd.io/@swarm/vre-webinar-slides">:film_projector: Full view</a> --- <a href="https://hackmd.io/@swarm/vre-webinar-slides?print-pdf">:page_with_curl: PDF print</a>
</small>

# Swarm VRE

> [e-mail]
> [GH: @smithara](https://github.com/smithara)
> All this is made possible by the work from EOX
> I am just providing some sugar on top!

> [e-mail]

<table><tr>
<td><a href="https://earth.esa.int/eogateway/activities/swarm-disc">
<img style="border:none; box-shadow:none;"
src="https://i.imgur.com/zI81dHt.png" width="140"></a></td>
<td><a href="https://eox.at">
<img style="border:none; box-shadow:none"
src="https://i.imgur.com/XwxxcTf.png" width="170"></a></td>
</tr></table>

---


https://earth.esa.int/eogateway/tools/swarm-vre

----

( Demo in VirES GUI then moving to VRE )

- [VirES](https://vires.services): browsing datasets; configuring data; configuring models; configuring plots; uploading & downloading
- [VRE](https://vre.vires.services): JupyterLab evironment with a pre-configured open source software stack to use with Swarm
- [`viresclient`](https://viresclient.readthedocs.io/en/latest/installation.html) provides access to VirES - you can alternatively install it in your own Python environment

---

### Get data in a few lines of Python: `viresclient` package

Install it anywhere: [`pip install viresclient`](https://viresclient.readthedocs.io/en/latest/installation.html)
(Preloaded in VRE - start at [the introduction notebook](https://swarm-vre.readthedocs.io/en/latest/Swarm_notebooks/02a__Intro-Swarm-viresclient.html))

from viresclient import SwarmRequest

ds = (
    SwarmRequest()
    .set_collection("SW_OPER_MAGA_LR_1B")
    .set_products(["B_NEC"])
    .get_between("2016-01-01", "2016-02-01")
    .as_xarray()
)

(`.as_xarray()` loads the data as an `xarray.Dataset`)
(or use [`.as_dataframe()`](https://viresclient.readthedocs.io/en/latest/api.html#viresclient.ReturnedData.as_dataframe) to get a pandas DataFrame)

----

ds

<xarray.Dataset>
Dimensions:     (NEC: 3, Timestamp: 2678400)
Coordinates:
  * Timestamp   (Timestamp) datetime64[ns] 2016-01-01 ... 2016-01-31T23:59:59
  * NEC         (NEC) <U1 'N' 'E' 'C'
Data variables:
    Spacecraft  (Timestamp) object 'A' 'A' 'A' 'A' 'A' ... 'A' 'A' 'A' 'A' 'A'
    B_NEC       (Timestamp, NEC) float64 -1.581e+03 -1.049e+04 ... -4.04e+04
    Radius      (Timestamp) float64 6.834e+06 6.834e+06 ... 6.834e+06 6.834e+06
    Latitude    (Timestamp) float64 -72.5 -72.56 -72.63 ... -80.63 -80.69 -80.76
    Longitude   (Timestamp) float64 92.79 92.82 92.85 ... 58.84 58.95 59.06
Attributes:
    Sources:         ['SW_OPER_MAGA_LR_1B_20160101T000000_20160101T235959_050...
    MagneticModels:  []
    RangeFilters:    []

Learn xarray:
https://xarray-contrib.github.io/xarray-tutorial/scipy-tutorial/00_overview.html

----

Newer versions of xarray give an html repr:
<iframe src="https://gistpreview.github.io/?00b17f32c3b669ffdb68364d90a87bc4" style="border:0px #ffffff none;" name="myiFrame" scrolling="no" frameborder="1" marginheight="0px" marginwidth="0px" height="600px" width="1000px" allowfullscreen></iframe>

---

### What's happening underneath?

<img style="border:none; box-shadow:none;"
src="https://i.imgur.com/vmCDPRI.png" width="1000">

---

### Subsetting and computation?

ds = (
    SwarmRequest()
    .set_collection("SW_OPER_MAGA_LR_1B")
    .set_products(
        measurements=["B_NEC"],
        auxiliaries=["QDLat", "QDLon", "MLT"],  # 
        models=["IGRF"],                        # Computed on demand
        residuals=True,
        sampling_step="PT10S",
    )
    .set_range_filter("Flags_B", 0, 1)
    .get_between("2016-01-01", "2016-02-01")
    .as_xarray()
)

----

#### Apply your own analysis directly using the Python ecosystem

ds.plot.scatter(x="QDLat", y="B_NEC_res_IGRF", col="NEC",
                hue="SunZenithAngle", cmap="viridis_r", s=0.1)

> Residuals to IGRF:
Magnetic field perturbations due mainly to auroral oval

http://xarray.pydata.org/en/stable/plotting.html

---

### Okay, what data can I access?

https://viresclient.readthedocs.io/en/latest/available_parameters.html


<!-- | Collection full name | Collection type | Description |
| -------------------- | --------------- | ----------- |
| SW_OPER_MAGx_LR_1B | MAG | Magnetic field (1Hz) from VFM and ASM |
| SW_OPER_MAGx_HR_1B | MAG_HR | Magnetic field (50Hz) from VFM |
| SW_OPER_EFIx_LP_1B | EFI | Electric field instrument (Langmuir probe measurements at 2Hz) |
| SW_OPER_IPDxIRR_2F | IPD | Ionospheric plasma characteristics (derived quantities at 1Hz) |
| SW_OPER_TECxTMS_2F | TEC | Total electron content |
| SW_OPER_FACxTMS_2F | FAC | Field-aligned currents (single satellite) |
| SW_OPER_FAC_TMS_2F | FAC | Field-aligned currents (dual-satellite A-C) |
| SW_OPER_EEFxTMS_2F | EEF | Equatorial electric field |
| SW_OPER_IBIxTMS_2F | IBI | Ionospheric bubble index | -->

----

#### New auroral electrojet and boundary products

----

#### Ground observatories (Swarm AUX_OBS)

----

#### Geomagnetic field models

---

### Improving accessibility of Swarm

> :zzz: [Official documentation of products ](https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/data-handbook/level-2-product-definitions)

> :+1: [Notebook guides to interact with products ](https://swarm-vre.readthedocs.io/en/latest/notebooks_preface.html)

----

#### Swarm_notebooks

- View and give feedback on [Swarm-VRE docs](https://swarm-vre.readthedocs.io/en/latest/notebooks_preface.html)
- Interact with them on VRE
- Load them from the Launcher in VRE
- .. or from links in the docs

<table><tr>
<td>
<img style="border:none; box-shadow:none;"
src="https://i.imgur.com/LSE6tu6.png" width="250">
</td>
<td>
<img style="border:none; box-shadow:none"
src="https://i.imgur.com/R0qQmr1.png" width="300">
</td>
</tr></table>

---

### Architecture of VRE concept

<img style="border:none; box-shadow:none;"
src="https://i.imgur.com/UmNyOQQ.png" width="800">

(the simple picture!)

----

<img style="border:none; box-shadow:none;"
src="https://i.imgur.com/I4DaeSC.png" width="800">

---

### Future development

- Better notebooks:
- Recipes closer to science applications
- Working together with data from other sources (e.g. [hapiclient](http://hapi-server.org/servers/#server=CDAWeb&dataset=OMNI_HRO2_1MIN&parameters=BY_GSM,BZ_GSM&start=2000-01-01T00:00:00Z&stop=2000-02-01T00:00:00Z&return=script&format=python))
- Easier to contribute to
- New datasets from Swarm (e.g. GVO - geomagnetic virtual observatories)
- *Quicklook* service with companion notebooks
- *Dashboards* ([example](https://vre.vires.services/user-redirect/lab/tree/shared/Swarm_notebooks/dashboards/0000_concept_demo.ipynb)) based on notebooks
- More functionality in `viresclient` + opportunities for related *Swarm* packages
- More integration with geospace Python ecosystem

---

### Talk with me: give feedback & get help

Contact info & open office hours:
https://smithara.github.io/
I am available for any discussions and training, email me: [e-mail]

### Future webinars & discussion

I will be improving mechanisms for discussion & collaboration
We should have regular future meetings on specific scientific topics

---

### Further reading

- Swarm-VRE guide
https://swarm-vre.readthedocs.io
- Magnetic Earth - intro to geomagnetism
https://magneticearth.org
- Reproducable & collaborative data science (The Turing Way)
https://the-turing-way.netlify.app
- Collection of open geoscience software
https://github.com/softwareunderground/awesome-open-geoscience
- Python in heliophysics
http://heliopython.org/
- Reproducability in space sciences (Resen / InGeo project)
https://ingeo.datatransport.org/home/resen

<!-- # magneticearth.org

<iframe width="100%" height=1000 src="https://magneticearth.org"></iframe> -->