# Archive

## 10/05/2023

- Review accessibility of SwarmPAL now, including usage on VRE
- What are the next targets to look at?
- performance, onboarding guides, ... or functionality like visualisation
- NB. DSECS final project meeting probably 22/23 May
- see about creating preset visualisation like 
- performance is an issue for interactive use

## 12/04/2023

- Reviewed new TFA system
- Review SwarmPAL 0.1 release
- DSECS: any issues?
- TFA: any issues?
- Documentation etc update: https://github.com/Swarm-DISC/SwarmPAL/issues/20
- Should we add contacts (email) to toolbox intros?

## 15/03/2023

- Discussed DSECS toolbox issues
- Review what to include in documentation
https://swarmpal.readthedocs.io/en/staging/
https://github.com/Swarm-DISC/SwarmPAL/issues/20
- SwarmPAL publishing: Update the [CITATION.cff](https://github.com/Swarm-DISC/SwarmPAL/blob/main/CITATION.cff) with coauthors; discuss how people should acknowledge their usage of SwarmPAL

## 01/03/2023

- Discussed usage of datatree and `data.swarmpal.apply(process)` approach
See https://swarmpal.readthedocs.io/en/paldata/toolboxes/shared/paldata.html

## 23/11/2022

- TFA project final review meeting on 21/11
- (notes:)
- SwarmPAL: PalData system (https://github.com/Swarm-DISC/SwarmPAL/issues/12)

## 09/11/2022

- PyHC meeting this week, see https://heliopython.org/meetings/fall2022/
- Article published: "Python tools for ESA’s Swarm mission: VirES for Swarm and surrounding ecosystem" https://doi.org/10.3389/fspas.2022.1002697
- blogpost upcoming on [EOX website](https://eox.at)
- Upcoming VirES release will include better performance models (implemented in [eoxmagmod package](https://github.com/ESA-VirES/MagneticModel)) (maybe x10 improvement for full CHAOS model)
- Still need to plan hackathon in December (push back to January?)

## 26/10/2022

- Welcome to new folk & show how these meetings work
- SwarmPAL status update
- Brainstorm ideas for future meetings

Actions:
- Arrange a hackathon in December(?) to build examples. Invite to DISC mailing list to find e.g. students
- AS to improve generality of data class in swarmpal
- Check out Kamodo (https://ccmc.gsfc.nasa.gov/tools/kamodo/) for plugging other models in, and data-model fusion
- the demo is nice: https://ccmc.gsfc.nasa.gov/Kamodo/demo/

## 10/10/2022-14/10/2022: DQW at Uppsala

- swarmdisc.org/dqw-archive/
Presentations from DSECS & TFA & SwarmPAL

# 2022/09/28

1. Presentations at DQW (Wed 12th October, https://swarmdisc.org/dqw-2022/programme/)
- DSECS & TFA project talks before SwarmPAL talk at end of session
- [SwarmPAL slides](https://docs.google.com/presentation/d/1XCTXCZ9bXeykI-JdagN4PAZWs6DEs39lbQiF-3bfFIY/edit?usp=sharing) (work in progress)
2. SwarmPAL demonstration at DQW
- AS to show some example usages of the package
- Could setup a demo for people to try on the VRE (a notebook which first installs the package then walks through an example)
- I've set up a tool on binder now: https://github.com/smithara/swarmpal-demo
(creates a temporary environment on a free service, with the package installed - both JupyterLab & VSCode interface)
3. Went through synchronsing the secs development branch

# 2022/09/14

0. viresclient documentation improvement?
https://viresclient.readthedocs.io/en/docs_improve/capabilities.html
2. Discuss i/o flexibility for data input manually or from file
3. Discuss adapters to other data models in pyhc.org/projects/

# 2022/08/31

> Initial regular developer meeting

1. SwarmPAL: Swarm Product Algorithm Laboratory
- see biopal.org for BIOMASS
1. give it a more meaningful name
2. not overlap with other "swarmx" names on the internet already
3. align with existing ESA project
2. presenting SwarmPAL at the data quality workshop in October
- to set up a v0.1 release as a demo
3. how these meetings should work
- biweekly Wednesday at 13:00 CEST
- will set up mailing list open for anyone to self-subscribe: used to circulate small discussion & pre/post-meeting agenda & minutes
Now at https://groups.io/g/swarm-dev
- will send a calendar invite for the next meeting
- Initially focussed on SwarmPAL development with DSECS & TFA project teams but could extend in the future
4. anything else?

## 2022/07/11

> TFA

- talking points for next week
- tfalib -> OOP setup with classes: TFA Data (preprocessing) & TFA Process (each step is a subclass from a "Process" abstract base class: Cleaning, Filtering, Wavelet .., which apply to the data)
- 

## 2022/07/05

> DSECS

Agenda:
- Process of building on current staging branch of swarmx
- Changes should be made on a new branch based on `staging`. If `staging` is getting ahead, *merge* changes into your branch from staging. Try to split work into shorter chunks that can be composed on temporary branches (that only live during the dev work for that chunk), whose commits are *squashed* and merged into staging through a Pull Request.
- Documentation is built automatically for the staging branch at https://swarmx.readthedocs.io/en/staging/
- Problems with current setup
- any trouble installing the package for development and connecting your IDE?
- able to run the notebooks in the docs part?
- able to run pytest?
- Progress of python conversion of Matlab code
- Structure of the DSECS python package
- `swarmx.toolboxes.secs`, *secs* naming okay for the module? maybe it can be extended with tools for the normal secs case in the future?
- could incorporate or use https://github.com/greglucas/pysecs and/or https://github.com/klaundal/secsy/
- should change `SecsInputs` to `DSecsInputs`
- Possible changes from Matlab-code (variable and function naming etc.)
- Testing
- will need to make some decision about how to handle test data
- Meetings
- Could begin a fortnightly optional drop-in meeting at a fixed time?

Notes:
- automated quality flags that indicate suspicious data, e.g. extreme unphysical values, poor fit indicated by residuals
- naming conventions
- autoformatting with black and pre-commit

## 2022/05/02

> Meeting between AS and DSECS members (TH & SK) - discuss DSECS data input

Agenda:
- Intro SwarmX repo and process
- Discuss data input requirements
- See https://swarmx.readthedocs.io/en/dsecs-input-attempt/toolboxes/secs/using_secs.html
- Would that approach work?
- What needs to be added?
- Apex coords: Add apexpy as dependency to swarmx
- name them like `ApexLatitude` ?
- Different data access method from just start/end time? e.g. by orbit number
- Subselect to mid-lat range / segmented into consecutive passes
- Building layer of algorithm application
- lower level of functions expecting numpy array inputs, with higher level function using the data object supplied by `SecsInputs`
- use the lower level functions to write some test code taking artificial data?
- AOB:
- Getting credit
- package will be citable via zenodo with all contributors attached
- each toolbox should have a paper to accompany it
- SwarmX docs will have a "Acknowledgements/Citations" page to detail how and what users should cite

## 2022/02/16

> Meeting between AS and members of the TFA team to discuss SwarmX.

- AS gave an overview of the rationale of SwarmX and the process for how we make it work. General agreement that it is a good approach
- Discussed development environments/tools a bit. CP is currently using Anaconda and Spyder for Python work.
- L/Lm/L* are needed for extended features. Could be provided by IRBEM via SpacePy - this is a pain to add as a dependency for SwarmX - AS mentioned possibility to add within VirES Server as auxiliary parameters