# TDMS Master Meeting Doc

# Meeting (final 😢) 2023-06-13

* Working on Ed... some remaining issues with En.
* Windows instructions now tested and merged and.... online?
* Remaining issues to be tidied but otherwise fine.
* ARC project closeout.

### Discussion

HDF5

* Compatible with caveats.
* Complex numbers and zero-sized arrays.

Sam to add a note to dev docst ststemt tests avbout physically meaningless tests and arc_12 is meaninful.

* Examples to follow - Peter will email.

# Meeting 2023-05-23

* dx.doi.org/10.5281/zenodo.7950604
* RPS (is this important to Peter?)
* https://dashboard.rc.ucl.ac.uk/software/125
* Careful merging of changes.
* Licensing MATLAB answers.
* Seems it's actually the opposite of what I told you Will. But can't confirm. Test with the runtime thing.
https://stackoverflow.com/questions/35865253/can-i-redistribute-libmat-dll-with-my-program
https://uk.mathworks.com/help/matlab/matlab_external/sharing-mat-file-applications.html

## 2023-05-23 with Ed

* Used powershell
* En used VisualStudio code.
* x

# Meeting 2023-05-11

* Easter was nice.
* Merge 262, 203 tag v1.
* No physics only admin.
* Coverage.
* Detatch from MATLAB.
* Will has ripped it out from Tensor3D and working on Matrix.
* Writing out is the next.
* Creating iterateftdt matrix creatuion via MATLAB
* [Action] write with v7.3 in the README.
* Windows. Joy? Need more docs?
* WSL.
* Postdoc is proficient... but not a sw dev.
* Spent 2h. Couldn't get it to compile.
* CMake [prereq?]. SDK (of go via WSL recommend that?)
* Ed.
* LLVM is Apatche is this ok with GPL (us + FFTW) if so can we release binaries.
* MATLAB?
* Zenodo @ v1.

------

Sam's week:

* Rebase Will's PRs.
* Doc funding for Peter.
* Website polishing not Peter.
* Versioning PR.
* GH LFS.

# Meeting 2023-04-13

### Discussed.

* Happy Easter.
* Update to FDTD matrix input arguments is ready to go in.
* Concludes the current feature requests.
* Peter: physical meaning of variables still TODO.
* Abstract submitted was accepted. Talk 27th June.
* Need good examples.
* Need to update the doc pdf.
* Autobuilt every test.
* Version zero → version 1.
* Does peter want ust to change focus before his talk?
* No.
* Sam: flatlining.
* Royal society logo?
- [x] Leave it off.
- [x] **Biomedical** engineering.
* The development of this code has benefited from funding from the following bodies:
- [x] Add Commonwealth scholarships fund.
* ["Commonwealth Scholarships Commission"](https://cscuk.fcdo.gov.uk/about-us/scholarships-and-fellowships/) EPSRC, Australian Research Council

# Meeting 2023-03-28

### Discussed.

* Time budget is on track.
* Recieved new tests: compact source flag.
* Sam planning to focus on documentation and finalisation.
* dz: $\lambda/4$ problem is only matlab script-facing. So not actually a **bug**, bug.
* Being free of MATLAB is a priority for Peter.
* Potentially its better to remove the executable dependency. Certainly better use of ARC personpower.
* Future: port to python (it's an easier person-power issue to solve).

- [ ] To check: does one need a MATLAB license to link against?
- [ ] Sam's five-minute check: does GitHub LFS help?

* Wills plan:
* Run tdms on this feature.
* Update inputs. Don't change exe.
* Give it to github.

---
# Meeting 2023-03-07

### Discussion plan.
* ~~Will to explain test restructure and input generation. Preferrably with a diagram?~~
* Then separate questions from Will.
* ~~Website stuff.~~ And endmatters plan.

### Discussed.
* To check:
* Royal Society logo (can we use?) and grant number.
* Names of collaborators from Peter.
* dz/2 only for PSTD dz is Yee cell size.
* Should change the approach here. Either field always takes x,y,z,lambda, refractive index.
* Or we put in some kind of struct.
* IterateFDTD matrix: Will to reconcile the Peter version of iterate_fdtd_matrix with the Tom version from [#37](https://github.com/UCL/TDMS/pull/37/files#diff-70bee4900c3a6885ff0744110464b9fb6e1564a199ff671f8386d028dd6b1d0e) and change behavior to accept if effname and hffname are defined when passing in an ilfile.
* Input flags.
* PSTD and interpolation useCD (useFD) ==1 is the default and should probably be useFDTD (which is zero by default).
* Ditto the default is cubic. And we switch to use band-limited.
* Cross testing from last meeting

# Meeting 2023-02-07

* Will has been implementing [#224](https://github.com/UCL/TDMS/pull/224) and [#227](https://github.com/UCL/TDMS/pull/227).
* Sam has been C++ checking hence scope bugs discovered. (Have not had a full week yet.)
* Discussion w/ Peter: :bug:s as described in pre-meeting notes.
* Will can make the change suggested in [#225](https://github.com/UCL/TDMS/issues/225) un order to implement. Peter will take [#225](https://github.com/UCL/TDMS/issues/225) formally and dig in later.
* About [#226](https://github.com/UCL/TDMS/issues/225): should use the same pattern. Move 154-8 outside of if statement.
* Segfaulting test: Lines 111ff. of loop_variables.cpp. If you add the new functionality, get differing tests: good to quantify the disagreement in output files? 1e-1? 1e-2?
* Wills criss cross whiteboard tests.
* $\lambda/6$ was used - should be in a regime where cubic is better.
* Q for Peter any other priorities in the milestones?

# Pre-meeting 2023-02-07

Peter's feature issue is split into two parts: [#224](https://github.com/UCL/TDMS/pull/224) and [#227](https://github.com/UCL/TDMS/pull/227). The latter depends on the former, as it requires main loop functionality that is implimented there. The former is passing all the tests, however the below implies that our test coverage is not complete in the first place.

Things to run past Peter:
- [#225](https://github.com/UCL/TDMS/issues/225) and [#226](https://github.com/UCL/TDMS/issues/226): which are both blocking [#224](https://github.com/UCL/TDMS/pull/224).
- Blocking [#227](https://github.com/UCL/TDMS/pull/227): `arc_17` and `arc_18` tests throw seg-faults on TDMS versions all the way back to (at least) `v0.0.1`. IE, before the interpolation changeover. This occurs in the code that tries to [optimise the Jsource loop](https://github.com/UCL/TDMS/blob/5349a753dcb32ca4dcdb676afc795ece8788271f/tdms/src/simulation_manager/loop_variables.cpp#L111-L122) - specifically, we can get to here and `Ksource` can be unassigned (`real` & `imag` are `nullptr`s).
- Which version of TDMS is Peter using to obtain the test data? And what is this doing differently?
- Should we ask Peter to regenerate the test data using the latest version of TDMS on `main`?
- We _also_ need to regenerate the input files for the original tests (`01` through `example_fdtd`) accordingly. Then re-upload these to Zenodo.
- The `arc_17` and `arc_18` inputs and outputs are not bijective. `arc_18` doesn't have a `fs_input` file, but appears to have an `fs_output` file.

# Meeting 2023-01-24

* Project status
* Plans milestones
* Issue 208 by diagrams.
* Refactor plans?

Action: send list for the unknown variables.
* Pupil
* n_det_modes
*

# Notes on [#208](https://github.com/UCL/TDMS/issues/208) pre-2023-01-24

### Main Loop Alterations

These changes are rather self contained, and do not require any major reworking of the current test framework or how `tdms` behaves on the command line.
_Where_ in the codebase these changes occur will be in either `execute_simulation` or `simulation.execute`, depending on post or pre [PR215](https://github.com/UCL/TDMS/pull/215)).

- Whenever `{I,J,K}_source` is non-empty, the corresponding field update lines in the main loop need to be executed. These are a subsection of lines [2121 through 2231](https://github.com/UCL/TDMS/blob/384a91ab8d0b24bc2d48233d710d2f4e4cf14f64/tdms/src/simulation_manager/execute_simulation.cpp#L2121-L2231).
- If all `{I,J,K}_source` are non-empty, then outputs should not be normalised. The functions listed in the issue have been superseded by class methods, with the lines not to be executed being [here](https://github.com/UCL/TDMS/blob/1924201bdad21465f6aa05677b4294eed7993832/tdms/src/iterator.cpp#L4222-L4242) (or in the `post_loop_processing()` method if [PR215](https://github.com/UCL/TDMS/pull/215) is merged).

#### To clarify:
- There is already conditional logic surrounding each part of the codebase to be edited - is the new condition of `{I,J,K}_source` all being empty a replacement, an additional constraint, or an alternative?

### Specifying `usecd` and `intmethod` in the input file

These changes are going to touch multiple files at once, _and_ require that we update the system test framework since we no longer need to read in command-line options.
- The `InputMatrices` class needs to be able to read in the `usecd` and `intmethod` variables and set the corresponding `enums` accordingly. `ArgumentSpace` looses this functionality. Reporting the values of these options should be moved from the `main()` body into the `InputMatrices` class.
- `SimulationManager` can be initialised with less arguments if `InputMatrices` now reads the aforementioned attributes (subject to [PR215](https://github.com/UCL/TDMS/pull/215)).
- The default interpolation method is currently band-limited, however the default needs to change to cubic. Also, there are currently two `enums` that encode the interpolation method that is to be used - these need to be reconciled.
- Test files need to be updated and re-uploaded to Zenodo. In addition, the `test_sytem` script needs to be altered, since it is no longer necessary to specify command-line options in the tests.
- The new tests need to be uploaded to Zenodo (and we should decide whether or not we are using cubic or band-limited interpolation). Additionally, we might want Peter to run his reference inputs through the current version of `tdms`, where the interpolation method is consistent (as older versions used a mix of BLi and cubic).

# Notes from the TDMS catchup meeting 2022-12-13

* Apologies for not coding the input file pstd/fdtd. And inputs better handling of null etc.
* Peter's on test beam so might not have time this week.
* Interpolation is ready to be pushed. ASA tested.
* Need to keep cubic as an option.
* Waiting on "gold standard" test case: **but** propose to sign off. Expect the algorithm is correctly implemented.
* PR 151 is ready to go many tests updated but then fine.
*

# Notes from the TDMS catchup meeting 2022-11-29

* Analytical test case was interesting.
* Why not uniform error? z-component mght be expected to be worse. But Band-limited.
* Could investigate deeper. But danger that this becomes a research project.
* RE next steps: extract the data from high res sampled fields. The error blows up for peter's tests. Guess it's some trivial bug. (Incorrect downsampling?)
* Once fixed then we'll have covered all field interpolations.

TODO: change band limited function in "versus MATLAB" tests to be $\sin2\pi x$.

* Read the "revert to cubic" from input file. For the time being we do a command line option.
* Will test at boundaries of PML etc. Then scatter. Then sign off.

# Notes from TDMS catchup meeting with Peter | 2022-11-08

* [Peters analytic example](https://github.com/UCL/TDMS/pull/151#issuecomment-1305659093) doesn't include the B field.
* A fast solution: high spacial res. PSTD sim.
* Then down-sampled the field.
* Get an error $\sim\pi$.
→ would like to see the error output with the BLi.
* ATM error is ~30% in problem field components.
* An error below 1% is pretty good and we might be there.
* Will can do this "easily" w/ TDMS current version.

* Peter shows the script in the latest arc14.zip
* In run_pstd_bscan.m datref is the "gold standard" data.

      > run_pdst_bscan
      > erran

* Numerical error Ez & Hy are large.
* Camps sample is user-specified vertices where the field is sampled (only E field?).

* Will can run this maybe later today.

* MATLAB datatype and file type removal. Tom refactored parts of iterator.cpp which gets us some of the way there.
* "Should" be able to switch relatively easily.

* `tdms --finite-difference` option in the newest version.
* PSTD has some subtle differences in setup. Might make more sense to have user-specified method choice in input files.
* Probaly means removing this option and reading/recording in i/o files.
* Ultimately we will "hide" this from non-expert users.
* Peter will make a plan (for both of these) and share it.
* Peter is at a conference next week. So will be asynchronous.
* Pupil? Is this really a pupil? Like an eye pupil?
* PhasorInc?
* `PhasorIncrement` (do you do every integer in the cell mesh or every first/second etc). `SurfaceVertexSpacing` `SurfaceSpacingStride`.
* CMaterial, CCollectionBase, DCollection DCollectionBase, DMaterial
* Peters thesis 4.7-4.10, 4.13-4.16
* Citation file OK? Is there a citable reference?

# Questions/notes/agenda for TDMS catchup meeting with Peter | 2022-11-01

## Project duration

* All OK for time. Sam can't access Worktribe.
* Happy with todo "list".
* Two extra issues to be added...
1. exi functionality : two sources of field
Currently apply both sources, wlt only specify one (notes below).
What was populated, might now be empty.
2. Specify: region, time iterations.

### Feature 1.

In general if you have a

$$
g (t) = \int_{-\infty}^{\infty} G(f) e^{-i\pi ft} {\rm d}f
$$
Time domain (exi functionality)

$$
\hat g(t) = \Re[G(f_0)\cdot e^{-i\pi f_0t}e^{-((t-t_0)/w)^2}]
$$
Fourier transform this ... + some magic...

End up with
$$
G(f) = G(f_0)e^{-(f-f_0)^2w^2}
$$
Extrapoating field as an approximation is actually fine. Can divide by this factor and normalise it out.

At the moment it's possible to have both of these inputs defined.

## Main discussion, interpolation

* Will's interpolation functionality: are we OK with the diff?
* Show and tell plots. Can we decide what makes sense?
* We also WLTU the plots themselves.

Plane with discontinuity where the source is inrroduced. This could play games with the interpolation.

* Any idea of quantified difference? / how to test.
* Arc03 has descritisation of lambda/4.
* Would expect cubic interpolation schemes to perform badly. Band limited should be the correct one. So potentially Will's is "correct". And the reference is "wrong".
* Could manually check the interpolation (look at non-interpolated data in the output)
* Or analytic test(s)
* Free space field (problem at discontinuity).
* Cylinder.

exi is interpolated
exout is un-interpolated