# Object fingerprinting and tracking Dec 2022

# Attendees
* Scott Hosking
* Ben Evans
* Andrew Fleming
* Arianna Sallili-james
* Kristina Ulicna
* Evangeline Corcoran
* Oliver Strickson
* Marjan Famili
* Nathan Day
* Sebastian Ahnert
* Bea Costa Gomes
* Louisa Van Zeeland
* Alan Lowe

## Agenda for Workshop - Tuesday 6th December 2022, Crick Institute, London

1000 arrivals + coffee
1015-1030 – introductions
1030-1045 – Arianna intro + slides from anyone working primarily on shape analysis
* Approaches and applications
* Commonalities
*
1045-1125 – Fingerprinting/shape analysis discussion
1125-1130 – Comfort break
1130-1140 – Slides from anyone working primarily on tracking
1140-1230 – Object tracking discussion
* Graph-based methods
* Btrack applications
* Alternatives?

1230-1300 – Lunch

1300-1400 – Common challenges, longer-term ambitions, integration with Scivision
* Synthetic data, labels and benchmarking
* Generic shape descriptor pipelines
* Tracking approaches – btrack or otherwise
* Datasets and models for Scivision
* Roadmap and upcoming opportunities (inc. key dates)
*
1400-1430 - overspill

## Notes from workshop

## First talk: Arianna Salili-James (NHM)
* Anything can be thought of as a shape
* mean from shape analysis (Karcher mean) - the mean still looks like the object
* distance metric between two shapes
* Geodesic (smooth curves) / shortest path between shapes
* shape methods
* SRVF path-straightening
* LDDMM
* Geometric currents
* Eigenshape Analysis (not elastic shape analysis method)
* Classification
* KNN classifer on distance matrix
* The majority of NHM specimens are insects
* extracting physical tags/labels found on specimens from 3D images to provide annoations/labels for ML

### Datasets
#### Outlines of Vases / Shells / Leaves
https://figshare.com/articles/dataset/Outlines_of_ancient_Greek_vases/14551002
https://figshare.com/articles/dataset/Outlines_of_Swedish_leaves/14551005
https://figshare.com/articles/dataset/Outlines_of_Gastropod_shells/14551044
#### ALICE Pinned Specimens (*Examples -- not the specific data used in talk*)
https://data.nhm.ac.uk/search/lowly-spare-beetle
https://data.nhm.ac.uk/search/freshwater-insects

### Libraries
* Distance computations (SRVF framework) - https://fdasrsf-python.readthedocs.io/en/latest/geodesic.html
* Karcher mean / tPCA (SRVF Framework) - https://fdasrsf-python.readthedocs.io/en/latest/curve_statistics.html
* Distance computations (Currents) - https://github.com/olivierverdier/femshape

### References
Salili-James, Arianna, et al. (2022) Classifying organisms and artefacts by their outline shapes. Journal of the Royal Society Interface 19.195: 20220493.

Srivastava, A. et al. (2011). Shape analysis of elastic curves in euclidean spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(7):14151428.

Marsland, S, and Sommer S. (2019). Riemannian geometry on
shapes and diffeomorphisms. Riemannian Geometric Statistics
in Medical Image Analysis : 135.

## Marjan's Talk
* Affinity-VAE vs Beta-VAE
* I have looked at improving classification using Beta-VAE by introducing additional regularisation of the latent space based on the shape affinity of known classes within our dataset
* The published can be seen here: https://arxiv.org/pdf/2209.04517.pdf

* [name=Ben] - It seems to me that there is opportunity both with Affinity-VAE and btrack to think about developing a generalised callable module to provide more versatile, configurable and potentially better representations of distances (or affinity) between shapes, based on the analysis methods Arianna was talking about. This could become one of the fundamental building blocks of the object fingerprinting and object tracking themes and a USP for Scivision

## Evie's talk
* seed pod / flower shapes
* Alan forward this link: https://en.wikipedia.org/wiki/Constructive_solid_geometry ([name=Alan] - I thought this may be interesting in general to think about shapes as constructed from primitives, and then looking at fragmentation processes, such as with icebergs, as a decomposition back into fewer primitives) [name=Ben] - I'm looking for ways to meaningfully tesselate candidate smaller icebergs into a parent iceberg shape, but also sensible ways of determining when icebergs are touching and need to be split into separate objects so this sort of thing could potentially be a useful basis. Is there a more general set of use-cases for representing fragmentation processes? Maybe landscape or habitat connectivity? Anything biomed?

## Nathan's talk
* Tracking use case: Recording the intracellular growth of Mtb (Mycobacterium Tuberculosis) in human macrophages

#### Open datasets for cell tracking
http://celltrackingchallenge.net/datasets/

(From Kristina's data)
https://rdr.ucl.ac.uk/articles/dataset/Cell_tracking_reference_dataset/16595978

# Actions:
* Sebastian, Louisa and Arianna to chat about shape extraction from acoustics for inclusion in scivision
* this could be the basis for a joint proposal
* Lets start by setting up a workshop
* Ben/Alan to consider running a quarterly(?) shape and tracking workshop
* This will help to inform what we need for scivision - what the new PDRA should work on to join all our use-cases together
* Marjan, Arianna and others(?) to setup a meeting to discuss metrics for identifying shapes
* Evie, Arianna, Sebastian - identify other plant based datasets within NHM and Kew for scivision
* could become a joint proposal at somepoint, but put some ideas in place ready for future opportunities
* Nathan, Ben, Ed to discuss getting tracking relevent datasets into scivision in Jan/Feb