---
# System prepended metadata

title: MLC Seminar on Surgical Video Analysis

---

# MLC Seminar on Surgical Video Analysis

## MLC community news

Share some news from your side of the bubble with the Dresden ML community. This can be anything from papers you published or you liked, blogposts, job advertisements, ...

- HZDR and HTW Dresden again co-organize the "Teaching ML" workshop at ECML/PKDD 2021, for more details follow our website at https://teaching-ml.github.io/2021/
- a TU Dresden member has kindly joined the MLC organisation team (More volunteers are welcome!)
- Dresden company searching for a Senior Bio-Image Analyst offering ML experience in computer vision https://www.scionics.com/jobs.html#SBIA
- 

## Title: Deep learning for video recognition and applications in surgical video analysis by Isabel Funke

More details about this talk can be obtained from [the mlc homepage](https://mlcdresden.gitlab.io/pages/seminars/funkeseminar-on-20210504/).

## Code of Conduct

We facilitate this meeting under the [Dresden Code of Conduct](https://dresden-code-of-conduct.org/de/). **Please be mindful of your peers and supportive in your communication!**

## Questions and Notes

- the sports-1M dataset: https://github.com/gtoderici/sports-1m-dataset (looks like the 1M refers to the number of videos contained, the `labels.txt` lists 487 singular entries -> I guess then there are 487 classes contained)
- the kinetics-400 dataset: https://deepai.org/dataset/kinetics-400 (`~650k video clips covering 700 human action classes`)

- Q: Why would one want to automatically assess training? To grade a student without need of someone actually looking at the performance?
    - similar problems exist for classic ML approaches (who needs a DL network that says if a dog/cat is in an image)
    - first step to see if this is doable
    - classifier might be integrated into teaching
    - might help surgeons to establish reassurance (in case surgeons start being tired)
- Q: on slide 14: how would one choose the time window width? (depending on the action and the time scale that it has, choosing the right time window might be crucial)
    - good point, segment window length is a hyperparameter (article reported that 10 segments worked well)
- Q: for practical applications, how are label predictions affected if there are multiple types of activities contained in a clip? I assume that approaches discussed upto slide 16 (video recognition) are trained with a cross-entropy loss. So what happens if I have a video with someone hula hooping AND someone throwing a boomerang. how is the softmax effected by this? 
    - most of the training data has only one label (result of curating)
    - other datasets exist with multiple labels per video
- Q: What about video recognition where objects should be localized/detected (segmented) as well? Are these a whole different family of architectures or similar to the ones introduced?
    - focus of research 
    - video datasets for localisation exist -> not of interest (yet) for Isabel's application
- Q: can the 3D architectures developed mainly for temporal tasks also be applied to 3D medical images (e.g. MRI) where the third dimension does not correspond to time but rather space?
    - good question
    - volumetric community has their own architectures (3D Unets)
    - pretraining on videos tricky if not impossible
- Q: have attention mechanisms also been investigated for video recognition (maybe to replace recurrent networks) yet?
    - for surgery phase recognition in videos, this happened
    - might need further research
    - non-local blocks (attention mechanism) -> access information that is not local
    - recent related publication by facebook research using vision transformers for segmentation in videos (AFAIK), explained by Yannic Kilcher https://youtu.be/h3ij3F3cPIk 


