---
# System prepended metadata

title: Instance Flow
tags: [2021-UnityRL]

---

---
tags: 2021-UnityRL
---

# Instance Flow 
## Idea
* RAFT after some instance segmentation approach (MaskRCNN or YOLACT)
    * Get the instance of both frame first 

* Use the pre-trainded model from DensePose to generate

* How to incorporate RAFT into MaskRCNN?
    * Given the  bounding box of human instance in the current & previous frame, we can get the moving ranges of a human. 
    * Then the ***Moving range bounding box*** is fed into RAFT, this might improve speed and performance as the range is small?
    
## TODO List

## Paper survey

### YOLACT -- Real-time Instance Segmentation
* [Paper Link](https://arxiv.org/pdf/1904.02689.pdf) | [Github](https://github.com/dbolya/yolact) |


#### Performance
![](https://i.imgur.com/VzGim3n.png)

#### Architecture
![](https://i.imgur.com/7WDwYQI.png)

### RAFT-3D: Scene Flow using Rigid-Motion Embeddings
* [Paper Link](https://arxiv.org/abs/2012.00726)

![](https://i.imgur.com/2MJJaoD.png)

### OPtical flow
http://www.scholarpedia.org/

### Does computer vision matter for action?
https://arxiv.org/abs/1905.12887

### Deep Rigid Instance Scene Flow
* [Paper Link](https://arxiv.org/abs/1904.08913)
![](https://i.imgur.com/pSx5LJY.png)

### SEPARATING BACKGROUND AND FOREGROUND OPTICAL FLOW FIELDS BY LOW-RANK AND SPARSE REGULARIZATION
![](https://i.imgur.com/jKBmh8J.png)

### An Efficient Optical Flow Based Motion Detection Method for Non-stationary Scenes
* [Paper Link](https://arxiv.org/pdf/1811.08290.pdf)

### An Efficient Linear Method for the Estimation of Ego-Motion from Optical Flow
https://www.researchgate.net/publication/221114910_An_Efficient_Linear_Method_for_the_Estimation_of_Ego-Motion_from_Optical_Flow
## Tracking & Future Prediction
### Predicting Deeper into the Future of Semantic Segmentation
* [Paper Link](https://arxiv.org/pdf/1703.07684.pdf)

### FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking
* [Paper Link](https://arxiv.org/pdf/2004.01888.pdf)

### Multiple Object Forecasting: Predicting Future Object Locations in Diverse Environments
* [Paper Link](https://arxiv.org/pdf/1909.11944.pdf)

### Video Instance Segmentation Tracking with a Modified VAE Architecture
* [Paper Link](https://openaccess.thecvf.com/content_CVPR_2020/papers/Lin_Video_Instance_Segmentation_Tracking_With_a_Modified_VAE_Architecture_CVPR_2020_paper.pdf)

 