# Vascular perfusion notes

## Insight of the dataset
### Input video

### Label distribution

## Preprocess the input videos
* Aim: to separate the main region of interest (ROI): micro-vessels from main vessels and surrounding tissues.
* micro vessel segmentation
* Frame by frame method:
* Ridge detector: ridges are eigenvalues of matrix of second order derivate of image, also known as hessian matrix.
* simple thresholding: due to the different pixel intensity of background tissues and blodd vessels, it's possible to obtain vessel trajectories via intensity thresholding as well.
* Volumetric method:
* Maximum intensity projection: is a method for 3D data that projects in the visualization plane the voxels with maximum intensity that fall in the way of parallel rays traced from the viewpoint to the plane of projection.
* opitcal flow
* Dense optical flow method
* Sparse optical flow method

## The proposed 2-step algorithm
### Motivation
The motivation behind this 2-step classification after segmentation approach is largely due to the fact that only small vessels are relevant to microcirculation classification. Gaint vessels and its surronding tissue play minor or none role in the classification problem.

## blood flow path segmentation

# the path of all segmented vessel maps can be found at:
'./dsg-uhb/flow_seg/'

# to develop a machine learning model, the splitted 2D maps are at:
'./dsg-uhb/tianyu/keras_data/train/class_a/'
'./dsg-uhb/tianyu/keras_data/test/class_a/'

* segment frame-wise blood flow

    import cv2
    def ridge(image, thres=180):
        ridge_filter = cv2.ximgproc.RidgeDetectionFilter_create()
        kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (8,8))
        morph = cv2.morphologyEx(image, cv2.MORPH_DILATE, kernel)
        ridges = ridge_filter.getRidgeFilteredImage(morph)
        mask = ridges > thres
        return standardization(np.invert(morph[...,0])) * mask

* we used ridge detection filter for vessel segmentation.
* As we found the SNR of our naively segmented images are quite low. We, therefore, used morphological operations such as dilation followed by erosion to denoise the segmentation map. Note, we observe the kernel size inside the `cv2.getStructuringElement()` function influences the quality of denoising.
* Later, we thresholded the map to futher reduce the background influence.
* lastly, we obeserve the input data is dark-field microscropy, to highlight the blood flow, we invert the intensity of inputs and sum up all paths across all frames to obain the final blood flow map.
> [we might want have healthy and ill patient segmentations here!]

## local blood flow classification
* Currently, we trained ResNet-34 on top of the segmented blood flow maps. The challenges we have are the training set is rather small and the label distribution is extremely imbalanced.
* To address the problem of the limited training set, we finetuned our classification network on the pretrained weights from ImageNet classification. Addtionally, we performed different kinds of data augmentation techniques such as random cropping, affine transformation, random contrast, random zoom in/out.
* To address the class imbalanced problem, we calculated a weighting factor for each class according to the formula: w_c = 1 - |P| / |P|+|N|.

# TODO
* try pretrained DL models on segmentation
* try to classify based on the latent of VAE