# Automatic tracing of DNA damage on microscope films.

## Intro

<p style='text-align: justify;'> The main objective of the project has been the analysis, of a series of movies that have been obtained from a fluorescence microscope. In the videos, it can be seen how in the cell nucleus fluorescence-marked repair foci had formed due to double breaks in the DNA strands (DSB) as a result of ionizing radiation.
<br/><br/>
Several different phenomena can damage the integrity of the genetic material of mammalian cells, such as ultraviolet light, ionizing radiation, mutagenic compounds, metabolic activities and errors in DNA replication. The most serious of these is DNA double-strand breaks, which often lead to cell death, potentially oncogenic mutations or chromosomal rearrangements, if not repaired in time.
<br/><br/>
The distribution of clustered DNA lesions, strand cross-links and base/nucleotide alterations determine the cellular DSB response. X-rays generate DSBs which are characterized by low linear energy transfer and are considered relatively simple. However, DSBs generated by accelerated heavy particles of high LET, are accompanied by numerous lesions, including DSBs. Regarding the latter type, complex DSBs, require extensive processing by several enzymes. They also have a higher relative biological efficacy in inducing genomic rearrangements or cell death.
<br/><br/>
There are differences in cellular responses to simple and complex DSBs. Several studies showed that proton-induced DSB repair requires different enzymatic machinery compared to X-ray-induced injury. Which could be used to prepare new therapeutic strategies. Despite this, there is limited knowledge of cellular responses to DNA injury and new methods are needed to further study the DNA damage response in the detail.
<br/><br/>
The proteins involved in DSB repair share an important feature, namely their affinity for the broken ends of DNA and the surrounding chromatin, resulting in the formation of focal accumulations around DSBs. They were first discovered more than two decades ago in this part called "foci", which are indicative of well-functioning repair machinery. In particular, real-time imaging of cells expressing modified fluorescent fusion proteins has revealed important insights into the functioning of DNA repair in response to various injuries, mainly those induced by visible and UV light.
</p>

### Interpreting films as 3D object
<p style='text-align: justify;'>
Films can be interpreted as a set of images. They have dimensions of image shape for each frame and also a time dimension, like camera film in Figure 1. However there is an option to think about films as 3D objects - the dimensions cover length, width and depth as in Figure 2. In this case, we can use all of the 3D image technics for the sake of analysis.
</p>


<center>Figure 1: A typical interpretation of a film</center>
<center>source: https://dobrewsparcie.wzp.pl/</center>

<br/><br/>
<center></center>


<center>Figure 2: 3D object interpretation</center>
<center>source: tutorialandexample.com</center>

### Typical video tracking
<p style='text-align: justify;'>
Video tracking is the process of locating and labelling a moving object over frames in the film. It has a variety of uses, some of which are: human-computer interaction, security and surveillance, video communication and compressions, augmented reality, traffic control, medical imaging and video editing. Video tracking can be a time-consuming process due to the amount of data that is contained in each video. Adding further to the complexity is the possible necessity of using object recognition techniques for tracking, a challenging problem in its own right. The tracking visualization is presented in Figure 3.

The typical approach is to find an object and try to find if there is any object in a defined radius in the previous frame. If there is - this the same object and we copy its ID if not, we say this is a new object and it gets a new ID.
</p>

<center>Figure 3: Tracking cars on CCTV camera.</center>
<center>source: https://nanonets.com/blog/object-tracking-deepsort/</center>

## Metodology
### Frangi filtering

<p style='text-align: justify;'>
The Frangi filter is typically used to detect vessel-like or tube-like structures and fibres in volumetric image data. The Frangi filter is based on the eigenvalue analysis of the Hessian matrix in multiple Gaussian scales. Coarse scale structures are typically obtained by smoothing the image with a Gaussian filter. The Hessian matrix is a square matrix of second-order partial derivatives of the smoothed image.
</p>

### Foci and nuclei tracking (Deep tracking)
<p style='text-align: justify;'>
Tracking is divided into two essential parts, one is responsible for localising nuclei, while the other manages locations of foci.
</p>

<center>Figure 4: Original frame before any modifications.</center>
<br/><br/>

#### Nuclei
<p style='text-align: justify;'>
While the lion’s share of nuclei can be distinguished by the human eye with ease, in some cases the boundary between nucleus and background is vague. To define borders of nuclei simple algorithm has been presented. In the first place considered image is blurred for sake of noise removal, later Laplacian[1] is computed, allowing the user to track changes in structures throughout the whole image. Laplacian also undergoes blurring, and after that morphological operation of closing is applied. As a result, we are presented with several nuclei that are easily separable from the background.
</p>

<center>Figure 5: Result of the application of laplacian.</center>
<br/><br/>

<p style='text-align: justify;'>
The following step was defining the threshold which would allow us to consider only parts of the image which we recognised as nuclei. To achieve that, a histogram describing a relationship between the intensity of pixels and the count of pixels with that intensity was created. Since most of the pixels in every image were background pixels of lower intensity and nuclei were noticeably brighter, we decided on a cut-off point defining a threshold. It was chosen for intensity emerging after the end of the greatest peak in pixel count, where the gradient of pixel count rebound to positive values. As a final step we created two bounds restricting boundary creation for nuclei:
</p>

<p style='text-align: left;'>
1) Nuclei cannot be located on the edge of the image </p>
<p style='text-align: left;'>
2) Size limits provided by the user
</p>

<center>Figure 6: Borders of nuclei based on Laplacian threshold. Nucleus on the bottom is considered out of bounds because it overlaps with the image edge. </center>
<br/><br/>

#### Foci
<p style='text-align: justify;'>
The primary method we used for localisation of foci is based on Frangi filtering[2], while the typical usage of Frangi filter is to detect tube-like structures in images, we utilize it in videos for tracking of structures movement. While filtering marked off foci from the background in a considerable manner, we have been faced with the issue of fading and reappearing of some foci. Tracking of each focus is achieved by choosing local maximum from Frangi filtered image and checking previous frames for the presence of any foci in surroundings. In case of detection in the previous frame, the id of the preceding focus is given to the new one, otherwise earlier frames are checked in the same manner. If none of the former foci satisfied given conditions, the new focus would be given a new id.
</p>

<center>Figure 7: Foci localization based on Frangi filtration.</center>
<br/><br/>

## Results
<p style='text-align: justify;'>
In this section, we will discuss the robustness and accuracy of the proposed tracking method. We are aiming to showcase areas where code performs well, as well as discuss problems that might occur and are yet to be fixed. There are still many issues present that make the quantitative description of the code difficult to perform, which is why we focused on a qualitative description of code performance.
<br/><br/>
The desired output is as follows:
For every nucleus inside the frame, the code should find the nucleus contour and assign to it its unique number. Each luminescent spot present within the nucleus is also numbered. As a result, a list of coordinates of every numbered spot relative to the nucleus centre of mass is printed. Tracking occurs throughout the image sequence, and the list is updated every frame.
<br/><br/>
The code can find spots and nuclei that are barely visible to the human eye, but despite that, some of the visible spots are ignored.
</p>

<center> Figure 8: Spot between 83 and 111 ignored by the code</center>
<br/><br/>

<p style='text-align: justify;'>
Finding the correct contour of the nucleus requires images with as little noise as possible and well-adjusted contrast.
</p>

<center> Figure 9: Examples of correct and incorrect contour</center>
<br/><br/>

<p style='text-align: justify;'>
The number of discovered spots is usually higher than what we can observe on the image and there might be a few reasons for that.
</p>

1. The code finds spots that exist, but are hardly distinguishable from noise.

<center>Figure 10: Nucleus area with noise</center>
<br/>

2. Bigger and brighter spots are sometimes counted as multiple points, which causes the overall number of spots discovered in nucleus to be inflated.

<center>Figure 11: Multiple points discovered in a larger spot</center>
<br/>

<p style='text-align: justify;'>
Rapid changes of the image and inconsistent quality have a considerable impact on the performance of the code. They cause discontinuity of tracking and this is the main problem that occurs for this code.
Only for shorter sequences (<6 consecutive frames), even with worse image quality, the code finds and tracks luminescent spots as well as individual nuclei very well.
For longer sequences, we observe sudden changes of assigned nucleus numbers, with a certain number label ‘jumping’ from one nucleus to another. It results in inconsistent nucleus and spot numbering, incorrect spot coordinates and also a wrong number of spots assigned to the nucleus.
</p>

<center>Figure 12: Switched nucleus label</center>
<br/><br/>

<p style='text-align: justify;'>
In the case of longer films with less visible noise, the number of discovered spots is usually higher than what is visible on the image but overall the tracking works smoothly. The spots that disappear and reappear usually retain their original numbers, unless the spot was large and was counted several times. In such cases, the assigned number can get switched to a higher one.
</p>

<br/><br/>
<center>Figure 13: Spot label 9 switched to label 27 after some time</center>
<br/><br/>

<p style='text-align: justify;'>
A problem that might occur while contouring is that the code sees two or more nuclei as one entity and because of this the contours are merged. This can be often solved by adjusting the ‘radius’ parameter in the ‘track_and_show’ function.
</p>

<br/><br/>
<center>Figure 14: Merged nuclei</center>
<br/><br/>

<p style='text-align: justify;'>
Noise and artefacts present in data are unavoidable. The code needs further improvements and implementation of methods that would allow more user control over filtering and tracking results and better overall generalization. Performance tests on data sets with different specifications should be done for better assessment.
</p>

## Conclusions

### Nuclei segmentation using Laplace filtering
<p style='text-align: justify;'>
The delivered solution appears to be working rather well in cases, where the human eye can sort out the localisation of nuclei. The software will not work for every problem, assumed entry image is supposed to contain island-like structures (a brighter spot surrounded by darker background). In case of no significant differences, our method won't provide proper results. One of the primary directions for the future is an improvement of nuclei segmentation.
</p>

### Searching foci using Frangi filtering
<p style='text-align: justify;'>
Interpreting films as 3D geometrical objects appears to be the right choice for presented problem. Frangi filtering solves 2 main problems, removes noise while also provides proper segmentation. The noise problem is crucial because classical noise reduction algorithms seemed to blur the number of foci and consequently lead to omitting them in the analysis. However, it have to be kept in mind, that filtering is looking for continuous pipes in 3D and therefore it may link spots which are not directly connected.
</p>

### Deep tracking
<p style='text-align: justify;'>
The idea of the extension of the standard video tracker to make it search for points in past frames appears to be working effectively on films where targets do not disappear from the field of view but rather are obscured by other objects. To accelerate the computing speed of the tracker it is proposed to create a python class object out of it.
</p>

## Citations

[[1] Skimage Laplacian](https://scikit-image.org/docs/stable/api/skimage.filters.html?highlight=laplace#skimage.filters.laplace)

[[2] Skimage Frangi Filtering](https://scikit-image.org/docs/0.14.x/api/skimage.filters.html?highlight=frangi#skimage.filters.frangi)