<div align="center">
<img src="resources/images/deepsense_vision_icon.png"><br/><h1>Deepsense Vision</h1>
</div>

<div align="center">
<h4>State-of-the-art Computer Vision for PyTorch</h4>
</div>

<div align="center">
<a href="#TODO"><img src="https://img.shields.io/badge/Python-3.7%20%7C%203.8%20%7C%203.9-blue"></a>
<a href="https://opensource.org/licenses/Apache-2.0"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg"></a>
<a href="http://deepsense-vision.deepsense.ai/"><img src="https://img.shields.io/badge/docs-online-green"></a>
</div>

<div align="center">
<a href="#getting-started"><b>Getting Started</b></a> |
<a href="#learn-more"><b>Learn More</b></a> |
<a href="#available-models"><b>Available Models</b></a> |
<a href="#installation"><b>Installation</b></a>
</div>
<br/>

<div align="center">
<img src="resources/images/models.gif" width="500" />
</div>

* Deepsense Vision (DS Vision) provides implementations as well as wrappers for state-of-the-art models for various computer vision tasks:
* object detection
* instance segmentation
* semantic segmentation
* keypoints detection
* The library aims for simplicity for both users and developers - having common API and a minimum level of abstraction, each model is a
fully standalone module and can be copy-pasted into your project
* DS Vision delivers pretrained models that can be integrated into your project with a few lines of code
* The library has support for experiment tracking tool [Neptune](https://neptune.ai/product/experiment-tracking)


## Getting Started

### Quick tour

##### Training
In order to start model training, you need to do the following:
* specify network architecture (e.g. Mask R-CNN)
* overwrite default network & training parameters (e.g. learning rate, batch size, etc.)
* provide train & validation datasets

from deepsense_vision.datasets.deepsense_dataset import DeepsenseDataset
from deepsense_vision.models.maskrcnn.maskrcnn_model import MaskRCNNModel

# create model
maskrcnn_model = MaskRCNNModel()
maskrcnn_model.to_gpu()

# create model config
maskrcnn_config = maskrcnn_model.parse_config(
    # see default config:
    # deepsense_vision/models/maskrcnn/maskrcnn_config.py
    # values below override defaults
    {
        "classnames": ["100USD", "1cent"],
        "experiment_dirpath": experiment_dirpath,
        "num_train_iters": 200,
        "valid_freq_iters": 100,
        "verbose_freq_iters": 10,
        "lr_step_size": 100
    }
)

train_dataset = DeepsenseDataset(
    metadata_path=metadata_path,
    classnames=maskrcnn_config.classnames,
    mask_level=maskrcnn_config.mask_level  # "instance"
)

valid_dataset = DeepsenseDataset(
    metadata_path=metadata_path,
    classnames=maskrcnn_config.classnames,
    mask_level=maskrcnn_config.mask_level
)

maskrcnn_model.train(
        train_dataset=train_dataset,
        valid_dataset=valid_dataset,
        config=maskrcnn_config
    )

##### Inference

from deepsense_vision.models.maskrcnn.maskrcnn_model import MaskRCNNModel

maskrcnn_model = MaskRCNNModel()
maskrcnn_model.load_checkpoint(checkpoint_filepath)
maskrcnn_model.to_gpu()

prediction = maskrcnn_model.predict_from_filepath(image_filepath)  # instance of deepsense_vision.components.prediction.Prediction

### Learn More
<div align="center">

| Section | Description |
| ------------------------- | ------------------------------ |
| [Documentation](http://deepsense-vision.deepsense.ai/) | Full API documentation
| [Creating Dataset](deepsense_vision/datasets/README.md) | Guide on supported data formats & tutorials to create custom datasets
| [Object Detection](examples/train_inference/fastercnn_train_inference.ipynb) | Train & inference example using Faster R-CNN
| [Instance Segmentation](examples/train_inference/maskrcnn_train_inference.ipynb) | Train & inference example using Mask R-CNN
| [Semantic Segmentation](examples/train_inference/patchnet_train_inference.ipynb) | Train & inference example using PatchNet
| [Keypoint Detection](examples/train_inference/keypointrcnn_train_inference.ipynb) | Train & inference example using Keypoint R-CNN
| [Model Zoo](examples/zoo) | Pretrained models for easy inference

</div>

## Available Models

1. [PatchNet](deepsense_vision/models/patchnet) - a generalized version of U-Net developed by [deepsense.ai](https://www.deepsense.ai)
2. [Faster R-CNN](deepsense_vision/models/fasterrcnn) released with the paper [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://arxiv.org/abs/1506.01497)
3. [Mask R-CNN](deepsense_vision/models/maskrcnn) released with the paper [Mask R-CNN](https://arxiv.org/abs/1703.06870)
4. [YOLOv3](deepsense_vision/models/yolov3) released with the paper [YOLOv3: An Incremental Improvement](https://arxiv.org/abs/1804.02767)
5. [Keypoint R-CNN](deepsense_vision/models/keypointrcnn) - an extension to Mask R-CNN for detecting keypoints

## Installation

1\. Create new virtual environment with Python 3.7, 3.8 or 3.9:

$ python3.7 -m venv venv37DV
# or
$ virtalenv -p python3.7 venv37DV

2\. Run virtual environment:

$ source venv37DV/bin/activate

3\. Upgrade `pip` and `setuptools`:

(venv37DV) $ pip install -U pip setuptools

4\. Install `deepsense_vision`:

You have multiple options:

- install specific version, e.g. `v0.6.0`:

(venv37DV) $ #TODO

- install current state of the master branch:

(venv37DV) $ #TODO

## License

Deepsense Vision is released under the [Apache 2.0 license](LICENSE).

## Contributing

We appreciate all contributions to improve Deepsense Vision. Please refer to [CONTRIBUTING.md](CONTRIBUTING.md) for the contributing guidelines.

## Citation
If you use Deepsense Vision in your research please use the following BibTeX entry.

@misc{deepsense_vision,
  author =       {Deepsense.ai},
  title =        {Deepsense Vision},
  howpublished = {\url{}}, #TODO add link of github repo here
  year =         {2021}
}

Copyright (c) 2021 [deepsense.ai](https://www.deepsense.ai)