Author: Akshay Kulkarni Note: WACV '21 paper; Official Code Release Brief Outline They propose Domain Adaptation via Cross-domain Mixed Sampling which mixes images from two domains along with their corresponding labels. These mixed samples are trained on, along with the labelled data itself. Introduction Unsupervised Domain Adaptation (UDA) deals with the problem where labelled data is available for a source domain, but only unlabelled data is available for the target domain.

Notes Author: Akshay Kulkarni CVPR '21 paper; Code Release SOTA on sem. seg. UDA benchmarks (as of 10/04/21). However, their code shows use of DeepLabv3+ type of decoder instead of DeepLabv2 (as claimed in the paper). Thus, it is unclear whether performance improvements are architecture-dependent or not. Brief Outline They use representative prototypes (class feature centroids) to address 2 issues in self-training for UDA in semantic segmentation. Exploit the feature distances from prototypes to estimate likelihood of pseudo-labels to facilitate online correction during training.

Notes author: Akshay Kulkarni AAAI '21 paper; Code Release Brief Outline They propose a bidirectional style-induced DA method (BiSIDA) that employs consistency regularization to efficiently exploit information from the unlabeled target dataset using a simple neural style transfer model. Introduction To perform domain alignment on a pixel-level or feature-level, existing works (Tsai et al. CVPR '18, Hoffman et al. ICML '18) typically use adversarial training and training with the aligned data is supervised using the labeled source data. However, this introduces extra complexity and instability in training.

NeurIPS '20 paper; Code not released as of 30/11/20. Author: Akshay Kulkarni Brief Outline This work investigates open compound domain adaptation (OCDA) for semantic segmentation which deals with mixed and novel situations at the same time. They first cluster the compound target data based on style (discover), then hallucinate multiple latent target domains in source using image translation, and perform target-to-source alignment separately between domains (adapt). Introduction Most existing UDA techniques focus on a single-source single-target setting instead of a more practical scenario where target consists of multiple data distributions without clear distinctions.

BMVC '18 paper; no official code release Author: Akshay Kulkarni Brief Outline This work proposes an active learning strategy for semantic segmentation by using an information measure combined with an estimate of human annotation effort (inferred from a learned cost model). exploiting the spatial coherency of an image.

ECCV '20 paper; Project Page; Code not released as of 20/09/20. Author: Akshay Kulkarni Brief Outline This paper proposes a framework for domain adaptation (DA) in semantic segmentation with image-level weak labels in the target domain. They use weak labels to enable the interplay between feature alignment and pseudo-labeling, improving both in DA. Introduction Existing UDA methods for semantic segmentation are developed mainly using 2 mechanisms

CVPR '20 paper; Official Code Release Author: Akshay Kulkarni Brief Outline This paper proposes cross-consistency training, where an invariance of the predictions is enforced over different perturbations applied to the outputs of the encoder (in a shared encoder and multiple decoder architecture). Introduction Semi-Supervised Learning (SSL) takes advantage of a large amount of unlabeled data and limits the need for labeled examples. The current dominant SSL methods in deep learning are

Author: Akshay Kulkarni Brief Outline They propose to blend active learning (Gissin and Shalev-Shwartz, 2019) and image mixup (Zhang et. al. 2017) to tackle data-efficient knowledge distillation from a blackbox teacher model. Introduction Synthesize a big pool of images from few training examples by mixup and then use active learning to select the most helpful subset from the pool to query the teacher model. Treat the teacher model's outputs as ground truth labels and train the student network using them. Image mixup (Zhang et. al. 2017, Guo et. al. 2019, Berthelot et. al. 2019) synthesizes a virtual image by a convex combination of two training images. While the resultant image maybe cluttered or semantically meaningless, it resides near the manifold of the natural images.

Author: Akshay Kulkarni Note: CVPR '20 paper, Official Code Release Brief Outline This paper proposes a method to adapt to the target domain's texture. They diversify the texture of source images using a style transfer algorithm, which prevents the model from overfitting to a specific texture. Then, they finetune the model with self-training to get direct supervision of the target texture.

Author: Akshay Kulkarni Note: Code for Tsai et. al. 2018 mentions some implementation details for this method. There is no open-source implementation for this method available (as of 20/04/2020). Brief Outline They propose an unsupervised domain adaptation that explicitly discovers many modes in the structured output space of semantic segmentation to learn a better discriminator between the 2 domains, ultimately leading to a better domain alignment. Introduction They leverage pixel-level semantic annotations available in the source domain, but instead of working on the output space (like Tsai et. al. 2018), the adaptation occurs in 2 stages.

Author: Akshay Kulkarni Note: CVPR '20 Oral, Code Brief Outline They propose a two-step self-supervised DA approach to minimize the inter-domain and intra-domain gap together. They conduct inter-domain adaptation and from this, they separate the target domain into an easy and hard split using an entropy-based ranking function. For the intra-domain adaptation, they propose a self-supervised adaptation technique from the easy to the hard split.

Author: Akshay Kulkarni Note: The paper was published at NIPS'16 (NeurIPS since 2018). Brief Outline A domain adaptation approach that can jointly learn adaptive classifiers and transferable features from labeled data in the source domain and unlabeled data in the target domain. Introduction Domain Adaptation (DA) is machine learning under the shift between training and test distributions.

Author: Akshay Kulkarni Brief Outline A simple method for UDA where the discrepancy between the source and target distributions is reduced by swapping the low-frequency spectrum of one with the other. The method is illustrated for semantic segmentation. Introduction Unsupervised domain adaptation (UDA) refers to adapting a model trained with annotated samples from one distribution (source), to operate on a different (target) distribution for which no annotations are given. Simply training the model on the source data does not yield satisfactory performance on the target data, due to the covariate shift. In some cases, perceptually insignificant changes in the low-level statistics can cause significant deterioration of the performance of the trained model, unless UDA is performed.

Author: Akshay Kulkarni Brief Outline They introduce 2 criteria to regularize the optimization involved in Unsupervised Domain Adaptation (UDA). The first criterion, inspired by visual psychophysics, is that the map between 2 image domains be phase-preserving. The second criterion aims to leverage ecological statistics (or regularities in the scene), regardless of the illuminant or imaging sensor. Introduction

Author: Akshay Kulkarni Brief Outline The pre-trained teacher network is considered as a fixed discriminator and a generator generates training samples which can obtain maximum response from the discriminator. Simultaneously, a smaller network is trained using the generated data and the teacher network. Introduction Most existing network compression and speed-up algorithms have a strong assumption that the original training data is available. However, training data may be unavailable due to privacy and transmission limitations.

Author: Akshay Kulkarni Note: Actual title of paper is "A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning" and it was published as a CVPR '17 paper (pdf link). Brief Outline The distilled knowledge to be transferred is defined in terms of flow between layers, and is calculated as the inner product between feature maps from 2 layers. This training is followed by training the student for the original task. Introduction This is effectively the third paper on Knowledge Distillation after Hinton et. al. 2015 and Romero et. al. 2015.

Author Akshay Kulkarni Brief Outline The paper aims to provide an intuitive understanding of significant DL-based approaches to segmentation. Subtasks of Image Segmentation: Semantic Segmentation: Each pixel is classified into one of the predefined set of classes such that pixels belonging to the same class belongs to a unique semantic entity in the image. Note that the semantics (logic) in question depends not only on the data but also the problem being addressed. Saliency Detection: Focus on the most important object in a scene. Instance Segmentation: Segments multiple instances of the same object in a scene.

Author: Akshay Kulkarni Note: Actual title of paper is "Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer" and it was published as a ICLR '17 poster. Brief Outline By properly defining attention for CNNs, it can be used to significantly improve the performance of a student CNN network by forcing it to mimic the attention maps of a powerful teacher network. Introduction Artificial attention lets a system “attend” to an object to examine it with greater detail.

Author: Akshay Kulkarni Note: For proper understanding, the knowledge of Variational AutoEncoders (VAE) is highly recommended. This post provides a very good and in-depth explanation. Brief Outline Active learning algorithms attempt to incrementally select samples for annotation that result in high classification performance with low labelling cost. This paper introduces a pool-based active learning strategy which learns a low dimensional latent space from labeled and unlabeled data using a VAE. Introduction

Introduction This is a review paper that focuses on the theoretical aspects of Generative Adversarial Networks (GANs). It also identifies questions (issues) in GANs and summarizes approaches to these issues. Primary GAN Setup Notation Useful Definitions KL Divergence Given two probability distributions $p_d(x), p_g(x)$ with positive support $\forall x$. The KL divergence is given by