Category-Level Adversarial Adaptation for Semantic Segmentation Using Purified Features

Abstract

We target the problem named unsupervised domain adaptive semantic segmentation. A key in this campaign consists in reducing the domain shift, so that a classifier based on labeled data from one domain can generalize well to other domains. With the advancement of adversarial learning method, recent works prefer the strategy of aligning the marginal distribution in the feature spaces for minimizing the domain discrepancy. However, based on the observance in experiments, only focusing on aligning global marginal distribution but ignoring the local joint distribution alignment fails to be the optimal choice. Other than that, the noisy factors existing in the feature spaces, which are not relevant to the target task, entangle with the domain invariant factors improperly and make the domain distribution alignment more difficult. To address those problems, we introduce two new modules, Significance-aware Information Bottleneck (SIB) and Category-level alignment (CLA), to construct a purified embedding-based category-level adversarial network. As the name suggests, our designed network, CLAN, can not only disentangle the noisy factors and suppress their influences for target tasks but also utilize those purified features to conduct a more delicate level domain calibration, i.e., global marginal distribution and local joint distribution alignment simultaneously. In three domain adaptation tasks, i.e., GTA5 → Cityscapes, SYNTHIA → Cityscapes and Cross Season, we validate that our proposed method matches the state of the art in segmentation accuracy. Show more