SpA-Former:An Effective and lightweight Transformer for image shadow removal

Abstract

In this paper, we propose an Effective and lightweight Transformer for image shadow detection and removal named SpA-Former to recover a shadow-free image from a single shaded image. In contrast to conventional methods that require two stages for shadow detection and then shadow removal, the SpA-Former is a one-stage network capable of learning the mapping function between shadows and no shadows, and does not require a separate shadow detection. SpA-Former is composed of Transformer encoder and CNN decoder, where the CNN decoder contains the GAN network. In the Transformer encoding stage, Gated Feed-Forward Network(GFFN) is devised to control the information flow. In the CNN decoding stage, Two-wheel RNN joint spatial attention(TWRNN) and Fourier transform residual block (FTR) are designed to achieve satisfactory results in shadow removal. The combination of Transformer and CNN is able to feed global features from the Vision Transformer encoder into CNN to enhance the global perception of CNN branches, taking into account the complementarity of local features and the global. The SpA-Former's inference speed is 0.0459s, and the final Parameters and FLOPS are only 0.47MB and 15G, achieving the current lightweight of image shadow removal. The source code of MemoryNet can be obtained from https://github.com/zhangbaijin/SpA-Former-shadow-removal Show more