Abstract: Recent advancements in scaling pre-trained language models have led to significant performance improvements across various tasks. Yet, adapting these large models poses substantial ...
In this paper, we propose a conceptually simple but very effective attention module for Convolutional Neural Networks (ConvNets). In contrast to existing channel-wise and spatial-wise attention ...