Partial Large Kernel CNNs for Efficient Super-Resolution

Recently, in the image super-resolution (SR) domain, Transformers have outperformed Convolution Neural Networks (CNNs) with reduced computational complexity and parameters by modeling long-range dependencies input-dependently. However, these works often overlook the memory-access cost issue of the SR Transformers, which deals with larger feature maps without any downsampling stages. In this paper, we demonstrate that SR CNNs are significantly more memory-efficient than Transformers, reducing considerable peak memory usage and latency. Building on this efficiency, to narrow the performance gap between CNNs and Transformers, we integrate key mechanisms of Transformers into CNNs via partial large-kernel convolutions combined with element-wise attention. As a result, we present the memory-efficient SR CNNs, termed Partial Large Kernel CNNs for Efficient Super-Resolution, achieving state-of-the-art results on four benchmark datasets at <inline-formula> <tex-math notation="LaTeX">$\times 4$ </tex-math></inline-formula> scale, while reducing latency by 68.1% and maximum GPU memory occupancy by 80.2% compared to SRFormer-light.