PADP: progressive and adaptive data pruning for efficient incremental learning

Abstract Data pruning is a key technique for reducing training costs and improving model performance. However, most existing methods rely on fixed pruning rates or single metrics, which are largely designed for static training settings, making them unsuitable for incremental learning, where data distributions and model states change dynamically. To address this, we propose PADP, a progressive and adaptive data pruning method for incremental learning, which dynamically evaluates sample difficulty and its difficulty changes during training to enable adaptive sample selection for each incremental learning task. Specifically, we propose two metrics, the instant difficulty score and the difficulty variation score. The former evaluates the learning difficulty of a sample, while the latter evaluates the variation in difficulty over a training interval. These two metrics are combined to guide pruning decisions. To prevent certain classes from being completely removed, we also introduce a class-balance retention mechanism. Experimental results show that PADP outperforms existing data selection methods on CIFAR-100 and Tiny-ImageNet, generalizes across multiple incremental learning frameworks, and still maintains or exceeds the original accuracy even when training time is reduced by up to 52.90% compared to using the full dataset, demonstrating its effectiveness and practical value.