A Hybrid Intersection Filtering and Recursive Feature Elimination Technique for Efficient Feature Reduction in High Dimensional Datasets

High-dimensional datasets are commonly encountered in real-world machine learning applications and often degrade classification performance due to redundant and irrelevant features. In addition, the presence of excessive features increases computational complexity and processing time. Feature selection is therefore a crucial preprocessing step to improve model accuracy and efficiency. This study proposes a hybrid feature selection approach called Intersection Filtering based on Recursive Feature Elimination with Cross-Validation (IF-RFECV), which integrates wrapper-based and filter-based strategies to obtain a stable and optimal subset of features. The proposed method first applies Recursive Feature Elimination with Cross-Validation (RFECV) using multiple classification models to rank and select relevant features. Subsequently, an intersection filtering mechanism is employed to identify features that are consistently selected across different RFECV-based models, thereby reducing model-dependent bias and improving feature robustness. The effectiveness of IF-RFECV is evaluated using four benchmark datasets with varying dimensionality obtained from the KEEL and UCI repositories. Several classification algorithms, including Gradient Boosting, K-Nearest Neighbor, Naïve Bayes, Decision Tree, Random Forest, and Support Vector Machine, are used to assess model performance. Experimental results demonstrate that IF-RFECV produces a more compact feature subset compared to conventional RFECV while achieving superior performance in terms of accuracy, precision, recall, and F1-score on most datasets, particularly those with higher dimensionality. Although IF-RFECV requires slightly higher computational time due to its two-stage process, the performance gains and improved generalization justify this trade-off. These findings indicate that IF-RFECV is an effective and robust feature selection technique for high-dimensional classification problems.