Unveil Fundamental Graph Properties for Neural Architecture Search

ABSTRACT Deep learning profoundly impacts various areas, such as face recognition and language translation. Owing to the increasingly high computational costs of training neural architectures, it is intractable to manually examine the performance of various neural architectures, promoting the area of Neural architecture search (NAS) that enables artificial intelligence (AI) automation. Despite the advance of NAS in automatically discovering the optimal neural networks, the fundamental understanding of the structure of neural architectures is limited. To fill the gap, we propose a method dubbed NASGraph that converts neural architectures to graphs whose graph properties determine their performances, enabling us to search for or design strong architectures. On various standard NAS benchmarks, NASGraph outperforms baseline NAS methods and requires remarkably fewer computation resources. The numerical results illuminate the relationship between neural architecture and its performance, complementing other approaches. Therefore, combining our approach with these approaches leads to performance improvement, but not combining the others. Our study offers a new perspective on network science for AI, potentially advancing various aspects of machine learning and uncovering the black box of convolutional neural networks.