ISA-Based GEMM Acceleration: Maximizing Computational Intensity for Scalable Performance

The growing demand for intensive artificial intelligence applications, such as deep neural networks and Large Language Models (LLMs), has driven the creation of architecture extensions and accelerators for matrix multiplication, a fundamental operation in machine learning. Several modern Instruction Set Architectures (ISAs) have introduced matrix extensions, each employing distinct approaches to data storage and execution. This work examines the scalability of computational intensity (CI) across different matrix data layout models and ISA-level processing, focusing on proposals centered on the RISC-V architecture. We create an analytical model that associates CI with the submatrix format and the available storage budget. Our results reveal that the outer product model achieves CI similar to that of architectures that perform full-block multiplication, but with much lower storage costs. We also demonstrate an upper bound for the computational intensity of outer product approaches under storage constraints. Finally, we suggest a technique that reuses existing RISC-V vector registers for matrix computation, organizing the data into two-dimensional grids to optimize CI. The proposed approach achieves up to 99.6% of the upper bound with 256-bit registers and 99.2% with 2048-bit registers. We conclude that it is possible to achieve high efficiency without increasing the architectural area, allowing scalability and compatibility with already standardized vector extensions.