Hybrid Quantum-Classical Transfer Learning for Parameter-Efficient Code Generation

The remarkable success of large language models (LLMs) in code generation has transformed software development, with systems like Codex, CodeGen, and StarCoder demonstrating strong capabilities in synthesizing functional code from natural language descriptions. However, this progress comes at a steep computational cost. Modern code generation models require hundreds of millions to billions of parameters&#x2014;GPT-3 contains 175 billion parameters, CodeGen-16B uses 16 billion, and the trend continues toward ever-larger models. This scale creates fundamental challenges in training cost, inference latency, memory consumption, and environmental impact. We present a quantum transfer learning framework as an alternative approach to classical model compression for code generation. By replacing selected MLP layers with variational quantum circuits (VQCs), we exploit the exponential representational capacity of quantum systems to achieve <inline-formula> <tex-math notation="LaTeX">$\Theta (D \log D)$ </tex-math></inline-formula> parameter scaling instead of classical <inline-formula> <tex-math notation="LaTeX">$\Theta (D^{2})$ </tex-math></inline-formula>. Our hybrid classical-quantum architecture balances three competing objectives: leveraging existing pre-trained knowledge, achieving parameter compression through quantum representation, and maintaining compatibility with near-term quantum hardware constraints. We demonstrate our approach on CodeGen-350M, a production-scale code generation model, adopting a bottleneck architecture that substantially reduces parameters compared to the original MLP layer. Through a fair comparison against a classical bottleneck baseline with near-identical parameter count, we show that the VQC achieves superior representational quality&#x2014;lower perplexity and higher code generation accuracy&#x2014;validating the quantum layer&#x2019;s expressivity advantage in the compressed latent space. Our design employs data re-uploading, where classical input features are re-encoded at each variational layer via trainable RY gates, enhancing the circuit&#x2019;s expressive power on near-term devices.