Mathematical Analysis and Improvement of the Maximum Spatial Eigenfilter for Direction of Arrival Estimation

Abstract Maximum spatial eigenfiltering improves the accuracy of maximum likelihood direction-of-arrival estimators for closely-spaced signal sources but may interchangeably attenuate widely-spaced signal sources, producing a severe performance degradation. Although this behavior has been observed experimentally, it still lacks a mathematical explanation. In our previous work, we overcame these limitations using a differential spectrum-based spatial filter but this still caused a small degradation in the DOA estimate. In this paper, we develop a mathematical analysis of how the signal source separation and the Karhunen-Loève expansion affect the passbands of the maximum spatial eigenfilter. The farther the sources, the less significant is the maximum eigenvalue of the spatial correlation matrix and its corresponding eigenvector. Then, the magnitude response of the maximum spatial eigenfilter no longer approximates the spatial power spectrum and is not guaranteed to place multiple passbands around the signal sources. Consequently, we propose a spatial filter built from the eigenvectors of the entire signal subspace. This filter showed an overall runtime smaller than that of our previous work. It also provides a significant reduction in the threshold signal-to-noise ratio for closely-spaced signal sources and does not hamper the estimation for widely-spaced signal sources.