Q-Learning-Based Low Complexity Beam Tracking for mmWave Beamforming System

The millimeter wave (mmWave) communication systems suffer from high path loss due to strong absorption in the air. To overcome this, accurate beam steering angle information is prerequisite for beamforming technique. Further, considering mobile environments, tracking of time-varying channel direction information is required. However, conventional Kalman filter-based beam or angle tracking algorithms have disadvantage of need for time-varying channel model. Further, existing high resolution beam steering angle estimation algorithm such as auxiliary beam pair (ABP)-based algorithm has large overhead due to full beam search. Thus, efficient model-free beam tracking algorithm with low overhead is required. In this paper, we propose a low complexity beam tracking algorithm combining model-free Q-learning for practical mobile mmWave multiple-input multiple-output (MIMO) systems. Compared to existing ABP-based algorithm, the proposed algorithm requires only a few beam searches with low overhead. Also, the proposed algorithm is capable of high resolution angle estimation. Finally, the simulation result shows that the proposed beam tracking algorithm performs better than the existing algorithm without information of model.