Millimeter Wave Beamforming Training: A Reinforcement Learning Approach
Abstract
Beamforming training (BT) is considered as an essential process to accomplish the communications in the millimeter wave (mmWave) band, i.e., 30 ~ 300 GHz. This process aims to find out the best transmit/receive antenna beams to compensate the impairments of the mmWave channel and successfully establish the mmWave link. Typically, the mmWave BT process is highly-time consuming affecting the overall throughput and energy consumption of the mmWave link establishment. In this paper, a machine learning (ML) approach, specifically reinforcement learning (RL), is utilized for enabling the mmWave BT process by modeling it as a multi-armed bandit (MAB) problem with the aim of maximizing the long-term throughput of the constructed mmWave link. Based on this formulation, MAB algorithms such as upper confidence bound (UCB), Thompson sampling (TS), epsilon-greedy (e-greedy), are utilized to address the problem and accomplish the mmWave BT process. Numerical simulations confirm the superior performance of the proposed MAB approach over the existing mmWave BT techniques.
Full Text:
PDFReferences
T. S. Rappaport, et al., “Wireless Communications and Applications Above 100 GHz: Opportunities and Challenges for 6G and Beyond” IEEE ACCESS, vol. 7, pp. 78729 – 78757, 2019.
E. M. Mohamed, M. A. Abdelghany, and M Zareei “An Efficient Paradigm for Multiband WiGig D2D Networks,” IEEE ACCESS, vol. 7, pp. 70032-70045, 2019.
E. M. Mohamed, et al., “Relay Probing for Millimeter Wave Multi-Hop D2D Networks,” IEEE ACEESS, vol. 8, pp. 30560 – 30574, 2020.
T. Bai, R. Vaze, and R. W. Heath, Jr., “Analysis of Blockage Effects on Urban Cellular Networks,” IEEE Transc. On Wirele. Commun., vol. 13, no. 9, pp. 5070-5083, 2014.
A. Abdelreheem, E. M. Mohamed, and H. Esmaiel, “Location-based millimeter wave multi-level beamforming using compressive sensing,” IEEE Commun. Lett., vol. 22, pp. 185-188, 2018.
E. M. Mohamed, K. Sakaguchi, and S. Sampei, “Wi-Fi coordinated WiGig concurrent transmissions in random access scenarios,” IEEE Trans. on Vehi. Techn., vol. 66, no. 11, pp. 10357-10371, 2017.
I. Ahmed, et al., “A survey on hybrid beamforming techniques in 5g: Architecture and system model perspectives,” IEEE Communic. Surv. Tut., vol. 20, no. 4, pp. 3060–3097, Fourthquarter 2018.
IEEE 802.11ad Standard, “Enhancements for very high throughput in the 60 GHz band,” ed, Dec. 2012.
C. Jiang, et al, “Machine Learning Paradigms for Next-Generation Wireless Networks” IEEE Wireless Communications, vol. 24, no.2, pp. 98 – 105, 2017.
B. Li, Z. Zhou, H. Zhang, and A. Nallanathan, “Efficient beamforming training for 60-GHz millimeter-wave communications: A novel numerical optimization framework,” IEEE Trans. Veh. Technol., vol. 63, no. 2, pp. 703–717, 2014.
Gao, X., Dai, L., Yuen, C., Wang, Z.: ‘Turbo-like beamforming based on tabu search algorithm for millimeter-wave massive MIMO systems’, IEEE Trans. On Vehi. Techn., vol. 65, no.7, pp. 5731–5737, 2016.
A. Abdelreheem, E. M. Mohamed, H. Esmaiel, “Adaptive location-based millimetre wave beamforming using compressive sensing based channel estimation,” IET Communications, vol. 13, no. 9, pp. 1287-1296, 2019.
A. M. Nor and E. M. Mohamed, “Li-Fi Positioning for Efficient Millimeter Wave Beamforming Training in Indoor Environment,” Mobile Networks and Applications, vol. 24, no.2, pp. 517-531, 2019.
A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter wave cellular systems,” IEEE Journal of Selec.Topics in Signal Process., vol. 8, pp. 831-846, 2014.
M. B. Booth, V. Suresh, N. Michelusi, and D. J. Love, “Multi-Armed Bandit Beam Alignment and Tracking for Mobile Millimeter Wave Communications,” IEEE Commun. Lett., vol. 23, no. 7, pp.1244-1248, 2019.
I. Chafaa, E. V. Belmega, and M. Debbah, “Adversarial multi-armed bandit for mmWave beam alignment with one-bit feedback,” in Proc. 12th EAI Int. Conf. Perform. Eval. Methodol. Tools, pp. 23–30, 2019.
V. Va, T. Shimizu, G. Bansal, and R. W. Heath, Jr., “Online learning for position-aided millimeter wave beam training,” IEEE Access, vol. 7, pp. 30507–30526, 2019.
IEEE 802.15.3c Part 15.3: ‘Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for High Rat Wireless Personal Area Neworks (WPANS) Amendment’, 2009.
Wang, J., Lan, Z., Sum, C., et al ‘Beamforming codebook design and performance evaluation for 60GHz wideband WPANs’. Proc. IEEE Vehi. Techn., Anchorage, 2009, pp. 1-6.
F. Wilhelmi, “Collaborative spatial reuse in wireless networks via selfish multi-armed bandits,” in Ad Hoc Networks, vol. 88, pp. 129-141, 2019.
Refbacks
- There are currently no refbacks.
International Journal of Electronics and Telecommunications
is a periodical of Electronics and Telecommunications Committee
of Polish Academy of Sciences
eISSN: 2300-1933
