Millimeter Wave Beamforming Training: A Reinforcement Learning Approach

Authors

  • Ehab Mahmoud Mohamed 1) Electrical Engineering Dept., College of Engineering, Prince Sattam Bin Abdulaziz University, Wadi Aldwaser 11991, Saudi Arabia. 2) Electrical Engineering Dept., Faculty of Engineering, Aswan University, Aswan 81542, Egypt. http://orcid.org/0000-0001-5443-9711

Abstract

Beamforming training (BT) is considered as an essential process to accomplish the communications in the millimeter wave (mmWave) band, i.e., 30 ~ 300 GHz. This process aims to find out the best transmit/receive antenna beams to compensate the impairments of the mmWave channel and successfully establish the mmWave link. Typically, the mmWave BT process is highly-time consuming affecting the overall throughput and energy consumption of the mmWave link establishment. In this paper, a machine learning (ML) approach, specifically reinforcement learning (RL), is utilized for enabling the mmWave BT process by modeling it as a multi-armed bandit (MAB) problem with the aim of maximizing the long-term throughput of the constructed mmWave link. Based on this formulation, MAB algorithms such as upper confidence bound (UCB), Thompson sampling (TS), epsilon-greedy (e-greedy), are utilized to address the problem and accomplish the mmWave BT process. Numerical simulations confirm the superior performance of the proposed MAB approach over the existing mmWave BT techniques.     

Author Biography

Ehab Mahmoud Mohamed, 1) Electrical Engineering Dept., College of Engineering, Prince Sattam Bin Abdulaziz University, Wadi Aldwaser 11991, Saudi Arabia. 2) Electrical Engineering Dept., Faculty of Engineering, Aswan University, Aswan 81542, Egypt.

Ehab Mahmoud Mohamed received the B.E. degree in electrical engineering from South Valley University, Egypt, in 2001, and the M.E. degree in computer science from South Valley University, Egypt in 2006, and the Ph.D. degree in information science and electrical engineering from Kyushu University, Japan in 2012. From 2013 to 2016, he has joined Osaka University, Japan as a Specially Appointed Researcher.  Since 2017, he has been an associate professor at Aswan University, Egypt. Currently, he is on leave from Aswan University and working as an associate professor at Prince Sattam bin Abdulaziz University, Saudi Arabi. He is the general chair of IEEE ITEMS’16 and IEEE ISWC’ 18. He is a technical committee member in many international conferences and a reviewer in many international conferences, journals and transactions. His current research interests are 5G networks, cognitive radio networks, millimeter wave transmissions, Li-Fi technology, MIMO systems, and underwater communication. He is an IEEE member.

References

T. S. Rappaport, et al., “Wireless Communications and Applications Above 100 GHz: Opportunities and Challenges for 6G and Beyond” IEEE ACCESS, vol. 7, pp. 78729 – 78757, 2019.

E. M. Mohamed, M. A. Abdelghany, and M Zareei “An Efficient Paradigm for Multiband WiGig D2D Networks,” IEEE ACCESS, vol. 7, pp. 70032-70045, 2019.

E. M. Mohamed, et al., “Relay Probing for Millimeter Wave Multi-Hop D2D Networks,” IEEE ACEESS, vol. 8, pp. 30560 – 30574, 2020.

T. Bai, R. Vaze, and R. W. Heath, Jr., “Analysis of Blockage Effects on Urban Cellular Networks,” IEEE Transc. On Wirele. Commun., vol. 13, no. 9, pp. 5070-5083, 2014.

A. Abdelreheem, E. M. Mohamed, and H. Esmaiel, “Location-based millimeter wave multi-level beamforming using compressive sensing,” IEEE Commun. Lett., vol. 22, pp. 185-188, 2018.

E. M. Mohamed, K. Sakaguchi, and S. Sampei, “Wi-Fi coordinated WiGig concurrent transmissions in random access scenarios,” IEEE Trans. on Vehi. Techn., vol. 66, no. 11, pp. 10357-10371, 2017.

I. Ahmed, et al., “A survey on hybrid beamforming techniques in 5g: Architecture and system model perspectives,” IEEE Communic. Surv. Tut., vol. 20, no. 4, pp. 3060–3097, Fourthquarter 2018.

IEEE 802.11ad Standard, “Enhancements for very high throughput in the 60 GHz band,” ed, Dec. 2012.

C. Jiang, et al, “Machine Learning Paradigms for Next-Generation Wireless Networks” IEEE Wireless Communications, vol. 24, no.2, pp. 98 – 105, 2017.

B. Li, Z. Zhou, H. Zhang, and A. Nallanathan, “Efficient beamforming training for 60-GHz millimeter-wave communications: A novel numerical optimization framework,” IEEE Trans. Veh. Technol., vol. 63, no. 2, pp. 703–717, 2014.

Gao, X., Dai, L., Yuen, C., Wang, Z.: ‘Turbo-like beamforming based on tabu search algorithm for millimeter-wave massive MIMO systems’, IEEE Trans. On Vehi. Techn., vol. 65, no.7, pp. 5731–5737, 2016.

A. Abdelreheem, E. M. Mohamed, H. Esmaiel, “Adaptive location-based millimetre wave beamforming using compressive sensing based channel estimation,” IET Communications, vol. 13, no. 9, pp. 1287-1296, 2019.

A. M. Nor and E. M. Mohamed, “Li-Fi Positioning for Efficient Millimeter Wave Beamforming Training in Indoor Environment,” Mobile Networks and Applications, vol. 24, no.2, pp. 517-531, 2019.

A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter wave cellular systems,” IEEE Journal of Selec.Topics in Signal Process., vol. 8, pp. 831-846, 2014.

M. B. Booth, V. Suresh, N. Michelusi, and D. J. Love, “Multi-Armed Bandit Beam Alignment and Tracking for Mobile Millimeter Wave Communications,” IEEE Commun. Lett., vol. 23, no. 7, pp.1244-1248, 2019.

I. Chafaa, E. V. Belmega, and M. Debbah, “Adversarial multi-armed bandit for mmWave beam alignment with one-bit feedback,” in Proc. 12th EAI Int. Conf. Perform. Eval. Methodol. Tools, pp. 23–30, 2019.

V. Va, T. Shimizu, G. Bansal, and R. W. Heath, Jr., “Online learning for position-aided millimeter wave beam training,” IEEE Access, vol. 7, pp. 30507–30526, 2019.

IEEE 802.15.3c Part 15.3: ‘Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications for High Rat Wireless Personal Area Neworks (WPANS) Amendment’, 2009.

Wang, J., Lan, Z., Sum, C., et al ‘Beamforming codebook design and performance evaluation for 60GHz wideband WPANs’. Proc. IEEE Vehi. Techn., Anchorage, 2009, pp. 1-6.

F. Wilhelmi, “Collaborative spatial reuse in wireless networks via selfish multi-armed bandits,” in Ad Hoc Networks, vol. 88, pp. 129-141, 2019.

Downloads

Published

2024-04-19

Issue

Section

Wireless and Mobile Communications