Towards NextG Receiver: Online Real-Time Machine Learning with Domain Knowledge for Wireless Communications

Next-generation (NextG) cellular networks are envisioned to integrate artificial intelligence (AI) and machine learning (ML) into the air interface to meet increasingly stringent performance demands. Multiple-input multiple-output (MIMO) and its variants, such as massive MIMO, have been key enablers across successive generations of cellular networks with evolving design challenges and complexities. However, developing AI/ML-based solutions for MIMO operations in NextG systems is challenging due to the large number of possible system configurations, dynamic channel environments, and real-time operation adaptations. The highly dynamic nature of wireless environments and the operation adaptations necessitate AI/ML solutions to adapt to rapid channel variations and operation changes on a sub-millisecond basis. To this end, this dissertation develops various online and real-time AI/ML-based methods for the receive processing task in the NextG air interface with a focus on the MIMO orthogonal frequency-division multiplexing (OFDM) symbol detection, orthogonal time frequency space (OTFS) symbol detection, and MIMO-OFDM channel estimation task. To enable efficient learning, domain knowledge, such as the symmetric structure of the modulation constellation, the delay-Doppler (DD) domain input-output relationship, and the channel statistics, is inherently embedded in the design of the neural network. All introduced algorithms achieve outstanding performance while learning from only a limited number of over-the-air (OTA) training pilots on a 5G slot basis. This dissertation highlights the critical role of integrating AI/ML with domain knowledge in cellular communication systems, paving the way for the deployment of AI/ML-based techniques in NextG networks.

Name thesis:degree_name

Doctor of Philosophy

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Computer Engineering

Department dc:contributor.department

Electrical and Computer Engineering

Grantor dc:publisher

Virginia Tech

Year dc:date.issued

2025

Author dc:creator

• Xu, Jiarui

Chair dc:contributor.committeechair

• Liu, Lingjia

Committee members dc:contributor.committeemember

• Zheng, Lizhong
• Yi, Yang
• Reed, Jeffrey H.
• Eldardiry, Hoda Mohamed
• Abbott, Amos L.

• Machine Learning
• Online Learning
• Symbol Detection
• Channel Estimation
• Receive Processing
• MIMO-OFDM
• OTFS
• Channel Equalization
• Neural Network

Statement dc:rights

• In Copyright

Licence dc:rights.uri

• http://rightsstatements.org/vocab/InC/1.0/

Language dc:language.iso

en

Dc Identifier Other

vt_gsexam:43366

OAI identifier oai:identifier

oai:vtechworks.lib.vt.edu:10919/134944