Jingwei Sun, PhD, Assistant Professor in Computer & Information Science & Engineering, is the PI of a new NVIDIA Academic Grant Award, alongside collaborator My T. Thai, PhD. Their project, titled “Personalized BP-free Federated Learning System for LLMs on the Edge,” aims to revolutionize how large language models (LLMs) learn and adapt on real-world edge devices.
The NVIDIA Academic Grant Program is an initiative that advances academic research by providing access to powerful computing resources. Dr. Sun’s team has been sponsored with GPU time and edge devices, enabling rigorous development and testing.
The core of this research focuses on the challenge of training LLMs efficiently and privately on user devices. As LLMs make their way into everyday applications such as Apple Intelligence, these models learn and improve through user interactions. Traditionally, sharing these interactions to the cloud can raise privacy concerns. Federated learning addresses this by enabling collaborative training without exposing local data. However, standard federated learning methods require backpropogation (BP), which is too resource-intensive to run on smartphones, tablets, and other edge devices.
Dr. Sun’s team is pioneering a BP-free system that eliminates these heavy computations, introducing a data-aware sampling scheduler that optimizes learning based on local data distributions. This method reduces redundancy and conserves device resources.
Importantly, this framework also enables personalized adaptation: devices can fine-tune their models with factual knowledge unique to each user, while still benefiting from teamwork across all devices. The result is a scalable solution that keeps user data private, delivers efficient operation, and provides user-specific personalization.
The team’s framework will be deployed across NVIDIA’s FLARE platform, leveraging high-performance GPUs for training and real-world evaluation on NVIDIA hardware-based federated learning systems.
With this grant and NVIDIA’s support, the Warren B. Nelms Institute is poised to push the boundaries of personalized, privacy-conscious AI at the edge, enhancing both how our devices understand us and how we stay in control of our own data.