Original title: Confidant: Customizing Transformer-based LLMs via Collaborative Edge Training
Authors: Yuhao Chen, Yuxuan Yan, Qianqian Yang, Yuanchao Shu, Shibo He, Jiming Chen
Confidant, a framework for mobile devices, aims to customize and train complex language models like Transformers. This addresses the challenge of deploying these models on devices with limited resources. It partitions the model and uses pipeline parallel training, dividing the work across devices efficiently. Additionally, it allocates specific tasks to different hardware components, optimizing the use of mobile CPUs and GPUs. Initial tests reveal promising outcomes, showcasing up to 45.3% memory reduction and an 8.03x speed boost in practical scenarios. This innovation could enable powerful language models to run effectively on everyday smartphones, opening doors to a wider range of applications without overwhelming device resources.
Original article: https://arxiv.org/abs/2311.13381