MELD · Multi-Dimensional Collaborative Deployment Mechanism of MoE-based Edge LLMs for 6G Ubiquitous Intelligence

Edge deployment of Large Language Models (LLMs) plays a vital role in ensuring low latency, reducing communication overhead, and enhancing privacy, bridging the gap unaddressed by cloud and on-device LLMs. However, edge LLMs face daunting challenges due to resource constraints and highly dynamic, heterogeneous environments. Notably, the Mixture-of-Experts (MoE) architecture, as seen in models like DeepSeek, has emerged as a promising solution for edge deployment. MoE enables sparse activation, dramatically lowering computational load and supporting collaborative, distributed deployment. This adaptability makes MoE-based LLMs well-suited for challenging edge scenarios. Still, several barriers persist. MoE-based LLMs typically have larger parameter sizes than dense models, requiring substantial cache memory, which strains edge resources. Additionally, frequent and voluminous inter-server data transfers, combined with limited bandwidth in edge networks compared to cloud data centers, form a critical performance bottleneck. The complexity is further compounded by the diverse and fluctuating demands of edge resources and applications, making collaborative resource allocation and efficient scheduling particularly difficult. To address these issues, advanced strategies are proposed in this project. Inter-server and intra-server collaborative deployment methods partition models based on expert activation paths and similarities, ensuring efficient distribution across edge servers and optimal expert scheduling within each server. Mixed-precision quantization enables dynamic adaptation of expert bit-widths, balancing resource constraints, application requirements, and expert popularity. Innovative token pruning and fusion mechanisms reduce data transfer frequency and volume, enhancing overall inference efficiency. This project establishes the theoretical foundations and practical methodologies for realizing high-performance and ubiquitous edge LLMs.