Deploying deep learning (DL) on mobile devices has become increasingly prevalent. DL software libraries are crucial for efficient on-device inference, alongside algorithms and hardware. However, there has been limited understanding on the performance of modern DL libraries. We fill this gap by benchmarking 6 popular DL libraries and 15 diverse models across 10 mobile devices, which reveal an unsatisfactory landscape of mobile DL: their performance is highly disparate and fragmented across different models and hardware, and the impacts often surpass algorithm or hardware optimizations, such as model quantization and GPU/NPU-based computing. Finally, we provide practical implications for stakeholders in the DL library ecosystem, and envision a more ambitious picture of future mobile AI landscape in the LLM era.

Qiyang Zhang

Benchmarking Mobile Deep Learning Software

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info

Category	Quartile
TELECOMMUNICATIONS	4