Scheduling of Deep Learning Applications onto Heterogeneous Processors in an Embedded Device

Duseok Kang, Jinwoo Oh, Jongwoo Choi, Youngmin Yi, Soonhoi Ha

Research output: Contribution to journalArticlepeer-review

27 Scopus citations

Abstract

As the need for on-device machine learning is increasing recently, embedded devices tend to be equipped with heterogeneous processors that include a multi-core CPU, a GPU, and/or a DNN accelerator called a Neural Processing Unit (NPU). In the scheduling of multiple deep learning (DL) applications in such embedded devices, there are several technical challenges. First, a task can be mapped onto a single core or any number of available cores. So we need to consider various possible configurations of CPU cores. Second, embedded devices usually apply Dynamic Voltage and Frequency Scaling (DVFS) to reduce energy consumption at run-time. We need to consider the effect of DVFS in the profiling of task execution times. Third, to avoid overheat condition, it is recommended to limit the core utilization. Lastly, some cores will be shut-down at run-time if core utilization is not high enough, in case the hot-plugging option is turned on. In this paper, we propose a scheduling technique based on Genetic Algorithm to run DL applications on heterogeneous processors, considering all those issues. First, we aim to optimize the throughput of a single deep learning application. Next, we aim to find the Pareto optimal scheduling of multiple DL applications in terms of the response time of each DL application and overall energy consumption under the given throughput constraints of DL applications. The proposed technique is verified with real DL networks running on two embedded devices, Galaxy S9 and HiKey970.

Original languageEnglish
Article number9019698
Pages (from-to)43980-43991
Number of pages12
JournalIEEE Access
Volume8
DOIs
StatePublished - 2020

Keywords

  • Deep learning scheduling
  • genetic algorithm
  • heterogeneous processor
  • mobile device

Fingerprint

Dive into the research topics of 'Scheduling of Deep Learning Applications onto Heterogeneous Processors in an Embedded Device'. Together they form a unique fingerprint.

Cite this