Conference Papers Year : 2024

Partitioned Scheduling and Parallelism Assignment for Real-Time DNN Inference Tasks on Multi-TPU

Abstract

Pipelining on Edge Tensor Processing Units (TPUs) optimizes the deep neural network (DNN) inference by breaking it down into multiple stages processed concurrently on multiple accelerators. Such DNN inference tasks can be modeled as sporadic non-preemptive gangs with execution times that vary with their parallelism levels. This paper proposes a strict partitioning strategy for deploying DNN inferences in real-time systems. The strategy determines tasks' parallelism levels and assigns tasks to disjoint processor partitions. Configuring the tasks in the same partition with a uniform parallelism level avoids scheduling anomalies and enables schedulability verification using well-understood uniprocessor analyses. Evaluation using real-world Edge TPU benchmarks demonstrated that the proposed method achieves a higher schedulability ratio than state-of-the-art gang scheduling techniques.
Fichier principal
Vignette du fichier
NP_Gang_Strict_Partition_DAC_2024.pdf (632.7 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04803800 , version 1 (26-11-2024)

Identifiers

Cite

Binqi Sun, Tomasz Kloda, Chu-Ge Wu, Marco Caccamo. Partitioned Scheduling and Parallelism Assignment for Real-Time DNN Inference Tasks on Multi-TPU. 61st ACM/IEEE Design Automation Conference (DAC), Jul 2024, San Francisco, United States. pp.1 - 6, ⟨10.1145/3649329.3655979⟩. ⟨hal-04803800⟩
2 View
9 Download

Altmetric

Share

More