Automatic Extraction of the Inference Structure of a CNN as a DAG for Temporal Analysis of Many-Core Systems

Abstract

Convolutional Neural Networks (CNNs) have become indispensable in Artificial In- telligence applications due to their ability to automatically extract hierarchical features. However, when deployed in real-time and embedded systems, accuracy is not the only metric of interest. In such environments, timing predictability and the guarantee of meet- ing strict deadlines are essential. This work addresses this challenge by bridging Deep Learning inference with real-time analysis through the systematic transformation of CNN models into Directed Acyclic Graphs (DAGs). The primary contribution of this project is the design of MIND Model Inference Net- work Designer software, which automatically extracts the inference structure of a CNN and converts it into a DAG compatible with the MIA Multicore Interference Analysis tool. To capture different execution strategies, we developed a set of novel and innova- tive regrouping policies, such as Macroscopic Layer-Wise Modeling (MLWM), Structured Spatial Partitioning (SSP), Redundant Convolution via Channel Tiling (RCCT), and Dense Fusion (DF). These policies balance computation, memory access, and communi- cation overheads in parallel, making it possible to analyze their impact on the many-core Kalray MPPA architectures. Furthermore, we proposed architecture-aware cost for Kalray’s architectures: Bostan 256 cores and Coolidge 80 cores with TCA co-processor. By formalizing the Worst-Case Execution Time (WCET) and Worst-Case Memory Access (WCMA) at different preci- sions FP32, FP16, INT8, we provided a relatively accurate computation model that en- ables Worst-Case Response Time (WCRT) computation through MIA. The testing phase validates our methodology with a prototype CNN and multiple parameter variations, highlighting the trade-offs between speedup, compute efficiency, interference, memory transfer costs, etc. This work demonstrates the feasibility of systematically analyzing CNNs for Real- Time Systems on many-core processors, achieving a complete workflow: from CNN to DAG extraction, to regrouping policy application, cost modeling, YAML generation, and MIA-based timing analysis. The outcomes not only validate the proposed approach but also open perspectives for extending analysis to larger models.