Real-Time Multi-Learning Deep Neural Network on an MPSoC-FPGA for Intelligent Vehicles: Harnessing Hardware Acceleration With Pipeline


Tatar G., Bayar S., Çiçek I.

IEEE Transactions on Intelligent Vehicles, cilt.10, sa.4, ss.1-12, 2024 (SCI-Expanded) identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 10 Sayı: 4
  • Basım Tarihi: 2024
  • Doi Numarası: 10.1109/tiv.2024.3398215
  • Dergi Adı: IEEE Transactions on Intelligent Vehicles
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Sayfa Sayıları: ss.1-12
  • Marmara Üniversitesi Adresli: Evet

Özet

This study introduces a new method to enhance ADAS's safety and error prevention capabilities in intelligent vehicles. We address the significant computational and memory demands required for real-time video processing by leveraging BDD100 K, KITTI, CityScape, and Waymo datasets. Our proposed hardware-software co-design integrates an MPSoC-FPGA accelerator for real-time multi-learning models. Our experimental results exhibit that, despite an increase in ADAS tasks and model parameters compared to the state-of-the-art studies, our model achieves 24,715 GOP performance with 4% lower power consumption (6.920 W) and 18.86% less logic resource consumption. The model processes highway scenes at 22.45 FPS and attains 50.06% mAP for object detection, 57.05% mIoU for segmentation, 43.76% mIoU for lane detection, 81.63% IoU for drivable area segmentation, and 9.78% SILog error for depth estimation. These findings confirm the system's effectiveness, reliability, and adaptability for ADAS applications and represent a significant advancement in intelligent vehicle technology, with the potential for further improvements in accuracy and memory efficiency.