Synthetic Scenes, Real Performance: Evaluating Object Detection Models Trained on Virtual Data


Yoluk M., Bahcekapili I., Bahar O. F., YILDIZ K.

9th International Artificial Intelligence and Data Processing Symposium, IDAP 2025, Malatya, Turkey, 6 - 07 September 2025, (Full Text) identifier

  • Publication Type: Conference Paper / Full Text
  • Doi Number: 10.1109/idap68205.2025.11222178
  • City: Malatya
  • Country: Turkey
  • Keywords: computer vision, game engine, object detection, perception, synthetic data, unity
  • Marmara University Affiliated: Yes

Abstract

Advances in deep learning have significantly increased object detection accuracy and efficiency; however, their reliance on large-scale annotated datasets presents significant challenges, particularly in sensitive, rare, or logistically complex environments. Synthetic data generated through controlled 3D simulations offers a viable solution by enabling automatic and cost-effective image labeling. This study investigates the effectiveness of using synthetic datasets generated through the Unity game engine and the Unity Perception package to train deep learning object detection models. A diverse set of realistic synthetic images was generated using Unity's High Resolution Rendering Pipeline (HDRP) and augmented with various randomizers to manipulate object placement, textures, colors and rotations. A custom synthetic dataset consisting of 10,000 images, each labeled for a Boeing 737 aircraft, was generated. The models were then trained on these synthetic images. In real-world tests, Faster R-CNN showed high generalization performance with 71.94% AP, while YOLOv11's performance was significantly lower with 26.46% AP. Future work can further improve model accuracy by examining hybrid datasets combining real and synthetic images and testing various model architectures.