Real-Time And Efficient Scene Graph Generation for Real-World Applications:
End-to-End Investigation

2023 - Multi-Layer Scene Graph Generation for Service Robotics
2023 - Multi-Layer Scene Graph Generation for Service Robotics

Project information

  • Category: PhD Thesis
  • Project start: October 2021
  • Expected completion: March 2025

Summary

Scene Graphs are powerful representations that abstract the content of images or videos in the form of relations triplets grounded to visual regions. Generating Scene Graphs through the task of Scene Graph Generation (SGG) seems especially promising for applications in Robotics such as in Human-Robot Collaboration (HRC) in domestic context where Scene Graphs can be used to model the environment and the interactions between the robot and the human. However, several years after the first inception of the task, the usage of Scene Graphs in real-world applications is still limited due to the poor performance of Scene Graph Generation (SGG) models on out-of- distribution samples. In this thesis, we propose to bridge the gap between theoretical methods of Scene Graph Generation and their practical implementations in real-world settings, successfully contributing to the democratization of the usage of Scene Graphs. We propose to investigate every bottlenecks, from data collection to model design to implementations on robotic platforms.