AutoNeRF: Training Implicit Scene Representations with Autonomous Agents - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Conference Papers Year : 2024

AutoNeRF: Training Implicit Scene Representations with Autonomous Agents

Abstract

Implicit representations such as Neural Radiance Fields (NeRF) allow to map color, density and semantics in a 3D scene through a continuous neural function. However, these models typically require manual and careful human data collection for training. This paper addresses the problem of active exploration for autonomous NeRF construction. We study how an agent can learn to efficiently explore an unknown 3D environment so that the data collected during autonomous exploration enables the learning of a high-quality neural implicit map representation. The quality of the learned representation is evaluated on four robotics-related downstream tasks: classical viewpoint rendering, map reconstruction, planning, and pose refinement. We compare the impact of different exploration strategies including frontier-based and learning-based approaches (end-to-end and modular) with different reward functions tailored to this problem. Empirical results show that NeRFs can be trained on actively collected data using just a single episode of experience in an unseen environment and that AutoNeRF, a modular exploration policy trained with reinforcement learning, enables obtaining a higher-quality NeRF for the considered downstream robotic tasks. Finally, we show that with AutoNeRF an agent can be deployed to a previously unknown scene and then automatically improve its navigation performance by adapting to the scene through a cycle of exploration, reconstruction, and policy finetuning.
Fichier principal
Vignette du fichier
IROS2024_AutoNeRF_Camera_Ready.pdf (3.82 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04688608 , version 1 (05-09-2024)

Licence

Identifiers

  • HAL Id : hal-04688608 , version 1

Cite

Pierre Marza, Laetitia Matignon, Olivier Simonin, Dhruv Batra, Christian Wolf, et al.. AutoNeRF: Training Implicit Scene Representations with Autonomous Agents. IROS 2024 - International Conference on Intelligent Robots and Systems, Oct 2024, Abu Dhabi, United Arab Emirates. pp.1-8. ⟨hal-04688608⟩
25 View
9 Download

Share

More