Pose-Free Neural Radiance Fields via Implicit Pose Regularization

by   Jiahui Zhang, et al.

Pose-free neural radiance fields (NeRF) aim to train NeRF with unposed multi-view images and it has achieved very impressive success in recent years. Most existing works share the pipeline of training a coarse pose estimator with rendered images at first, followed by a joint optimization of estimated poses and neural radiance field. However, as the pose estimator is trained with only rendered images, the pose estimation is usually biased or inaccurate for real images due to the domain gap between real images and rendered images, leading to poor robustness for the pose estimation of real images and further local minima in joint optimization. We design IR-NeRF, an innovative pose-free NeRF that introduces implicit pose regularization to refine pose estimator with unposed real images and improve the robustness of the pose estimation for real images. With a collection of 2D images of a specific scene, IR-NeRF constructs a scene codebook that stores scene features and captures the scene-specific pose distribution implicitly as priors. Thus, the robustness of pose estimation can be promoted with the scene priors according to the rationale that a 2D real image can be well reconstructed from the scene codebook only when its estimated pose lies within the pose distribution. Extensive experiments show that IR-NeRF achieves superior novel view synthesis and outperforms the state-of-the-art consistently across multiple synthetic and real datasets.


page 2

page 6

page 8


VMRF: View Matching Neural Radiance Fields

Neural Radiance Fields (NeRF) have demonstrated very impressive performa...

MELON: NeRF with Unposed Images Using Equivalence Class Estimation

Neural radiance fields enable novel-view synthesis and scene reconstruct...

Neural Object Learning for 6D Pose Estimation Using a Few Cluttered Images

Recent methods for 6D pose estimation of objects assume either textured ...

Robustifying the Multi-Scale Representation of Neural Radiance Fields

Neural Radiance Fields (NeRF) recently emerged as a new paradigm for obj...

Visual Compiler: Synthesizing a Scene-Specific Pedestrian Detector and Pose Estimator

We introduce the concept of a Visual Compiler that generates a scene spe...

Improving Robustness for Pose Estimation via Stable Heatmap Regression

Deep learning methods have achieved excellent performance in pose estima...

Efficient Planar Pose Estimation via UWB Measurements

State estimation is an essential part of autonomous systems. Integrating...

Please sign up or login with your details

Forgot password? Click here to reset