Harnessing Low-Frequency Neural Fields for Few-Shot View Synthesis

by   Liangchen Song, et al.

Neural Radiance Fields (NeRF) have led to breakthroughs in the novel view synthesis problem. Positional Encoding (P.E.) is a critical factor that brings the impressive performance of NeRF, where low-dimensional coordinates are mapped to high-dimensional space to better recover scene details. However, blindly increasing the frequency of P.E. leads to overfitting when the reconstruction problem is highly underconstrained, \eg, few-shot images for training. We harness low-frequency neural fields to regularize high-frequency neural fields from overfitting to better address the problem of few-shot view synthesis. We propose reconstructing with a low-frequency only field and then finishing details with a high-frequency equipped field. Unlike most existing solutions that regularize the output space (\ie, rendered images), our regularization is conducted in the input space (\ie, signal frequency). We further propose a simple-yet-effective strategy for tuning the frequency to avoid overfitting few-shot inputs: enforcing consistency among the frequency domain of rendered 2D images. Thanks to the input space regularizing scheme, our method readily applies to inputs beyond spatial locations, such as the time dimension in dynamic scenes. Comparisons with state-of-the-art on both synthetic and natural datasets validate the effectiveness of our proposed solution for few-shot view synthesis. Code is available at \href{https://github.com/lsongx/halo}{https://github.com/lsongx/halo}.


page 2

page 5

page 6

page 7

page 8

page 14

page 15

page 16


WaveGAN: Frequency-aware GAN for High-Fidelity Few-shot Image Generation

Existing few-shot image generation approaches typically employ fusion-ba...

FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularization

Novel view synthesis with sparse inputs is a challenging problem for neu...

WaveNeRF: Wavelet-based Generalizable Neural Radiance Fields

Neural Radiance Field (NeRF) has shown impressive performance in novel v...

AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training

Neural Radiance Fields (NeRFs) are a powerful representation for modelin...

VGOS: Voxel Grid Optimization for View Synthesis from Sparse Inputs

Neural Radiance Fields (NeRF) has shown great success in novel view synt...

ActiveNeRF: Learning where to See with Uncertainty Estimation

Recently, Neural Radiance Fields (NeRF) has shown promising performances...

Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction

In recent years, huge progress has been made on learning neural implicit...

Please sign up or login with your details

Forgot password? Click here to reset