WaveNeRF: Wavelet-based Generalizable Neural Radiance Fields

by   Muyu Xu, et al.

Neural Radiance Field (NeRF) has shown impressive performance in novel view synthesis via implicit scene representation. However, it usually suffers from poor scalability as requiring densely sampled images for each new scene. Several studies have attempted to mitigate this problem by integrating Multi-View Stereo (MVS) technique into NeRF while they still entail a cumbersome fine-tuning process for new scenes. Notably, the rendering quality will drop severely without this fine-tuning process and the errors mainly appear around the high-frequency features. In the light of this observation, we design WaveNeRF, which integrates wavelet frequency decomposition into MVS and NeRF to achieve generalizable yet high-quality synthesis without any per-scene optimization. To preserve high-frequency information when generating 3D feature volumes, WaveNeRF builds Multi-View Stereo in the Wavelet domain by integrating the discrete wavelet transform into the classical cascade MVS, which disentangles high-frequency information explicitly. With that, disentangled frequency features can be injected into classic NeRF via a novel hybrid neural renderer to yield faithful high-frequency details, and an intuitive frequency-guided sampling strategy can be designed to suppress artifacts around high-frequency regions. Extensive experiments over three widely studied benchmarks show that WaveNeRF achieves superior generalizable radiance field modeling when only given three images as input.


page 2

page 4

page 7


Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenes

Recent neural view synthesis methods have achieved impressive quality an...

Harnessing Low-Frequency Neural Fields for Few-Shot View Synthesis

Neural Radiance Fields (NeRF) have led to breakthroughs in the novel vie...

Wavelet-Based Network For High Dynamic Range Imaging

High dynamic range (HDR) imaging from multiple low dynamic range (LDR) i...

Light Field Denoising via Anisotropic Parallax Analysis in a CNN Framework

Light field (LF) cameras provide perspective information of scenes by ta...

A Differential Volumetric Approach to Multi-View Photometric Stereo

Highly accurate 3D volumetric reconstruction is still an open research t...

Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction

In recent years, huge progress has been made on learning neural implicit...

Learning a Wavelet-like Auto-Encoder to Accelerate Deep Neural Networks

Accelerating deep neural networks (DNNs) has been attracting increasing ...

Please sign up or login with your details

Forgot password? Click here to reset