Neural Radiance Fields for Novel View Synthesis

Tech stack: PyTorch, NumPy, COLMAP, FFmpeg, Volumetric Rendering, Positional Encoding, MLP, Adam Optimizer
A from-scratch implementation of Neural Radiance Fields (NeRF) — a deep learning approach to 3D scene representation that learns a continuous volumetric scene function from posed RGB images and synthesizes photorealistic novel views via differentiable volume rendering. Built the entire pipeline including ray generation, positional encoding, the coarse/fine MLP architecture, volumetric integration, and the training loop. Validated on synthetic benchmarks (Lego, Ship) and extended to custom real-world datasets captured on an iPhone 15 Pro.
Highlights

Implemented the complete NeRF pipeline end-to-end: ray generation from camera intrinsics/extrinsics, sinusoidal positional encoding (separate frequency bands for spatial position and viewing direction), and the 8-layer MLP with skip connection that maps 5D inputs (x, y, z, θ, φ) to volume density σ and view-dependent RGB.
Implemented differentiable volume rendering with alpha compositing along sampled rays — accumulating transmittance Tᵢ and opacity αᵢ to produce final pixel colors that gradients flow through.
Implemented stratified sampling for unbiased coverage of the rendering integral, plus hierarchical sampling with separate coarse and fine networks — using the coarse network's weights as a PDF to focus the fine network's samples on informative regions.
Built a custom real-world dataset pipeline: extracted frames from iPhone 15 Pro video using FFmpeg, ran COLMAP for structure-from-motion pose estimation, and converted the output into NeRF-compatible transforms_*.json format.
Documented and debugged real engineering issues, including a non-square aspect-ratio bug in the focal-length parameters that produced unrecognizable renders, and a subtle bug where "disabling" positional encoding by zeroing the sine-wave period produced black images instead of correctly bypassing the encoding.