Skip to content

Case study 01

View Synthesis of Three-Dimensional Spaces Using Artificial Intelligence

Timeline: 2022

Project thumbnail for the novel view synthesis research project

The vision

This project started as a research question: can a model generate a convincing new view of a three-dimensional scene from only a limited number of existing views? I approached it through conditional GANs and built a controlled Unity setup to generate the data I needed.

To make the experiments comparable, I created several difficulty levels. Simpler scenes worked better, while more complex ones exposed the limits of the models pretty quickly. That contrast was useful because it made the strengths and weaknesses of each approach much easier to see. One practical use case I had in mind was automotive systems, especially around parking and camera-assisted navigation.

What I liked most about the project was the mix of theory and implementation. It was not just about reading papers. I also had to generate data, run experiments, compare outputs, and turn the whole process into a written result that was clear enough to explain to other people. The final paper was later published on figshare.

Strategic goals

  1. 01

    Test a concrete approach

    Explore whether image-to-image translation models could synthesize useful new viewpoints from limited scene data.

  2. 02

    Understand the limits

    See how quickly quality dropped once the scenes became more complex and the generated details had to carry more information.

The challenge

The models handled simpler scenes reasonably well, but they struggled once scene complexity and variation increased.

Resolution

Keeping the dataset controlled made the comparisons more useful, and trying LSGAN helped improve stability and some of the visual results.

Synthetic 3D parking scene created in Unity
Comparison of different dataset difficulty levels used in training

W-Seminar research project with a published paper on figshare.

Context

Thumbnail image for the novel view synthesis project

Next Experience

Portfolio 2024