This is my MSc project in the area of Computer Vision, which entails analysis of the environment in the input footage, in order to build a model. This model is then used to render a new object into the scene, including interaction of light and shadows with the original geometry.
A big part of the challenge is camera tracking, in order to get an idea of the geometry of the scene. The result of this procedure is a motion of the camera through space, which is essential to the render phase.
The render phase itself is done by the additive differential rendering: two images are produced, one of the scene model including the object of interest, and one of only the scene model. By subtracting these from one another, an image is obtained containing shadows and global illumination.