Game and Media Technology

The projects on this page relate to my time at the University. Game and Media Technology is not just used in games. On the contrary, there are plenty of links with other disciplines! For example, graphics relates to imaging, animation to robotics, game engines to simulations and AI to strategic decisions. The central element is interesting to me: math.

Video-Based Scene and Material Editing

This is my MSc project in the area of Computer Vision, which entails analysis of the environment in the input footage, in order to build a model. This model is then used to render a new object into the scene, including interaction of light and shadows with the original geometry.

A big part of the challenge is camera tracking, in order to get an idea of the geometry of the scene. The result of this procedure is a motion of the camera through space, which is essential to the render phase.

The render phase itself is done by the additive differential rendering: two images are produced, one of the scene model including the object of interest, and one of only the scene model. By subtracting these from one another, an image is obtained containing shadows and global illumination.

Image-Based Material Editing

During my bachelor studies I worked on a technique called Image- Based Material Editing. This technique is about changing materials of scene object, with only limited knowledge about the scene. Based on light physics models, the original object is removed and then re-rendered using a new material.

Input Output

The technique is originally developed by Khan et al. I made an own implementation and performed various experiments.

Self-Driving Car

This project was carried out in a group of students, for the Dutch company Kiminkii. The goal was to provide computer vision software to steer a self-driving car. In particular road and obstacle recognition.

The input to the software are two camera feeds placed in stereo. We implemented a pipeline to inject various stages producing intermediate results. These included among other things:

  • Camera calibration
  • Channel decomposition
  • Preprocessing filters
  • Features
  • Feature matches
  • Depth map
  • Recognized objects
  • Recognized road delineations
  • Planned route
  • Steering wheel input

The "Feature matches" stage was most challenging since it had to be done in real time. We implemented various stages, among this one, on the GPU. With rigorous optimization we went from a frame per few hours (initial implementation on CPU) to multiple frames per second.