Real-time projection mapping (II)
This post from the lab reports on a personal project of the author: to augment the ‘Girls Night Out’ event at City Life Church The Hague, held on two nights in October 2012.
A 3D projection mapping was performed, using two digital projectors, to add visuals to the decor on stage. Videos of each of the show’s segments are shown below. Note that these were recorded from the author’s position on the stage, so from a strong side view, and should be considered for their documentary (and not their aesthetic) value.
The theme of the show was ‘Next Step’, challenging the audience (in a playful way) to think about where they are in their lives, and what could be their next step to take. The projection mapping was used in a number of segments of the show, to project on a set of stairs that formed the centerpiece of the decor:
- during the drama performance the stairs were turned into a waterfall
- during various monologues an ‘ambient’ visual effect was shown to enliven the stage without attracting too much attention
- during the sermon each speaking point was highlighted by showing its title on the stairs, appearing in a nice way
- during the first dance performance the stairs were enhanced with real-time effects, where the overall sequence was synchronized on a timeline, and the ‘details’ of each musical pattern were triggered using automatic onset detection based on an analysis of the Fast Fourier Transform of the audio signal
- during the second dance performance the stairs were enhanced with motion graphics created with Adobe After Effects
To produce the effects seen in above videos, a new version of Kyle McDonald’s ‘mapamok’ software was created, since a couple of new features were needed. Mapamok is a based on OpenFrameworks, a C++ framework for creative coding. The main innovation in mapamok is a very cool method for projector calibration, similar to the one we used in our first experiment with real-time projection mapping, but even better because it also estimates the internal parameters of the projector (not just the external ones, i.e. position and rotation).
Our version of mamamok has been extended with:
- support for multiple projectors
- support for multiple show segments
- support for texture-mapping based on videos and images
- modular support for real-time effects, e.g. for synchronising the visuals to important events in the music (‘onset detection’)