Timeframe: January 2022 – August 2022

The nature of 2D cutout animation is the recording of analog fragments frame-by-frame. This technique is an established way to create moving pictures. In ExperiMotion 1, cutout animation is combined with virtual reality. Here, the animation appears at a micro-level (i.e., animated details of objects) and at a macro-level (i.e., animated environments). 

ExperiMotion 1 takes place in cooperation with guest artist Max Hattler, whose current research focuses on synesthetic experience and visual music and the narrative potential of abstract animation. Fellow invited artist Sune Petersen's practice mainly evolves around generative video performance and playful performative installations. The animation style of ExperiMotion 1 is based on the film Shift (2012) by Max Hattler, and the animated objects are borrowed from the collection of Hans „Nick“ Roericht, a design professor at the Berlin University of the Arts. Another source of inspiration is the Hattlerizer developed by Petersen and Hattler. For the auditory part of the project, Sune Petersen is developing an algorithm to sonify the stop-motion loops based on the principles of optical sound.

Artistic research is an essential part of the working process. For this purpose, logs, video recordings, and voice recordings, as well as transcriptions and screengrabs of essential milestones, discussions, and meetings, are made to understand how exactly the decision-making process takes place during the span of ExperiMotion 1.

At the beginning of ExperiMotion 1, the VRinMotion team builds two prototypes in TouchDesigner and Unity, working with video feeds and capturing the stop-motion animation right out of the VR. After discussions with the invited artists, the team decides to proceed with Unity and import cutout animation loops via an internal network into the VR. For the Unity prototype, the team implements the Mixed Reality Toolkit (MRTK) by Microsoft. With this toolkit, artists, as well as participants, can place the cutout loops freely in VR space and change size and direction. Because the backface-culling in the shader that displays the animation as texture is turned off, the cutout loops are also visible from the backside, and it is possible to build 2 ½ dimensional objects. Haptic aspects of the stop-motion objects are also translated very well into the virtual environment because of the good resolution of the Varjo XR-3 Headset used for the project.

In addition to manual placement, spatial pre-arrangement of the loops is done in After Effects and vvvv and imported into the virtual environment for further manipulation and arrangement. In addition to placing animation loops in the foreground, the cutout animation can be imported into a background sphere, which can be irregularly distorted by a UV-shifting algorithm implemented in the shader.

Talk by Franziska Bruckner: Between the Frames: Expanding Stop-Motion for Virtual and Hybrid Environments at Expanded Animation. Mapping an Unlimited Landscape, Ars Electronica Center, Linz. (2022, September 9)