Shadow Puppet Sequencer
Featured on : MAKE, Daily DIY, Toys Gadget, Electronics Infoline, Web News Site
The Shadow Puppet Sequencer Table demonstrates an approach to narrative construction that is intimately coupled with sound design. The table provides a translucent surface upon which cutouts can be placed and moved, creating shadows that double as musical notes. The relationships between the shadow objects are thus spatial as well as musical. These relationships can be manipulated by moving the objects on the surface of the table. The narrative is propelled by changing the spatial–and thus musical–relationships of the objects over time.
The Shadow Puppet Sequencer Table is essentially a table with a translucent plexiglass top, a light above it, and a camera below. Objects or cutouts (shapes) are placed on the table in order to create shadows. The camera image is scanned using custom software (written in C++/openFrameworks). The position and size of the shadows are detected and this information is sent to a custom software synthesizer (written in ChucK) for auralization. The camera image is also projected behind the performer, marrying the visuals and sound as one single action for both the performer and audience.
At present, the software does not identify the shape of the object. While identification is possible, it is not necessary. Instead, each shape is treated as an outline on a 2-dimensional plane, defined by camera image. Under normal use, the Y-axis of the plane describes the frequency domain while the X-axis describes the time domain. The resolution of the frequency range is determined by the number of pixels on the Y-axis of the camera image (480). The resolution of the time domain is dependent on the speed of the scan, since new data only comes in through the camera every 30th of a second. The shapes are effectively treated like outlines of “grain clouds” as described by Curtis Roads in “Microsound.” The grains themselves are (at present) sine wave grains with a hanning envelope.
These engineering details are arbitrary, however, as the defining feature of the interface is its ability to be used as a narrative platform. The shapes can be abstract or they can be recognizable shapes (e.g. dolphins, people, birds), which can be characters or props as well as notation for a sound event. Since any shadow–not just those created by cutouts–will be rendered as sound, further embellishment can be given through the use of the hand shadows as the cutouts are animated.