Browser based gamified audiovisual art piece



Inter_active is a gamified installation by Juha van Ingen, inspired by the ancient Moksha Patam (the ladder to salvation), commonly known as the snakes and ladders game.

The artists wanted there to be two paths, which in practice meant two sequences of dozens of images side by side on the screen. The images are animated gifs, and the player advances on the path by pressing keyboard buttons.

The images are accompanied by a sound track, realized by selected sound artists. The sounds loop independently from the images, and the player is able to stop or play a sound. The sound selection also advances sequentially, and maximum of two sounds can be heard at the same time.

The main challenge is to see all of the animations and reach the end. The player is also able to create a combination of images and sounds, and thus become a creator in the process.

The work was realized with HTML5, CSS and Javascript (jQuery).


Sound of the Rings of Saturn – A sonification of Space

The rings of Saturn look like a spectrogram, so it occurred to me try how it would sound like. In essence, this a sonification of space!

Sound was created by using an authentic image of rings of Saturn as a spectral source to a series of filters. A 1 pixel slice of the image of the rings was extracted

The ring spectrogram was divided into three color planes, and the color intensity values were transformed into resonant filter cutoff frequencies. In essence one filter unit (per color plane) has 256 sounds playing simultaneously. The individual filters are placed along the x-axis so, that the stereo image consists of 256 steps from left to right. The last two sounds were created with 1024 voices and 3×340 voices.

The spectrum was compressed to a couple of ranges. In some sounds a small variation in certain divider factor per color plane is introduced for a slight chorus like effect. The original lossless sound bits are available on Freesound.


An interactive sound waves building tool based on live video

The interactive sound waves builder was presented at the Helsinki Hacklab stand at WÄRK:fest, a DIY/Hacker/Culture -festival. The system consists of a small table, webcam and Max/MSP/Jitter process. Visitors were able to manipulate the sound waves by (re)arranging colorful objects on the table.

The table was simultaneously scanned in two dimensions, and the extracted data was used to create the sound waves, pitches and amplitudes for the oscillators. Besides the musical piece the rearrangement of the colorful objects created a transforming visual piece.


Sonification of a graphical piece – Sounds of Calligraphy

The three pieces of Sounds of Calligraphy playlist are the continuation from the Sarana performance at the opening of Viiva & Viiru calligraphy group exhibition in Helsinki, Finland. The pieces are based on a real time process that analyzes the images in three color channels, and some of the oscillator sound waves are derived from the image properties too. The process has 128 oscillators and filters per channel, and they are positioned to the stereo panorama to exactly follow the figures. The resulting music is thus a true voyage across the image.

The sonification process was realized with Max/MSP/Jitter. A very warm thanks to the Viiva & Viiru group for their openness to the abstract.