The Synesthete’s Music Machine takes the form of a vintage-looking gramophone, but with a twist. Instead of a arm and needle, it uses a webcam to interpret the colours on the turntable into sound. The user can experiment with different “vinyl records” or create their own sounds by placing coloured blocks on the turntable.
Attached images are of a prototype version using a rectangular white box as a platform instead of a gramophone, but the functionality and use concept is the same. The platform is where the image is placed. The image can be any picture or physical objects that are coloured. The webcam sees the colours in the image and a custom software program will run an algorithm to translate the colours into sound based on the position and the quantity of that colour. For example, a small quantity of the colour yellow will mean a smaller volume of the sound assigned to the colour yellow. The user can choose their sounds from a graphical user interface on a computer screen, and experiment with different object placements to create their own sound sequence. The sound output is heard in real time and the scanning process can be seen on the screen as well. This was showcased in the Singapore mini-maker faire this year and children loved to play with the lego and sounds. An improved version will be showcased at the Penang mini-maker faire whereby the platform takes the form of a gramophone and the platform will be circular instead of rectangular.
More information can be found at http://lohjianhui.com/the-synesthetes-music-machine/