The exhibition explores the creative tension and interplay between Sound and Image. The works showcased engage with possible ways of narration whether guided or enhanced, altered or disrupted through the Sound/Image interface or explored solely through a sense of hearing or a sense of sight.
Curatorial team: Katarzyna Kosmala and Kumiko Kushiyama
In both childhood and adulthood, training is necessary to read music scores, which sometimes makes music composition and performance difficult to learn and enjoy. In this research, we proposed a system that enables users to play their own handwritten musical notation by using our intuitive musical interface.
Since the 1960s, Optical Music Recognition (OMR) has become established in the field of printed scores. Recently, Yamamoto proposed an interactive musical system that directly utilizes printed music scores as an instrument using matching keypoints. However, little research on handwritten notation has been done, nor on interactive systems for OMR. Therefore we created a system that combines notating with performing in order to make music more intuitive as a way to assist those learning how to read and write musical scores.
Tetsuaki Baba, Yuya Kikukawa, Kumiko Kushiyama, Gocen: Appropriating Simplified Handwritten Notation as a Musical Interface, Journal of Asia Digital Art and Design Association, ADADA, Vol.18, No.1, 2014 [PDF]
Yuya (Doctoral course) and Takaharu (Alumni) launched a startup company.
The company name is No New Folk. They are now under the DMM Akihabara group. Their first product is “Orphe”, which is a smart-shoes system designed for performance that functions both as a customizable lighting system and a musical instrument/audio-visual controller.
They have already succeeded to be funded on Indigogo.
Orphe is a smart-shoes system designed for performance that functions both as a customizable lighting system and a musical instrument/audio-visual controller. The sole of each shoe contains advanced motion sensors, around 100 full-color, serially-controlled LEDs, and a wireless module (patent pending). This technology allows users to intuitively express themselves in new and interesting ways by freely mapping interactions between their movements and light and sound. We also provide a system that makes it easy for users to share the assets they’ve created online, and we hope to grow a community of artists and performers working in different media and genres who can take inspiration from each other’s use of Orphe hardware and applications.
NAKANISYNTH is a synthesizer application that allows users to produce sound loops by freehand drawing sound waves and envelope curves. Since the only input required involves drawing two waveforms, users can easily produce various sounds intuitively without complex manipulation. This application has a keyboard where users can edit waveforms and make sounds simultaneously. Therefore, it is easy for the user to understand the relationship between a waveform and the sound it produces.
Kyosuke NAKANISHI, Tsubasa YOHA, Paul HAIMES, Tetsuaki BABA, Kumiko KUSHIYAMA, NAKANISYNTH: A Freehand Drawing Waveform Synthesizer, ID:110, Art Paper, 12th International Conference for Asia Digital Art and Design,15-16 November, 2014 [PDF]
Paul is currently working on a research project here at Tokyo Metropolitan University’s System Design faculty looking at how to make information on natural hazards (such as earthquakes, active volcanoes and typhoons) in Japan more accessible and usable. It’s very early stages but the plan is to build a smart phone app. We’ve recently launched a mini-site for the project at hazards.jp. This research will build on some of the findings from Paul’s PhD and the MyFireWatch project.