In the education of visually-impaired people, learning Braille is an important facilitator of basic communication through reading and writing. In recent years, education that incorporates information through tablet devices has become increasingly popular, but is difficult to use in conjunction with tactile presentation, such as Braille for visually-impaired people. Therefore, this project is developing tactile Braille presentation delivered through everyday tablet devices as a fun way of supporting Braille education. Through the development of this system for learning Braille, we hope to promote an interest in increasing the accessibility of Braille education.
In recent years, dance has been featured in a variety of media, and as such, has led to new forms of dance. Amongst these new forms are combinations of dance and costumes that utilise digital technology such as LEDs. However, these systems do not synchronise easily with other aspects of dance performance, such as music and lighting. Therefore, in this research we propose to develop a device that utilises an algorithm that allows for a performance where lighting reacts to the performer’s movements.
In both childhood and adulthood, training is necessary to read music scores, which sometimes makes music composition and performance difficult to learn and enjoy. In this research, we proposed a system that enables users to play their own handwritten musical notation by using our intuitive musical interface.
Since the 1960s, Optical Music Recognition (OMR) has become established in the field of printed scores. Recently, Yamamoto proposed an interactive musical system that directly utilizes printed music scores as an instrument using matching keypoints. However, little research on handwritten notation has been done, nor on interactive systems for OMR. Therefore we created a system that combines notating with performing in order to make music more intuitive as a way to assist those learning how to read and write musical scores.
Tetsuaki Baba, Yuya Kikukawa, Kumiko Kushiyama, Gocen: Appropriating Simplified Handwritten Notation as a Musical Interface, Journal of Asia Digital Art and Design Association, ADADA, Vol.18, No.1, 2014 [PDF]
This research integrates interactive media technologies, such as motion detectors, light sensors and motion actuators, into the design of architectural modular screen structures in order to create new types of intelligent, interactive screen wall systems. The results will have multiple practical and artistic applications, including architectural sun shading, privacy screening and varied interactive media applications. This work will combine the beauty, utility and spatial qualities found in architectural screen design with the intelligence and interface of interactive media.
This research proposes an interface where users can build cooperative relationships through visual and auditory communication. In this context, “cooperative relationship” refers to users determining their actions while being aware of another user’s actions. We made a transparent screen that measures touch position by using depth camera, and visually mapped sound harmonics by Lissajous curve.
Shiftbeatはユーザが直感的に音楽演奏ができるインタフェースです。静電容量センサーによるジェスチャー認識を用いて直感的な入力を実現しました。入力センサに触る強さ、時間などの触り方の変化でパラメーターを調整できます。本作では、既存の音楽インタフェースでは入力しづらかった人為的なリズムのズレや音の強弱などを表現することを目標としました。 Shiftbeat is an interface that allows the user to intuitively perform music. Intuitive input was created by using gesture recognition through the use of a capacitive sensor. By adjusting the amount of pressure applied to the input sensor, you can adjust parameters, such as time intervals. We aimed to produce an artificial rhythm and an efficient way of producing sound that is not possible to perform with existing musical interfaces.
Hand Writing is a system which can efficiently improve the ability to read and write Chinese characters of elementary school students. We created a fun interactive gameplay system where students can practice Chinese characters using a digital pen. By writing Chinese characters with the digital pen, the system recognizes the hand-written characters and compares them with the Chinese characters stored in database. It is possible to manipulate characters on the screen. Different characters have different tones, and sounds play when characters collide. We aimed to utilize the interactive system to provide students with a more enjoyable experience when writing Chinese characters, thus increasing their willingness to learn. The system currently contains 80 characters from the first grade of school. A future research goal is to provide all the Chinese characters that students learn in upper grades of school.
In recent years, due to the inclusion of popular dance in compulsory education, and the popularity of dance idols, there is a growing interest in dance. However, dance is often carried out in accordance with the music and video, rather than the sound and video responding to the dance. Therefore, we proposed a device for directing multimedia. For the purpose of this study, we focused on the dance genre known as “Lock”, which features prominent hand motions when compared to other styles of dance. When worn by the performer, this device – through various sensors – recognizes the speed and force of the performer’s movements, which affects the music and video accordingly.
ComBand is a wristband device and communication system to allow haptic communication between remote locations. This device obtains the position and pressure of the user touching the device, and transmits the information to a communication partner by tactile display using a vibration motor and rotary motor. We enable haptic conversations through this device.