Designing a hyperinstrument with gesture interface for musical performance

Cronje, Mario (Stellenbosch : University of Stellenbosch, 2011-10)

Thesis (M. Phil.)--University of Stellenbosch, 2005.

Thesis

ENGLISH ABSTRACT: The field of gesture based research and the interaction between human and computer with the focus falling on musical applications, is well established internationally. However, in South Africa, research in this field appears dormant. The reasons for this state of affairs are complex and can be argued from different angles covering socio-economical, philosophical and educational perspectives. This document describes the design, creation and implementation of an operational gesture interface environment which holds the potential to be expanded in the future. The implementation draws on cost-efficient hard- and software in the design of elementary to more advanced musical and even non-musical virtual environments (VEs) harbouring potential for further research and performance. Hard- and software available at Stellenbosch University’s Konservatorium were used together with selected free downloadable software from the internet in creating VEs, which to a degree simulate other techniques of sound manipulation. The choice of software was guided by the availability of support and prominence in terms of usage. Software basically had to incorporate hand movement tracking and the mapping of data to manipulate several parameters. Three independent systems, each representing a different VE, were studied, experimented with and programmed in order to validate the thesis. The first system manipulates a complete electronic musical instrument. The second system incorporates the simulation of a real-life musical performance and the third system focuses on manipulating specific sequencing software by a basic alternative computer mouse implementation. The outcome of this thesis provides an environment within which several programming techniques are treated and combined to form a template for teaching this field, and future development and research. These techniques incorporate the manipulation of digital audio, deal with a digital communication protocol, basic computer graphics and other necessary programming algorithms. In addition, the thesis strives to provide an outline for the understanding, design and implementation of a VE installation. The three systems will be installed for operation during a presentation of this thesis. Together with the three operative systems, this document strives to act as an initial platform from which exciting futuristic research and activity can be launched.

AFRIKAANSE OPSOMMING: Die gebied van beweging gebaseerde navorsing en die interaksie tussen mens en rekenaar waar die fokus op musikale toepassings val, is internasionaal stewig gevestig. In Suid-Afrika egter, blyk navorsing op hierdie gebied sluimerend te wees. Die redes vir hierdie stand van sake is kompleks en kan vanuit verskillende hoeke wat sosio-ekonomiese, filosofiese en opvoedkundige perspektiewe insluit, beredeneer word. Hierdie dokument beskryf die ontwerp, skep en implementering van ‘n operasionele bewegings koppelvlak omgewing met potensiaal tot uitbreiding. Die implementering baseer op kosteeffektiewe hardeware en programmatuur in die ontwerp van eenvoudige tot gevorderde virtuele omgewings (VOs) vir musiek, en selfs nie-musikale dissiplines met die potensiaal tot verdere navorsing en implementering binne die musikale uitvoeringspraktyk. Hardeware en programmatuur beskikbaar aan die Konservatorium is gebruik tesame met ‘n seleksie van gratis programmatuur op die internet beskikbaar om VOs wat ander klankmanipulerings tegnieke simuleer, te skep. Die keuse van programmatuur is gelei deur die beskikbaarheid van ondersteuning en gewildheid en inkorpeer die volg van hand beweging asook die verspreiding van data om verskeie parameters te manipuleer. Drie onafhanklike sisteme wat elk ‘n ander VO voorstel, is bestudeer, mee ge-ekperimenteer en geprogrammeer om die tesis te valideer. Die eerste sisteem manipuleer ‘n volledige elektroniese musiekinstrument. Die tweede sisteem inkorporeer die simulasie van ‘n werklike musikale uitvoering en die derde sisteem fokus op die manipulasie van spesifieke sequencing programmatuur sonder die hulp van ‘n muis. Die uitkoms van hierdie tesis verskaf ‘n omgewing waarbinne heelparty programmerings tegnieke bespreek en gekombineer word in ‘n aanpasbare templaat vir die onderrig van hierdie veld asook toekomstige ontwikkeling en navorsing. Hierdie tegnieke inkorporeer die manipulasie van digitale klank, die omgang met ‘n digitale kommunikasie protokol, basiese rekenaargrafika en verdere noodsaaklike programmerings algoritmes. Verder streef die tesis daarna om ‘n raamwerk vir die begrip, ontwerp en implementering van ‘n VO daar te stel. Die drie sisteme wat in hierdie tesis bespreek word, sal operasioneel geïnstalleer word gedurende ‘n demonstrasie daarvan. Saam met die drie werkende sisteme streef hierdie dokument daarna om te dien as platform waarvan af opwindende futuristiese navorsing en aktiwiteite geïniseer kan word.

Please refer to this item in SUNScholar by using the following persistent URL: http://hdl.handle.net/10019.1/16602
This item appears in the following collections: