here is a little audio xTuio thing i have been making. its all experimental. does any one have any ideas how a 3d tuio/osc/midi node graph visualization program should work ?

ps... sander you may notice some your smallecule icons i'm using as textures. im not really at the graphical and iconographical stage of development yet so don't worry in the finished version the music boxes will more than likely not be boxes but some sort of 3d model representing the particular units.




if this type of thing interests anybody, im happy to provide the binaries to anybody that wants to test this as its the only way i can this to run on my machine is from the binaries. my cpu and gpu are pretty much maxed and barley made this screen capture. the good thing is that tuio, osc and midi are all networkable so one machine running visuals and the other running the audio is probably the best approach. not entirely at the dual machine or even multi-touch mode yet even though this is all inherently multi-touch.


here is the write up from my vimeo page in case your wondering what is actually going on .........


this is an early experimental multi-touch music project, i am working on. it is inspired by the reactable and utilizes the tuio protocol to transmit cursor data to unity via xTuio.
in addition to being rendered in unity, quartz composer is being used to parse the spectrums peak amplitude for each instrument that is also being feed into the unity render engine via osc to visualize the waveform between the 3d music box nodes.

there are two types of nodes that both transmit midi note and control data to ableton live for synthesis. the two type of nodes are a synth unit and fx unit. although the nodes act as two different classes they are inherently the same objects transmitting the same data to live.

meaning the fx unit nodes also can act as a synth node and utilize its midi note data and like wise the synth unit nodes also send midi control messages. the nodes are transmitting a random note in the range of 0 to 8. this note is subsequently parsed and retransmitted to live in a particular scale. in addition music box is also sending 8 harmonic notes based on transformation logic as well as transmitting the subsequent rotation, position and local scale vectors for each node as midi control message.

so what this means is you could use a two finger pinch zoom to scale the node up which in turn can be mapped to an instruments amplitude or what ever parameter you would want scale to be used for. although this video is only showing positional usage both position and rotation vectors are sent to live as midi cc. doing a pinch zoom pivot rotation will give the xyz rotational messages to subsequently map.

the fx units are chained to the synth unit nodes via a spring joint so that the position of the fx unit can modify the location and rotation of the synth node by utilizing rigid body physics. meaning that the control messages of the synth unit can also be effected by the fx unit and vice versa. this is all experimental and there is no right configuration at the moment.