SoundWaves – Wearable Wireless Instrument that transforms dance into rhythmic sounds.

Idea: Give dance an audio synthesis, in order to create a unique form of performance based art.

Basic Implementation:

Implementing different Sounds:

Crow:

Bell:

Test:

Final Presentation is a performance with SoundWaves.
Video:

Performance did not show case the complete potential or vision of SoundWaves. The long, drawn out noises end up making it hard to tell what’s going on for the audience. The first acts interaction with sound was too discrete. The chirping of the second act overpowered the other sounds in the background that would showcase the manipulate of ongoing noise. The use of a totem in the center is to introduce dialogue to the performance a create an interaction to be witnessed. in the future, the totem will most likely be a small speaker from which the sound is being emitted.

Performance Logistics:

The performance consists of multiple phases:

Phase 1.) Pure_FX. Discrete long sound to project an eerie mood, great for story teller or interpretive performance. From Wide array of sounds, selection based off foot orientation.

Phase 2.) FailSafe. Hard coded 808 drum sounds corresponding to ball and heel. Optional record and playback. Also includes two quick motion switches to change from one the drum sounds mapping to another.

Phase 3.) Groove_FX. Utilize shin movement to oscillate frequency of continuous sound. Great for swing movement and ground_work.

Phase 4.) Sequence_ZF. Foot controled sequencer of 808 drum sound. Feet and dance steps are orchestrators of beat, adding and removing triggers based on time of step. Useful in combination with FailSafe phase to add background beat to dance too.

SoundWaves is a wireless wearable instrument that synthesizes dance into coordinated sound.

Goal: Orchestrate sounds producing (sampler) and Combination of Hardware and Software.

Sampler:

Originally, planning to trigger sound files in max to play one at a time, but this approach proved to limited. Audio would need to complete finish play 808 drum sound or interrupt it prematurely to keep up with users triggering.

Ali.sampler object in max allowed for multiple sound files to be played at once and layer over tope of each other to gain a poly-morfic effect.

Code:

Start of with a keyboard for your feet that implements a drum sound based on dance step.

This week was fill with experimenting with various way to play sound files in max. Tested continuous cycles and frequencies with phasors and groove object. Adapted playback volume and file choice decision based on analog data. Attempted to choose sound based on peak jerk accelerometer data from each leg, but accelerometer data is not clean enough a fast movements to grant consistent readings. So

Establish streamline method of read data. Run each accelerometer data through a smoothing filter. The Y-axis reading for each leg was offset due to 1 g of gravity in the downward direction. To counteract this for quick readings, the derivates of each of the accelerometer data were taken, and routed to the analog peak choosing method for sound files.

Since the peak method proved unsuccessful, A smart option was utilized…… the Wekinator.

Wekinator is a Neural Network Machine learning algorithm that takes in a set number of variables and yield how any number of parameters you want to fluctuate and changes depending how you train it. The Wekinator is ideal for reading the position of certain foot position and matching them to a number for a sound file to be played. It’s limitation are that is doesn’t offer discrete numbers, which becomes haphazard when in-between positions.

Since Wekinator failed to produced desired results. Switched to Node based estimation of foot position.

Accelerometer determine acceleration in 3-directions (xyz). Gravity constantly impose a 1 g of downward force on the sensor.  Utilizing this, one can roughly determine the orientation of sensor. different step position have different orientation and leans of the shin. Utilizing this I can roughly differentiate steps in the X-Z plane of the floor. using the Node object in max set up 2D plane a specified area of different sound. Various combination of Heel or Ball press and the two shins reading triggers preset sounds. This method produce best results so far.

Combination of Hardware and Software:

Integration of Hardware and software was straightforward due to the established wireless.

A interesting trait of this instrument in that it’s Completely Concealable.

Goal: Complete Hardware and establish Wireless.

Hardware:

A great majority of dance movement derives from the orientation and manipulation of the feet. So a comfortable wearable device need to be sturdy enough to hold electronics and not jerk about with sudden movements, but also need to be easily put on and taken off as well as not ungainly and perturbing for the wearer. Shinguards are a publically common and convenient hardware starting point. Simply drilling into the hard plastic and attaching needed circuitry did the trick.

Success: Fabricated both Accelerometer mounted Shinguards.

Finished establishing Force sensitive resistors (FSR) in shoes. One for each ball and heel of foot. Ingredients: wire, conductive fabric and tape.

Modified perf-board to include voltage divider for FSR’s, external 9-volt battery, 5 volt regulator, and female header pins for MMA7361L three-axis analog Accelerometer and Wixel radio module. Later, a small hall effect sensor was added for performance purposes.

Wireless:

Serial communication via Arduino Pro Mini and wixels is too slow and cumbersome. Original plan: Use two transmitting Wixels to send messages to host computer. each message had a defined start character to identify which leg it came from, then would preceed with analog values sent in char range from 0 – 255 and end with defined end character. Problems with first approach: Both wixels would attempt to contact the host computer and end up mixing their messages. Sometimes analog sensor values were confused with start characters that had the same char.

Since host receiver could read two signals at once, The second attempts focused on pinging each wixel leg indenpently to trigger call back data to be transmit. Problem with this method was timing. The serial Interface in max needed to be bang for each character read, then immediately trigger a call back for the other leg’s wixels. Sometime the timing between sending the trigger signal and immediately reading the call back were off and the communication became fault.

Solution. The Wixels have a built in ADC RX & TX firmware. This Firmware allowed for sending straight analog values with packets for up to 6 analog pins on the Wixel. These 55-character packets would have a Wixel ID, a timer count since turn on, and 6 analog values in milli-volts!!!

That equates to sensitivity of 0 -3300 for each accelerometer and FRS read. The Receiving Wixel receives these packets through the Serial object in Max, routes them based on Wixel ID and unpacks their values to get the data.

A simple modification of the Perf boards allowed for the solution to be implemented. The ADC communication is much fast then Arduino Serial and has far less complication. Typical reading speed is in 40ms intervals for each packet.

Before:

After:

Project: SoundWaves – Wearable wireless instrument enabling user to synthesis rhythmic sounds through dance.

Milestone 1 Goal:
Decide on Sensors and music playing software.

Sensors:
Decided to use: Conductive Rubber, Accelerometers, and homemade force-sensative resistors.

Video:

Prototyping:

Prototype 1:
Bread Board

Perf Board

Prototype 2:
Used CNC router to cut boards

Fail. CNC router is not precise enough to cut traces of 24 mils with clearance from each other. Easier to make perf boards(I am considering print PCB’s from commercial Board maker.)

Hardware: Mounting upon Shinguards. Force sensitive resistors in Shoes. Dance is more about Lower body movement than upper body movement

Music Playing Software:
Choices
Synthesised Audio: Max_instruments_PeRcolate_06 – traditional instruments artificially reproduced in max, with a level of sensitivity to adjust elements such as pitch, Hardness, Reed stiffness, etc. Denied due to lack of quick adjustablilty to changes in values, still under consideration.

Samplers: Investigated use of free samplers such as Independence and SampleTank, but samplers became to convoluted to setup and the trail for free use expires after 10 days. Denied

Insystem Samplers: Audio DSL synth using midi-out Function in Max. over complicated. Denied.

Resonator/Wikonator: Still under consideration.

Simple Hardcoded Max mapping of gestures to sounds and sound files of 808 drum machine: Accepted.

Wireless:
Wixel are useful, but can only hook up one channel to system, can not read multiple channels on same line, or else getting confusing serial readings.
Bluetooth, Personalize channels but expensive.
Bluetooth and Wixel combination, under consideration.

Dance is an art form, and normally it needs a beat or rhythm to orchestrate the motions and movement. But what if one can create music through dance. Utilizing the articulation and versatility of foot step, Footnotes enable the user to command rhythms with their own step. Simple limit switches at the base of the feet transfer signal via wireless Xbee to music playing software running Maxs. Footnotes is a rough prototype, but a start into something potentially greater.

More…

More…

More…

More…