‘SketchSynth’ by Billy Keyes
SketchSynth: A Drawable OSC Control Surface
SketchSynth lets anyone create their own control panels with just a marker and a piece of paper. Once drawn, the controller sends Open Sound Control (OSC) messages to anything that can receive them; in this case, a simple synthesizer running in Pure Data. It’s a fun toy that also demonstrates the possibilities of adding digital interaction to sketched or otherwise non-digital interfaces.
Audio Graffiti and Music in Motion: Location Based + Spatial Sound
Some impressive spatial audio examples/works by Zack Settle and company. See Zack’s page for more…
Group Project: Wireless Data + Wireless Video System (part 2)
Overview
This project combines a Wixel wireless data system, servos, microcontrollers and wireless analog video in a small, custom-built box to provide wireless video with remote viewfinding control.
Hardware
Camera-Box:
- Wixel wireless module
- Teensey 2.0 (code found HERE)
- Wireless video transmitter
- 3.3v servo (2x)
- FatShark analog video camera
- 12v NiMH battery
- 9v battery
- 3.7v LiPo battery
- Adafruit LiPo USB charger
Control Side:
- Alpha wireless video receiver
- Analog to Digital video converter (ImagingSource DFG firewire module)
- Wixel Wireless unit
- Max/MSP (patch found HERE)
System Diagram:
Tips and Gotchas:
1. Max/MSP Patch Setup:
- Connect the your preferred video ADC to your computer
- Open the patch
- hit the “getvdevlist” message box , select your ADC in the drop-down menu
- hit the “getinputlist” message box, select the correct input option (if there are multiple on your unit)
- if you see ““NO SIGNALS” in the max patch:
- double check the cables… this is a problem with older analog video
- verify that the camera and wireless transmitter are powered at the correct voltage
2. Power Choices:
- We ended up using three power sources within the box. This isn’t ideal, but we found that power requirement for the major components (teensey, wixel, transmitter, camera) are somewhat particular. Also keep in mind that the video transmitter is the largest power consumer at around 300mA.
Applications:
1. Face Detection and 2. Blob Tracking
Using the cv.jit suite of objects, we built a patch that pulls in the wireless video feed from the box and uses openCV’s face detection capabilities to identify people’s faces. The same patch also uses openCV’s background removal and blob tracking functions to follow blob movement in the video feed.
Future projects can use this capability to send movement data to the camera servos once a face was detected, either to center the person’s face in the frame, or to look away as if it were shy.
We can also use the blob tracking data to adjust playback speed or signal processing parameters for the delayed video installation mentioned in the first part of this project.
3. Handheld Control
In an effort to increase the mobility and potential covertness of the project, we also developed a handheld control device that could fit in a user’s pocket. The device uses the same Wixel technology as the computer-based controls, but is battery operated and contains its own microcontroller.
Group Project: Multi-Channel Sound System (part 1)
Introduction:
The multi-channel sound system group is implementing a spatial instrument that allows us to interactively experience sound in space. The system includes software that controls a set of eight (currently) speakers that are positioned in space. The experiences will depend on the specific hardware setup and mechanics, which are still in the works. Our group has listed five possible hardware/software setups, and we are brainstorming the many experiences we can create with each setup. These ideas are listed in the section “Categories”.
Categories:
Each section corresponds to a specific hardware/software setup. For each, we include a few ideas that we have come up with so far.
- A mobile disk that can be worn (as a hat, etc) – “Ambiance Capture Headset/Scenes from a Memory”
Every day, we move about from place-to-place to spend our time as driven by our motivations. Home, Road/Car, Office/School, Library, Park, Cafeteria, Bar, Nightclub, Friend’s place, Quite Night- we all experience a different ambiance around us and a change of environment is usually a good thing. It may soothe us, or trigger a certain personal mode we have (like a work mode, a social mode or a party mode). What if we could capture this ambiance, in a ‘personalized’ way and create this around us when we want- introducing the Ambiance Capture Headset. This headset has a microphone array around it and it records all the audio around you- it may catalog this audio using GPS data. You come home, connect the headset to your laptop & an 8-speaker circle and after processing audio (extracting ambiance only, using differences in amplitude and correlation in time etc.), the system lets you choose the ambiance you’d want. You can quickly recall your day by sweeping through and re-experiencing where you’ve been.
- Head-sized disk with speakers positioned evenly around the disk. Facing inward – “Circle of Confusion”
In this section, our ideas tend to fall in two categories, either using the setup to confuse a listener’s perception of the world around them, or to enhance it in some way. In the first scenario, one idea is to amplify sounds that are occurring at 180 degrees from the speaker, in other words, experiencing a sonic environment that is essentially reversed from reality.
- Speakers hanging from the ceiling in arbitrary shapes – “EARS”
Sometimes you just need someone to listen to you, like a few ears to hear you out maybe? A secret, a desire, an idea, a confession. This is a setup that connects with people, and let’s them express what they want. It’s a room you walk into which has speakers suspended from the ceiling. You raise your hand towards one, and when that speaker senses you coming near, it descends to your mouth level so you may talk/whisper into it (speakers can act as microphones as well! or we may attach a mic to each). You may tell different things to the different speakers, and once you’ve said all you want, you hear what the speakers have heard before. This is chosen by the current position of speakers, as all speakers start to descend if you try to touch them. All voices are coded, like through a vocoder to protect identities of people. Hearing some more wishes, problems, inspirations, hopes you probably feel lighter than you did before.
- A 3D setup (perhaps a globe-shaped setup) – “World Cut”
You enter into an 8 speaker circle, having a globe in front. You spin the globe and input a particular planar intersection of the world- this planar intersection ‘cuts’/intersects a number of countries/locations. These intersected locations map to a corresponding direction in our circle, so you hear music/voices/languages from the whole ‘cut’ at a go in out 8 speaker circle. You can spin the globe and explore the world in the most peculiar of ways.
Current system setup
The diagram shows the current hardware setup. As we explain in the section below, it is subject to minor modifications.
Implementing a STABLE and ROBUST system for practical use:
The current hardware implementation is not practical, yet. The wiring obstructs the experience and the system itself is difficult to transport. The items in the section called “Categories” describes completely new hardware setups that will fix this problem. Shielded speaker wires or PCB boards may be part of a more stable. Making the project more robust could include making acrylic enclosures for the speakers. Finally, we’re looking into using larger speakers to improve the experience.
Initial Group Members:
Sean Lee
David E. Lu
Jake Jae Wook Lee
M Haris Usmani
Current Group Members:
M Haris Usmani (Persistent Member)
Robert Kotcher
Haochuan Liu
Liang He
Meng Shi
Wanfang Diao
Jake Berntsen
Assignment 2 “Musical Painting” by Wanfang & Meng(2013)
What：
This idea came from the translation between music and painting. When somebody draws some picture, the music will change at the same time. So, it looks like you draw some music:)
How:
We use sensor to test the light, get the analog input , then transform it to sound. I think the musical painting is a translation from visible to invisible, from seeing to listening.
Why:
It is fun to break the rule between different sense. To on the question, the differences between noise, sound and music, personally thinking, that the sound is normal listening for people. The noise may be disturb people. The music is a sound beyond people’s expect. So based on the environment, people’s idea will also change, we will create more suitable music .