HELLO WORLD.

You are reading the first milestone of Henry Armero’s so far unnamed project. What an exciting premise! What is it?

Well, currently The Plan is to create some manner of mask, costume, suit, and/or outfit that will have some means of detecting motion of the wearer. It will form an instrument that works as follows!: It encourages stillness, building off the slight involuntary motions of the wearer and the static of the sensors. However, when there is a big movement, like a blink or a twitch or an itch or a whatever, the building sound will get jarred and it will be a thing.

COOL OK SO, how will this be accomplished? Well, I didn’t want to just jump in and start planning around certain aspects in case they turned out to be mega shitty. So I started out by doing a test of the various techs just to see how they rolled. First, how would having a webcam with live processing go?

To decide that, I built a little max patch and played with the jit openCV boxes. And I think they work pretty well! My patch looks kind of like this, if you are curious. You can click for big!

Essentially, what it does is take the input video stream and call this OpenCV flow algorithm on it. That makes this Hue/Saturation image for you! The hue is the direction and the saturation is the magnitude of movement! As you can see in the picture, I gave a dramatic example, sweeping my lion’s mane etc etc etc etc etc. Anyway I throw all that direction data away (for now. I might use it for more fun later when I am making the final software widget thing) and turn it into a lower resolution 0 or 1 binary array thing. Then I add up all the 1s (moving pixels as opposed to stationary ones) and if the change in that total is above a certain threshold it means MORE THINGS ARE MOVING, THAT MEANS SOMETHING HAPPENED! SOUND THE ALARM! and the big yellow bang button turns on.

So what’s the point in that? It means I can do fairly precise inputs for the musical part, triggering samples or actions or whatever. This ‘algorithm’ works rather well, sending a bang whenever there is enough movement. If it was hooked up to a sound, we’d already be half done. I made a gif of it in action. It’s not bad. 🙂

Left is video, middle is the pixels in motion, and the right is whether it is considered a bang or not. It’s pretty satisfactory! VIDEO = GOOD

So next: What about electronic sensors and gyros and accelerometers? I examined these next, and it was pretty tedious and frustrating for a while, as circuitry always is. Mostly because Maxuino doesn’t want to live in my computer I don’t think. Everytime I anything it’s like HANG ON I HAVE TO GO AND MAKE MYSELF A SANDWICH BRB 5 MINUTES and Max completely falls apart. Come on, Max, pull yourself together! But honestly it’s fine once it gets working.

So at first I had some drama with getting sensors to give rational input, but eventually I ended up with a 1-axis gyro which worked pretty well. This was kind of surprising, because I was really looking for a 3-axis accelerometer sort of thing. But those all had some issue or another, so this had to do. And yeah, it works pretty well. I set up a little prototype with the sensor attached to (read: stuffed in) a glove I am wearing and the arduino strapped to my arm with a sweatband. (These super cheap articles of clothing all bought exactly for this purpose! Yay preparedness!) And then stole some of the previous max patch and pasted it in the maxuino helper patch to test if I could get a similar bang-trigger effect like with the video input. And it works pretty well! Look:

You can see a video stream that is completely unrelated to the datastream, and just there for your viewing pleasure. Then there is a kind of scatterplot showing the magnitude of the rolling averaged change of the reading or something like that. And if it’s above some threshold the light goes beep. And there’s some glitch with the GIF where the timing looks like it’s blinking too early? I’m not sure what that’s all about. In max it works fine though. Anyway yeah cool. I have some fun pictures of the setup also:

Don’t forget to click for big!

Anyway yeah it was pretty fun stuff and works pretty well.

SO WHAT IS THE PLAN MOVING FORWARD?

1. Next I am going to plop together an overall plan for the full suit and get started as soon as possible!
2. It will most likely have a big mask!
3. This big mask will contain a camera webcam!
4. It will also contain a light source!
5. The webcam + lightsource will be in charge of the video input from above! It will be radical.
6. The helmet might be a closed thing and it might be not. Either way it will be very frontheavy. This will turn it from an art project into in ENGINARTEERING PROJECT probably. I should not underestimate how long this part will take.
7. Then it will have probably a few sensors along the arms and legs! This will probably require multiplexer shenanigans. Oh the horror!
8. Will I replace the gyro with an accelerometer later? Maybe. The gyro worked pretty well, so it might be a fine idea not to overcomplicate things, as adding an accelerometer might not improve stuff much.
9. HENRY WHEN ARE YOU GOING TO START CONSIDERING THE MEANING OF ALL THIS? etc. My typing that in uppercase makes it seem like I am sick of Ali asking it, but I’m not really. I’m just sleepy and it is something I have failed to mention for this whole post so far. There will definitely be a big statement of WHAT IT ALL MEANS by the time the design document thing is all set, but for now things are kind of variable.
10. To me, the project is representative of those times when you have someone you’re talking to someone on the internet over a period of time or whatever, and you get that insecurity going on and aren’t sure if you should bother them more or less and don’t really know if you’re annoying or not. Sometimes you’re like ok let’s just do this on their schedule, and try to keep quiet (stay still!) but it inevitably falls apart. Everything really tends to fall apart when you’re getting that worked up about that kind of nonsense and don’t know how to deal with it like a sane person.
11. The project represents that aspect of waiting and being inactive and how frustrating it is sometimes! My friend the other day talked to her crush, it was very exciting. She was like hey want to go out for coffee or tea or something sometime? And he was like ok like today? And she was like yesss! And he said I’m busy today! And so she gave him her number. And then TIME PASSES AND HE HASN’T CALLED AND WHAT DO YOU DO? DO YOU FIND HIM AND BOTHER HIM OR MAYBE HE’S JUST TAKING HIS TIME. My friend is symbolically wearing this Stillness Suit. She knows that it’s generally not a good idea to make a move(ment), but maybe the low building sounds are really nerve wracking, or there are people watching, or you have an itch, and then you move and it all breaks down! Or maybe it doesn’t! And then you are back in the same state.
12. It’s kind of about that, if that makes any sense.
13. HENRY WHEN ARE YOU GOING TO START BUILDING THE SYNTH THING? I am putting that off until later? It is currently milestone 4. Like, milestone 4 is just MAKE SYNTH THING. I think that will go well, because I’ve done things of that sort before, and it’ll be ok. I will start if there is any downtime during the next few bits, what with waiting for parts or something.

That’s all for now really. Questions I have are

WHAT DO I DO IF THE MASK IS CLOSED? Do I put a screen somewhere? Do I get a projector? Because the slight movements that trigger the sound need to be highlighted somehow, you know? If I had a screen I could superimpose the blocky movement-map-pixels in wild colors and make an interesting visual. But then it becomes all clunky and stuff. I have to have a surface to project on, and then like cables running out of it and generally it would just be bleeeeh. A screen would be nicer, but how do I do that without the tellytubbies effect?

also uh. WHERE DO COMPUTER GO? I have a hugeass laptop. I guess I could make a backpack structure to put it in, but still that’s a big laptop. What should I do about that?

That’s pretty much all

END STATUS REPORT. GOOD DAY.

Wait one more thing! I did some javascripting in max! And if you need help with javascripting in max I can help! Especially javascripting in max with Jitter matrices! They are hard to find documentation for.