Well, well, well….
All my talk of OpenNI, C++, NodeJS, etc in recent months was pretty much all boring until you put it into practice and make something cool.
I present to you….the “Upright Spass”:
I’ve played around several months ago with the Kinect SDK playing a keyboard in thin air. What I was playing with then was Windows only, Kinect only, and need Adobe AIR to route things to websockets for the browser.
So using my new found powers over the past few months with:
- My Asus Xtion Pro Live depth camera
- C++ Addons in NodeJS
….I now have a nice little handtracking utility that runs in Node.js using OpenNI and NiTE to power my skeleton tracking.
I didn’t care for the horizontal layout of my old virtual piano – so I inverted the axis, and made the instrument control upright. Hence – “Upright Spass”….the anti-bass, the bass that is not a bass, just empty space.
And wow….the site I grabbed this image from is calling this keyboard (made in 1993) “vintage”. I feel old, damn.
Anyway – I’m running Ubuntu for this whole operation, so to route the MIDI from Node.js to my keyboard, I used Jack. Jack offers you a nice little audio server. You can patch in your MIDI through out to the E-MU MIDI USB device in. Voila, start make the link and start the Jack server.
So, I got this motion controlled midi thing all rigged up, and it’s REALLY hard to play. There were a few problems:
- Playing straight notes with 2 hands in an unfamiliar environment can lead to disharmony. Seriously, on top of being hard to play, it’s way too easy to play the wrong notes. So, I restricted the instrument space to only be able to play notes in a certain key signature. I randomly chose A# Minor.
- The coordinates of your 3D world will vary based on where you stand and where the camera is positioned. So, on top of sending the hand coordinates from my Node.js AddOn, I also sent the torso position. That way, all the hand positions can be calculated outward from the center of your body – and your vertical instrument is always in your center. Muscle memory is a major factor in learning to play an instrument, and you can’t learn to play if your instrument keeps shifting around on you. Ideally, I should get the user’s height and make calculations on where the instrument notes are from there as well, but I haven’t done so yet.
Even after solving a few of these problems, the Upright Spass is really hard to play. My performance was pretty much a disaster – but maybe I can tweak and practice and get passable at it.
My code for this is up on github. I mentioned the link for my Node.js AddOn previously – that’s here:
and this particular project, the Upright Spass, is here: