[Kinect] use with Matlab
I have recently purchased a Kinect camera (although I do not even own an X-Box). It’s main purpose is to get the multi-touch working on it. Well, multi-touch: multi-sky-touch that is. For this i would be programming in C, but for now it is easier to develop in Matlab, so let’s focus on Matlab.
Kinect Hardware connections
The Kinect is provided with a normal USB connector and split to a normal power adapter. It gets quite hot after a while.
There are three options to get the driver working on your Windows machine (and one for your OS X beast):
- CLNUI untested
If we are going to use Open-Kinect with Matlab, we will be using an external DLL written for Matlab after the Kinect is working. For this we will use the Image Acquisition Toolbox (IMAQ). I had a lot of trouble with this. Though, this seemed like the good option since the Image Acquisition-toolbox is made especially for these kinds of streaming video data, for which you would otherwise need buggy loops. So after some problems with IMAQ i switched to the other driver. Also, you need Visual Studio for this approach.
Open-NI and NITE
This worked great for me, although this is not entirely open-source, but i was not going to touch the source-code much anyway. So, lets get this working: Do this:
- Download Kinect Drivers (LINK) and press the DOWNLOAD button. Search for the file dpinst-x86.exe and run it.
- Download the unstable binary which worked for me (LINK) otherwise try the unstable binary. Scroll down and press LATEST UNSTABLE and download.
- Download the unstable NITE binary (LINK), same fashion as (2), the install key is 0KOIk2JeIBYClPWVnMoRKn5cdY4=
- Download the unstable binary from (LINK) and install.
- Plug in the Kinect. If you go to the device manager it should say Xbox NUI Audio unknown, this is obvious, and under PrimeSensor you’ve got your Cam and Motor
- For this step check http://www.studentguru.gr/blogs/vangos/archive/2011/01/20/how-to-successfully-install-kinect-windows-openni-nite.aspx
- Run a test to make sure it works from C:Program FilesOpenNISamplesBinRelease and C:Program FilesPrime SenseNITESamplesBinRelease. If they do not work, do not proceed. Switch binaries; stable or unstable this time.
- Get http://sourceforge.net/projects/kinect-mex/ , press DOWNLOAD and unpack: then get to Matlab and run one of the test .m files. You can only execute the files that have _sample in them, the rest is compiled stuff you should not touch. Also, don’t touch the .XML files. Now, hacking begins!
- If you have any problems, drop a line in the comments.
- Streaming RGB ánd depth image: 20 FPS
- Streaming depth image only: 30~35 FPS
- Streaming without viewing: 35~50 FPS
Here a sample video in which i 3D reconstruct my desk-area.
update The Kinect also records audio ofcourse. There are, as far as i know, no drivers yet to get audio from the Kinect. This holds at least true for Matlab, and i’m not sure if they’ve managed to import audio to windows anyway.
update2 This Kinect method does not work for Matlab 2008. Try upgrading your version.