Now that I’ve successfully managed to make a program that connects to the Kinect in Objective-C and makes it move up and down and all that, the next challenge is to get it to display some RGB data using OpenGL.
So, firstly I will see how they’ve done it in the libfreenect example…
Wow, it really seems like this getting of RGB data is not going to be as simple as tilting the Kinect. The libfreenect API uses callbacks to give the program new data as it is received from the cameras. Also as I am trying to mix C and Objective-C it is proving a lot more difficult to get working properly. I have spent a whole day trying to work out how to duplicate the process in Objective-C, how to get the OpenGL view working properly and how to get the callbacks working properly, but still I haven’t got it working yet.
I had a chill and stuff and then thought that the best way to do it is probably to use C based threads, as used in the libfreenect example, as well as Objective-C threads to monitor the data within the C threads.
Hopefully this will all work out for me because I have got very used to using Objective-C, Xcode and the Interface Builder. Errgh, I kinda sound emo there. best stop this now…!
So, the mission is, get some pthreads working in my code as done in the libfreenect example and power it out and get it working just as I wanted 😉 and FOR SCIENCE!
.. Christmas 🙂 …
I was doing a bit of research into OpenGL and libfreenect and then discovered (again) that there were wrappers that I could maybe use in my code. There is a C-sync wrapper that could possibly enable me to read input RGB and Depth data when I am ready for it instead of using callbacks. This could result in the program not being as efficient as some frames may be lost, or the processor could be trying to read frames when there are no new ones. But I’ll give it a look and see what happens. Hopefully it will help me out a lot 🙂
So, I have successfully managed to get it to let me control the tilt of the Kinect device and also retrieve the state of the tilt every second using another thread. Looking good so far 🙂 now for video.
Now I have implemented a way for the RGB data to be collected using another thread it is able to call the drawRect: method when there is data available. And so, it works! I have my program displaying the RGB data at intervals of 0.2 seconds. This gives a video image that is a little jumpy as I would expect, so now I will decrease the sleep time between getting new RGB data and see what happens.
I have set the sleep time between RGB data calls to 1 millisecond and it works very well 🙂 the image is very smooth and my program doesn’t seem to report that there is no new frames (this may be to the way libfreenect_sync.h works as it states that data is kept in place until a new frame is available…i think. I should probably check this).
So now the next step it to try and also get some depth data displayed and at the same time work out how I can determine the depth of individual pixels as I want.