[3D] Sky Touch 3D objects with Matlab

So as a viewer for 3D objects I thought that sky-touch would be a nice tool to use. That means that you would be waving in the sky (remember Minority Report) and the 3D object on the screen would move/zoom/rotate accordingly. Later, i would like to incorporate 3D-3D vision. With that, I mean that I would project this on a 3D screen. So then, you can virtually move 3D objects in the space around you, while you are looking at the object in 3D.

There are many obstacles to overcome, because waving in the sky is open to interpretation, so it needs to define commands. But that is for later.

The goal is to be able to move objects that have been photographed from different angles. For now i have chosen to rotate only a model, since i have not yet been provided with enough photographic round-shot material. I guess i would need a hundred or more pictures from different angles in order to produce something useful.

So this is the result of my first test

[youtube=http://www.youtube.com/watch?v=LRnHU1Z8OKU&w=560&h=345]

You have to tune the code a little to make it work:

  1. close all
  2. clear all
  3. clc
  4.  
  5. [x,y]=meshgrid(-2:0.1:2,-2:0.1:2);
  6. %surf(x,y,x.^2-y.^2)
  7. %view(x,20)
  8.  
  9. % Create context and read xml file at first
  10. context = mxNiCreateContext('SamplesConfig.xml');
  11.  
  12. %% Initialise FIGURE
  13. width = 640;
  14. height = 480;
  15. % depth image
  16. %figure(1), h1 = imagesc(zeros(height,width,'uint16'));
  17. %           set(gcf,'MenuBar','none'); set(gca,'Position',[0 0 1 1]);
  18.  
  19. %colormap gray
  20.  
  21. for k=1:1000
  22.  
  23.  
  24.     option.adjust_view_point = false;
  25.     % Acquire rgb and depth image
  26.     tic
  27.  
  28.     [~, depth] = mxNiImage(context, option);
  29.     bw=depth>400 & depth<1100;
  30.     bww=(1100-depth).*uint16(bw);
  31.     bwsq=bww.^2; %verhoogt beeld
  32.  
  33.     [YCoord, XCoord] = find(bwsq);
  34.     CentroidX = 500-mean(XCoord);
  35.     %CentroidY = 500-mean(YCoord);
  36.     CentroidY = mean(YCoord);
  37.  
  38.  
  39.     surf(x,y,x.^2-y.^2)
  40.     if isnan(CentroidX) == 0
  41.         view(CentroidX,CentroidY)
  42.     end
  43.  
  44.     axis([-2 2 -2 2 -4 4]); axis equal;axis square
  45.  
  46.     %plot(CentroidX,CentroidY,'ko','MarkerSize',15)
  47.     %axis([ 0 500 0 500])
  48.     % Update figure
  49.     %set(h1,'CData',bww);
  50.  
  51.     drawnow;
  52.     %disp(['itr=' sprintf('%d',k) , ' : FPS=' sprintf('%f',1/toc)]);
  53.     disp(['fps=' sprintf('%f',1/toc)]);
  54.  
  55. end
  56.  
  57. % Delete the context object
  58. mxNiDeleteContext(context);

Tim Zaman

MSc Biorobotics. Specialization in computer vision and deep learning. Works at NVIDIA.

You may also like...

4 Responses

  1. ushie says:

    plz give me the code.i dont knw how to fix this

  2. Rami says:

    How can I run this code? where XML files and another files needed?

  3. Carsten Fries says:

    Hi Tim,
    when I try your code, the 3d model does not rotate. Can you help me with that, please?
    Best regards,
    Carsten