Multi-touch build blog

So, today was a bit of a sad day, the table that I have obsessed over for more than a year now had to be dismantled to make room for some boxes (in preparation for moving home). This has left me able to concentrate on some other web stuff over the next couple of months but has also left me in anticipation of how it will re-materialise. I have always fancied making this thing vertical, so I think I will be giving it due consideration, watch this space!

Okay, this demo video has a few bugs but I couldn’t wait to post it up. I wanted to create a nice way of selecting objects in a 3D space regardless of the viewing angle etc. I went with a four fingered selection marquee type gesture. In the mix is a whole bunch of other stuff I have been adding in to a showcase style application too.

See below.

Selecting 3D objects using a marquee selection tool from Dan Higham on Vimeo.

Taking that last video a little bit further. I saw a jigsaw puzzle on the MS Surface table and wondered how hard it would be to break up that video object and make a kind of puzzle out of it.

Of course the hardest bit was to get a routine that could break up a texture in to even sized chunks at a reasonably quick pace. After messing with solutions in C# and even some unmanaged C++ (not successful, yet!) here is the result…

Since starting my XNA-based framework I have always been keen to get video mapped on to a quad and move it around in 3D space. Being left a little unimpressed with the built in support for video in Game Studio and the “Video on a Quad” demo I decided to go for a mixed approach of using DirectShow and a proper 3D model.

There is a little lag each time a new gesture is picked up by the system, need to work on that!

So as I said in my last post, I have been working on an framework in XNA. To be honest its been going pretty well, the only major problem I have come across was getting the picking in 2D to work with objects at multiple depths in 3D. Anyway, see what you think, the world rotation gesture is a bit broken but I wanted to get a video up…

I also beefed up my camera mount so I could get a better plan view of the table…

After a blogging sabbatical I have returned. Having got my WPF framework to a reasonable state I have returned back to XNA with renewed vision for a 3d framework, this is the platform I had originaly planned to build.

XNA Game Studio 3.1 brings among other things, support for displaying videos on quads. So, my first objective, rebuild pretty much everything I have done in WPF in XNA but in the 3D space paradigm. So far I have a rough demo that allows the user to pan around and zoom in and out (using pinching). I also now able to do “triangle based picking” (http://creators.xna.com/en-US/sample/pickingtriangle) which of course involves projecting a ray from a 2d point into 3d and seeing what intersects that ray. I have to say the community set up around XNA is first class and has provided some awesome tutorials on all of this.

Once I have something substantial and a bit more tactile there will be videos, I also need to finish setting up a ceiling mount for my camera above the table!

So, milestones looking something like;

- Basic environment, zooming, panning, resizing + rotating objects.
- Advanced interactivity – moving objects on the z-axis using pressure / blob size
- Physics library integration, suggestions of a good one gratefully received!

First release of the test tool is out! Still some bugs but pretty stable.

http://multipointtuio.iamhigham.com

First version of multipointtuio.iamhigham.com is up, no content yet.. all will be revealed in a week (or so). This is to promote use of my multi-touch test tool, just finishing the app off, trying to build in support for TCP/IP at the moment

Spending some time getting my multipoint testing tool up to scratch for a public release. Watch this space people!

I know, I know, I have done a similar demo before with XNA, but having moved to WPF I wanted to introduce Farseer back in to my new framework…