Student Post: BoofCV, Image Segmentation, and 3D Scanner

Greetings,

This past week Dr. Bowers and I met in order to come up with some ideas as to how to move forward with our problem of comparing meshes and/or images for error detection. Essentially, we were able to address some problems we’ve been having with some new ideas:

Image segmentation
Real Time/Video Structure for Motion
BoofCV/3D Stereo Clouds
3D scanner in cooperation with Dr. Dias of the Physics Department here at JMU

Image Segmentation

To start, the goal regarding image segmentation is to cluster pixels into prominent image regions and then use this within a stereo system for object recognition. We would like to be able to take a picture of say a desk without a specific image in it, then take another picture with the specific image inserted and be able to detect it through image segmentation. This would allow us to essentially remove the background while keep the image and its boundaries. (Pictures from Source 1 below)

Real Time Structure for Motion (Simultaneous Localisation and Mapping “SLAM”)

Another idea that I plan on exploring further is real time structure for motion where you can get a live stream from a camera and mesh it in real time to do the comparisons. This would be beneficial for what we are trying to do here since it would be in real time versus quickly meshing and comparing. (Interested? Source 2 & 3 at the bottom has more)

BoofCV (Real Time CV and Robotics)

BoofCV is an open source pure Java library built on top of the OpenCV (Open Source Computer Vision Software). There are 4 packages that BoofCV is organized into: 1) image processing, 2) features, 3) geometric vision, and 4) recognition. The areas of this package that I am interested in are dense optical flow which compares two images to estimate apparent motion between the pixels. This is useful for object detection, tracking, and image segmentation. Additionally, I’m interested in their geometric 3D stereo cloud package and stereo calibration which will come in handy when using stereo cameras. Lastly, the have a Java module written to interact with the Xbox 360 Kinect which would could possibly use as a stereo setup in the meantime to get some quick results. (Interested? Follow the link from Source 4)

Shining 3D Scanner (In part with JMU Physics)

In addition to these other methods for reconstruction and object detection, in the next couple of weeks, I will be working on setting up the Shining 3D Scanner from the Dr. Dias in the Physics Department to see if there is a way to get effective meshes and scans from this for a proof of concept!

Sources:

1. http://www.stat.ucla.edu/~sczhu/papers/reprint_IJCV_parsing.pdf

2. http://www.doc.ic.ac.uk/~ajd/Publications/salas-moreno_etal_cvpr2013.pdf

3. https://users.cs.duke.edu/~tomasi/papers/tomasi/tomasiIser95.pdf

4. http://boofcv.org/index.php?title=Main_Page