##PCL Algorithms for Point Clouds and Other 3D Data
Jeff Kramer aka zarvox Session Description October 22, 2011
He's seen the Kinect Fusion paper. He knows more about Fusion than ayone I've talked to thus far.
Lecture
###OpenKinect - PCL Bridge He has his own branch of libfreenect with hooks into PCL? Or just generic lifreenect? https://github.com/Qworg/libfreenect Use unstable branch!
512 x 640 x 480 voxel grid is 450MB per second…
Octrees to the rescue. 8 boxes, which subdivide as necessary. For speed up, average leaves and put them in parent node.
PCL supports octrees, creates them automatically from unstructured data.
Useful for background subtraction. Create octrees for both scene and background, then diff just the octrees with PCL.
###Object Detection People are rectangular prisms. People are also cylinders. Sample consensus fitting (RANSAC) to take foreground model (people) reduce it, RANSAC, then fit a cylinder to that cluster and decide if it's proportionally characteristic to what you're looking for (people, usually).
Get training point cloud. Create feature histogram. Build a tree from the histogram points, use FLAN. Finally, compare. Check chi^2 distance between scene and search feature tree. If distance is below a threshold, call it a match and perform transforms to match our training model to the one in the scene.
What about holes? Stixels, voxels without a y component, just fill space up to a threshol fill regions with no data based on assumptions about the object's bounding shape.
###Greedy Projection Triangulation Given projection of camera and viewpoint… eliminates contained points before translation to mesh, which speeds things up.
Another approach: Sample consensus initial alignment. Alternative to RANSAC.
###Multiple Kinects Get data as normal, then transform it. How to deal withg interference between the IR point clouds.
-
Make sure they don't intersect. Point in different directions.
-
Make sure they're aligned and the object is between them. (Bad for edges.)
-
Physical shuttering. Rotating shutters to make the IR field flash and on and off. Creates blind spots and halves the frame rate, but if timing is correct you get two completely separate point clouds.
-
Electrical shuttering. Modify the power system to the laser to make it cut in and out. He's working on this. He has this working, but it's slow. Matching the timing is the hard part.
-
IR filters? Since the laser outputs a fixed, single wavelength, there's nothing to filter without actually changing the optics and chip to compensate for the wavelength change. Very touchy IR system, peltier cooler, etc. Optics are optimized for the, and given the nature of the point cloud but
-
Polarizing filters? Nope, laser is culmated. What if objects don't keeep the polarization?
###Skeleteonization Two popular methods.
-
Microsoft method. Search and match. Depth image to body parts to 3D joint proposals. Giant giant reference library, that's the hard part. He says we would need tens of thousands of hours of sample data, even if we used Microsoft's SDK to generate the ground trouth training data.
-
OpenNI method. Find joints and track. You have to create right angles between your joints (psi position), creating a shoulder, elbow, and wrist position. From there, overlay skeleton. If you're averagely proportioned, this works well. If you're taller than 6'3" or shorter than 5'4" then you might be out of luck. Where there's a bend, there's a joint. Then split the body into chunks, then watch the chunks. Handle some occlusions through inference.
###Audience Questions
-
Speeding up point clouds Draw the point cloud with a geometry shader. Upload bitmap depth buffer to card and then run the geometry shader to displace it.
-
Concatenating point clouds Create feature points. Take the key points, and see how they align across frames with RANSAC. Prune the mediocre alignment points, then infer transform from relative point alignment between frames or between cameras.
-
Kinect Fusion They have a machine with a bunch of GPUs running it in real time.
-
Noise Reduction Expensive. Design around it.
-
State of ofxPCL? https://github.com/satoruhiga/ofxPCL