CODEWITHSUNDEEP
unity-game-engine3d-reconstructionmetaio
user3542126
user3542126

Reputation: 21

Is there a Unity plug-in that would allow you to generate a 3d model using your webcam?

I've looked into Metaio which can do Facial 3d reconstructions

video here: https://www.youtube.com/watch?v=Gq_YwW4KSjU

but I'm not looking to do that. I want to simply be able to have the user scan in a small simple object and a 3d model be created from that. I don't need it textured or anything. As far as I can tell Metaio cannot do what I'm looking for, or at least I can't find the documentation for it.

Upvotes: 1

Views: 2236

Answers (1)

Liam McInroy
Liam McInroy

Reputation: 4366

Since you are targeting mobile, you would have to take multiple pictures from different angles and use an approach used in this CSAIL paper.

Steps

  1. For finding the keypoints, I would use FAST, or a method using the Laplacian of Gaussian. Other options include SURF and SIFT.
  2. Once you identify the points, use triangulation to find where the points will be in 3D.
  3. With all of the points, create a point cloud. In unity, I would recommend doing something similar to this project, which used particle systems as the points.
  4. You now have a 3d reconstruction of the object!

Now, in implementing each of these steps, you could reinvent the wheel, or use C++ native plugins in Unity. This enables you to use OpenCV which has many of these operations already implemented (SURF, SIFT, possibly even some 3D reconstruction classes/methods, which use Stereo Calibration*).

That all being said... the Android Computer Vision Plugin(also apparently called "Starry Night") seems to have these capabilities. However, in version 1.0, only PrimeSense sensors are supported. See the description of the plugin**

Starry Night is an easy to use Unity plugin that provides high-level 3D computer vision processing functions that allow applications to interact with the real world. Version 1.0 provides SLAM (Simultaneous Localization and Mapping) functions which can be used for 3D reconstruction, augmented reality, robot controls, and many other applications. Starry Night will interface to any type of 3D sensor or stereo camera. However, version 1.0 will interface only to a PrimeSense Carmine sensor.

*Note: That tutorial is in matlab, but I think the overview section gives a good understanding of stereo calibration

**as of May 12th, 2014

Upvotes: 2

Related Questions