Construct 3D geometry from an image in android application

Question

We have got a requirement feasibility study to develop smart phone/tablet application preferably an android application which should be able to do the following

• At first user will walk with tablet or smart phone with camera on, inside a building to map the whole building or room.

• The application should be able to recognize the edges of the room and make a 3d structure out of it.

• Later user should be able to see a 3D layout created out of it. The 3D layout can be exported into standard file format such as igs, dwg, stp parasolid etc.

• The user will then make changes to the 3D layout like adding a window or marking certain place with some additional note etc. This new diagram will be imported back in the smart phone application.

• User will again walk through the building or room with the new imported layout (new proposed window or marking tells about toilet etc)super imposed. The layout should be updated as the user walks/turns.

I am exploring "Edge detection" algorithm for 2D box, I would like to know how to calculate depth and detect orientation and also more information regarding has anybody worked on such project.

I prefer more real time super imposing just like the Sudoku puzzle solved in Matlab. than the Sudoku puzzle solved by Google

Since this was feasibility study analysis, I thought of posting in so before digging deep in various aspect of this. Is this feasible? What are things I need to take care, which all applications I will be needing etc.

Also please point me to right place if this is not the right place to ask the question

Answer 1

Normally the stuff you're talking about is done with devices that comes with an inbuilt depth sensor like Microsoft Kinect Sensor. Kinect Fusion is a software that takes this depth input and converts it into a complete 3D model.

Other sensors options include LIDAR, which I believe is a high-end laser scanner for collecting depth data.

W.r.to what you're asking - rebuilding a 3d model from videos recorded on a mobile device camera - this is very much a research problem. I don't believe there're any ready-made software tools out there which does this out of the box. Disney recently released a research paper which does something similar to what you're asking.

I haven't really read the paper, so I don't know if there're any pre-conditions - in terms of camera abilities, light conditions, distance of measurement etc. But this is some place you can start and may be see their references for additional resources.

IMHO, I believe you'll can make tool if you do use a depth sensor than just a video of the model you're trying to record. The latter can be very error prone and needs a lot more research.

Good luck.