What AR libraries would be useful in order to develop an iOS app for showing information about certain objects?

Question

I am starting an AR project for a client which involves using AR in order to show information about certain objects. In this project, for example, the user would point the camera at a car. Depending on which part of the car the user is looking at (headlights, windshield) a button would appear. When the user presses that button, an information window would appear on screen, giving the user more information about that certain car part.

The client doesn't wish to place physical markers on the car (QR code / patterns), and so the car parts would have to be detected another way.

I have developed AR apps before, but based on user location and generated markers in the sky. I feel this system wouldn't be entirely relevant for the client's request.

Would anybody be able to point me in the right direction (iOS library) for this sort of project, and whether or not it would be entirely feasible.

Thanks for the input, Andy.

Answer 1

What you need is a model-based tracker/6DOF object tracker. As you want to track a car, it will certainly be featureless (or you will only get sparse features), so you should look at textureless non planar 3D (object) tracking solutions.

It's pretty much state of the art right now (lot of research, few products/SDK), but using library like OpenCV and with the appropriate literature (see below) you should be able to develop one. You can look at an open-source solution like the ViSP library which has a module for model based tracker but not an official iOS port. for commercial libraries, closest will be AR libraries supporting SLAM or "3D object tracking".

In term of techniques, you have different way to handle this problem, some pointers:

• You can use a model-based tracker relying on edge detection + initial CAD model of the object: 3D Textureless Object Detection and Tracking: An Edge-based Approach or Harald Wuest, Folker Wientapper, Didier Stricker Adaptable Model-based Tracking Using Analysis-by-Synthesis Techniques The 12th International Computer Analysis of Images and Patterns (CAIP), 27-29th August 2007, Vienna, Austria.

• You can use a model-based tracker relying on edge detection + (trained) template images

• You can use some SLAM techniques combined with a model based tracker. M. Tamaazousti, V. Gay-Bellile, S. Naudet Collette, S. Bourgeois, M. Dhome Real-Time Accurate Localization in a Partially Known Environment:Application to Augmented Reality on textureless 3D Objects. TrakMark 2011, Basel, Switzerland 26-29/10/2011

• if your system will only run indoor, you can look at some RGBD tracker S. Hinterstoisser, V. Lepetit, S. Ilic, S. Holzer, G. Bradski, K. Konolige, N. Navab Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes Asian Conference on Computer Vision (ACCV), Korea, Daejeon, November 2012 (access to the software)

Answer 2

It seems you are heading for an interesting topic. However, my concern is the accuracy of what you are trying to do. Location-based AR would be a starting point for your research work. Still the granularity would be less to your problem domain. Since you have worked on the location-based AR application, you might have noticed the accuracy that you can expect would be maximum upto 3 meters. Therefore, that level of accuracy cannot address your problem domain in an advanced way.

However, I have seen prototypes that addresses your problem domain. One good example would be the BMW Augmented Reality Manual. Check this link http://www.youtube.com/watch?v=P9KPJlA5yds

Hence, I never came across a proper Augmented Reality library for iOS or even Android which can address your problem domain in the marker-less AR context.

The information above is only for your knowledge, but not to discourage you in any way.