CODEWITHSUNDEEP
augmented-realityarucowebxr
user411103
user411103

Reputation:

Detecting real objects in a room from an AR app (WebXR)

Let's imagine a real painting gallery, where visitors can use their phone browser near the paintings in order to see (via AR) some information about that painting.

How can I do that with WebXR? Probably hardcoding the GPS location of each painting won't be accurate enough, especially indoors. Would OpenCV arUco markers printed and sticked by each painting, and detecting them via WebXR app, be the best solution? They don't seem to work well if you are further than 1m from the marker.

Upvotes: 1

Views: 962

Answers (2)

klausw
klausw

Reputation: 356

The initial version of the WebXR AR module doesn't provide camera images to the application, but that's planned as a followup. Please see the community group's proposal in https://github.com/immersive-web/computer-vision for more information.

For comparison, ARCore supports image recognition, and this would be a useful add-on feature for WebXR, either as part of the computer vision proposal or as a separate feature.

Upvotes: 2

Jclangst
Jclangst

Reputation: 1342

That sounds like an interesting application. Computer vision techniques are usually pretty good at categorizing objects (given a proper training set) but are somewhat poor at detecting and differentiating between individual members of a category (which is what you are proposing -- identifying individual paintings from a set of all of the paintings in the category).

The easiest solution is to go for something like arUco -- this is a perfect use case for that.

If you wanted something more general, you'd need to build a training set (very difficult if your paintings in question aren't already famous / well-documented) and train a model for deployment on edge devices like phones. This is easily 10-100x the work of the above (and you will likely have worse results), so unless you are trying to showcase the cutting edge of computer vision and have computer vision experts on hand, I wouldn't go down this route.

Edit:

I see your updated question explaining that the markers don't work very well. While I don't know how deep you are into this yet, don't be discouraged if it doesn't work perfectly out of the box. Computer vision is one of those things that requires a lot of tinkering with parameters and it can often seem like black magic when small changes to your parameter tuning can dramatically affect performance. I wish there was more that I could share, but when it comes to computer vision, there are no generic tricks that will get you great results all the time.

Upvotes: 1

Related Questions