CODEWITHSUNDEEP
iosarkit
john doe
john doe

Reputation: 9660

Placing Virtual Object Behind the Real World Object

In ARKit for iOS. If you display a virtual item then it always comes before any real item. This means if I stand in front of the virtual item then I would still see the virtual item. How can I fix this scenario?

enter image description here

The bottle should be visible but it is cutting off.

Upvotes: 3

Views: 1119

Answers (4)

Nishant Minerva
Nishant Minerva

Reputation: 114

This is called occlusion and with ARKit 3.5 it's available.

You can achieve the results with the People Occlusion and Object Occlusion in ARKit 3.5 and above.

Upvotes: 1

Guig
Guig

Reputation: 10189

You cannot achieve this with ARkit only. It offers no off the shelve solution for solving occlusion, which is a hard problem.

Ideally you'd know the depth of each pixel projected on the camera, and would use that to determine those that are in front and those that are behind. I would not try something with the feature points ARKit is exposing since 1) their position is innacurate 2) there's no way to know between two frames which feature point of frame A is which feature point in frame B. It's way to noisy data to do anything good.

You might be able to achieve something with third party options that'd process the captured image and understand depth or different depth levels in the scene, but I don't know any good solution. There's some SLAM technique that yields dense depth map like DTAM (https://www.kudan.eu/kudan-news/different-types-visual-slam-systems/) but that'd be redoing most of what arkit is doing. There might be other approaches that I'm not aware off. Apps like snap do this in their own way so it is possible!

Upvotes: 5

Pochi
Pochi

Reputation: 13459

What you are trying to achieve is not easy.

You need to detect the parts of the real world that "should be visible" using some kind of image processing. Or maybe the ARKit feature points that have the depth information, then based on this you have to add "an invisible virtual object" that cuts the drawing of things behind it. This will represent your "real object" inside the "virtual world" so that the background (camera feed) remains visible in places where this invisible virtual object is present.

Upvotes: 1

Hung Ya Tsai
Hung Ya Tsai

Reputation: 54

So basically your question is to mapping the coordinate of the virtual item on real world coordinate system, in short, you want to see the virtual item blocked by the real item, and you can only see the virtual item once you pass the real item.

If so, you need to know the physical relations of each object in this environment, and then you need to know exactly where you are to decide if the virtual item is blocked.

It's not an intuitive way to fix this, however, it's the only way I can think of.

Cheers.

Upvotes: 1

Related Questions