CODEWITHSUNDEEP
iosiphonexcodeunity-game-enginearkit
SolidSnake4444
SolidSnake4444

Reputation: 3587

ARKit and Unity - How can I detect the act of hitting the AR object by a real world object from the camera?

Think if someone in real life waved their hand and hit the 3D object in AR, how would I detect that? I basically want to know when something crosses over the AR object so I can know that something "hit" it and react.

Another example would be to place a virtual bottle on the table and then wave your hand in the air where the bottle is and then it gets knocked over.

Can this be done? If so how? I would prefer unity help but if this can only be done via Xcode and ARKit natively, I would be open to that as well.

Upvotes: 2

Views: 964

Answers (2)

Adam Roszyk
Adam Roszyk

Reputation: 409

You should take a look at ManoMotion: https://www.manomotion.com/ They're working on this issue and suppose to release a solution in form of library soon.

Upvotes: 0

rickster
rickster

Reputation: 126137

ARKit does solve a ton of issues with AR and make them a breeze to work with. Your issue just isn't one of them.

As @Draco18s notes (and emphasizes well with the xkcd link 👍), you've perhaps unwittingly stepped into the domain of hairy computer vision problems. You have some building blocks to work with, though: ARKit provides pixel buffers for each video frame, and the projection matrix needed for you to work out what portion of the 2D image is overlaid by your virtual water bottle.

Deciding when to knock over the water bottle is then a problem of analyzing frame-to-frame differences over time in that region of the image. (And tracking that region's movement relative to the whole camera image, since the user probably isn't holding the device perfectly still.) The amount of of analysis required varies depending on the sophistication of effect you want... a simple pixel diff might work (for some value of "work"), or there might be existing machine learning models that you could put together with Vision and Core ML...

Upvotes: 1

Related Questions