CODEWITHSUNDEEP
unity-game-enginevirtual-realityoculus
kattastroff
kattastroff

Reputation: 66

Can I use mouse on computer screen in a VR application?

Is it possible to use mouse to click things on the computer screen while on VR?

I'm developing a Unity VR application with Oculus Rift that should be used by two people: one with the HMD and another looking at the computer screen. The person without the HMD should be able to highlight some objects in the scene, and one of the simplest ways I can imagine to do it would be allowing this guy to click the objects in the scene and they will be highlighted.

This seems to be simple because Unity already renders the HMD view to the 2D computer screen. However, the easiest way I found to do it doesn't work with the OVRCameraRig (yes my object has a Collider and yes my Camera has a Physics Raycaster). I've also tried the raycasting method described here.

Upvotes: 3

Views: 1615

Answers (1)

Katianie
Katianie

Reputation: 609

You need to do something similar to Mouse picking with ray casting: http://antongerdelan.net/opengl/raycasting.html

Typically when you want to take a coordinate/vector from local space to model/world space you multiply by coordinate/vector by the model/world space matrix. If you want to go backwards (in your case) you multiply by the inverse matrix.

I used this personally in the past for non VR stuff but it can be easily adapted. I also wrote a quick guide for myself explaining the different spaces:

3D Spaces (Model/World, View/Eye, Projection)

Edit: Since you are already using Unity's built in ray caster, Here is a sample project using UE4 to do raycast based grabbing I made recently. With this approach I also use the UE4 ray caster: http://www.katianie.com/SourceCode/Unreal/VRGrabbing.zip

Upvotes: 2

Related Questions