CODEWITHSUNDEEP
uwphololenseye-tracking
chronosynclastic
chronosynclastic

Reputation: 1675

Eye gaze from SpatialPointerPose while using Holographic Remoting

I'm getting the data from HoloLens 2 eye tracker using the SpatialPointerPose API like this (safety checks not given for brevity):

currentSpatialCoordinateSystem = PerceptionInterop.GetSceneCoordinateSystem(UnityEngine.Pose.identity) as SpatialCoordinateSystem;
SpatialPointerPose pointerPose = SpatialPointerPose.TryGetAtTimestamp(currentSpatialCoordinateSystem, PerceptionTimestampHelper.FromHistoricalTargetTime(DateTimeOffset.Now));
var eyes = pointerPose.Eyes;
GazeOrigin = eyes.Gaze.Value.Origin.ToUnityVector3(),
GazeDirection = eyes.Gaze.Value.Direction.ToUnityVector3()

This seems to also work while using Holographic Remoting. However, I have some doubts about the timing/synchronization of the gaze data in this case because there will be some delay between the remote server and the HoloLens? Specifically, will TryGetAtTimestamp give me the correct timing here, considering that the code runs on the remote machine (either as a standalone app or using the remoting from play mode in Unity editor) ?

Edit: The above code only works while remoting in Unity play mode

Upvotes: 0

Views: 245

Answers (1)

Franklin Chen - MSFT
Franklin Chen - MSFT

Reputation: 4923

Holographic Remoting uses clock synchronization which means that timing is correct. The result and accuracy of SpatialPointerPose::TryGetAtTimestamp depends on the given timestamp. For instance, if the given timestamp is older than the last received pose data on the remote side, then the returned SpatialPointerPose will be an exact interpolation of received pose data values. If the given timestamp is newer than the last received pose data on the remote side, then Holographic Remoting predicts the SpatialPointerPose based on the latest pose data available.

Upvotes: 1

Related Questions