how to setup a 360 video projection in as3/away3d that will keep overlays straight

Question

I am trying to build a 360 degree video player that will act more or less like Youtube's desktop 360 player. I am not a 3D guy, but I have managed to build something with code I found on the web. I am using Away3D 2.5.2 for this.

The problem I am having is that my client has some graphic overlays pasted on the video. These overlays must remain straight, at least on the vertical side. While Youtube's player successfully keeps these straight - see here: https://www.youtube.com/watch?v=59MVhZWlKgQ - my own player curves the vertical lines a bit - see here: http://www.ysection.com/saar/360a/

The video itself is here: http://www.ysection.com/saar/360a/360test2-injected.mp4

*this video is just an example for this question only. I found it on YT and I have no rights to it

As you can see in these screengrabs, Youtube's player has some compensating "fisheye" distortion applied to the video (see around the edges) that helps keep the lines straight.

Both players play the same video which is a 4096x2048 equirectangular video.

Below is main code of concern to the question:

This is how I set up the camera:

camera = new HoverCamera3D();
camera.steps = 1;
camera.zoom = 15;
camera.focus = 25;
camera.minTiltAngle = -90;
camera.maxTiltAngle = 90;
camera.panAngle = 270;
camera.tiltAngle = 0;

This is the material setup:

video = new Video(2880,1440);
vBitmap = new BitmapData(2880,1440);
skyMaterial = new BitmapMaterial(vBitmap);
skyMaterial.smooth = true;

This is the 3D object setup:

skysphere = new Sphere();
skysphere.material = skyMaterial;
skysphere.radius = 2800;
skysphere.rotationX = 180;
skysphere.segmentsW = 40;
skysphere.segmentsH = 36;
skysphere.scale(-1);
scene.addChild(skysphere);

and this is what I have on enter frame:

vBitmap.draw(video,null,null,null,null,true);
if (move) {
    camera.panAngle = 0.3 * (lastMouseX - this.mouseX) + lastPanAngle;
    camera.tiltAngle = 0.3 * (lastMouseY - this.mouseY) + lastTiltAngle;
}
// hack
camera.panAngle = camera.panAngle+0.001-0.002*int(hackSwitch);
hackSwitch = !hackSwitch;
camera.hover();  
view.render();

Do you guys have any advice for me on how to make my player act like Youtube's?

thanks, Sa'ar

Answer 1

I would need to look at this in detail later, but at first view:

It appears YouTube is mapping the video to the inside of a capped cylinder (i.e. the camera is anchored to 0,0,0 inside of a 'capsule'). If you pan up/down with the overlays towards the edge the screen, the overlay vertical lines remain straight, tilted on the pan amount of course, but straight and not concave. Since you are allowed to pan up/down by 180 degrees, if you pan down at the conductor's feet, you will see the lines converge as if mapped to a sphere, but in 0 degrees pan, you are looking at the texture mapped to the vertical 'walls' of the cylinder thus no distortion on your overlay's vertical lines.

Instead of 3d mapping to a sphere primitive in Away3d, try using a away3d.primitives.CapsuleGeometry instead as a quick hack.

NOTE: Google could be really doing a true "Rectilinear Len" via a customer webgl shader, or maybe just mapping to a rectilinear grid (again mapped in the shape of a capsule?) but having non-uniform distances between each horizontal segment to provide that rectilinear look and feel, but I have not looked at the js code.

NOTE: A quick test for the YouTube player, map some square overlays near the floor and ceiling, and play it and pan up or down to see where the distortion begin. BETTER yet, just overlay grid lines (i.e. like a Brick wall) on the video that cover the entire vertical height and the entire horizontal width of the video and you will be able to 'map' the primitive object type that they are using by panning up and down when one of the lines is rotated at 0, 45 & 90 degrees off of center (again, I assume it will show a cylinder with spherical caps or a rectilinear cube or a sphere with a gnomonic/equalitorial based vertex geometry).