Discontinuous Normals on a Normalized Cube Mesh with Displacement Texture

Question

In my project, I have created a normalized cube using six plane meshes. To improve its visual quality, I added a displacement texture and calculated the normals for that texture to ensure proper lighting. However, I’m facing an issue where the resulting normals appear discontinuous at the edges, revealing the underlying cube structure rather than achieving a smooth, sphere-like shape. you can see here: example

Here is my shader code:

function vertexShader() {
  return `
    varying vec2 vUv;
    uniform sampler2D u_tex;
    uniform vec3 wTl;
    uniform vec3 ps;
    
 //normalize each plane   
 vec3 planeToSphere(vec3 p, vec3 localCenter){
    return 100.0*normalize(p-localCenter) + localCenter;
  }
  
varying vec3 v3n;
    void main() {
      vUv = uv;
      vec3 newPosition   =  planeToSphere(position,wTl) ;
      vec3 worldp = ( modelMatrix * vec4(newPosition,1.0)).xyz;
      float n = texture2D(u_tex, vUv).x;
      //newPosition = newPosition + 1.0*n*normalize(position-wTl); // this add displacement
      gl_Position = projectionMatrix  * modelViewMatrix * vec4( newPosition, 1.0 ) ;
    }
  `
}

function fragmentShader(){
  return `
    varying vec2 vUv;
    uniform sampler2D u_tex;
    varying vec3 v3n;

  vec4 displacementMapToNormalMap(sampler2D displacementMap, vec2 vUv){
    float scale    = 0.9;   // Adjust this to control the amount of displacement
    float epsilon  = 0.01;  // Small value for calculating gradients
    float strength = 1.;                   
    float center = texture2D(displacementMap, vUv).r; // Sample displacement map
    float dx = texture2D(displacementMap, vUv + vec2(epsilon, 0.0)).r - center;  // Calculate gradients in the X  directions
    float dy = texture2D(displacementMap, vUv + vec2(0.0, epsilon)).r - center;  // Calculate gradients in the Y directions
    vec3 normalMap = normalize(vec3(dx * scale, dy * scale, 1.0));               // Calculate the normal vector
    normalMap *= strength;                                                       // Apply strength to the normal vector
    return vec4(normalMap * 0.5 + 0.5, 1.0);                                     // Output the resulting normal as a color
  }
    
void main() {
    vec3 lightDirection = normalize(vec3(0.,1.,1.));
    vec4 normalMapColor = displacementMapToNormalMap(u_tex, vUv);          // Get the normal map color
    float grayscale = dot(normalMapColor.rgb, vec3(0.299, 0.587, 0.114));  // Convert the normal map color to grayscale
    vec4 grayColor = vec4(grayscale, grayscale, grayscale, 1.0);
    float lightIntensity = dot(normalMapColor.rgb, lightDirection);       // Calculate the lighting contribution
    vec4 finalColor = grayColor * lightIntensity;                         // Apply the lighting to the grayscale color
    gl_FragColor = finalColor;                                            // Output the resulting color
}
`
}

I have been working on this problem for weeks, but I have yet to make significant progress. During my research, I came across a relevant Stack Overflow question titled “How can I eliminate normal map seams on Cube Mapped Three.js BoxBufferGeometry?” you can see here.

One of the responses suggests not using the tangent frame provided by Three.js and instead generating tangents using the vertex data to align them at the seam. However, this is above my understanding of three.js. I am unfamiliar with this technique and would appreciate some clarification.

I would greatly appreciate any guidance or suggestions on how to resolve the issue of discontinuous normals and achieve a smooth appearance for the normalized cube with a displacement texture. Thank you in advance for your help.

update 7/16/2023: dFd functions are used to sneak peak at the neighbor pixel values, this is due to the fact that pixels are rendered in small 2x2 batches. Instead of using epsilon. currently, this is the best solution. It’s not perfect:

  vec4 displacementMapToNormalMap(sampler2D displacementMap, vec2 vUv){
    float scale    = 0.9;   // Adjust this to control the amount of displacement
    float strength = 1.;                   
    float center = texture2D(displacementMap, vUv).r; // Sample displacement map
    float dx = texture2D(displacementMap, vUv + dFdx(vUv)).r - center;  // Calculate gradients in the X  directions
    float dy = texture2D(displacementMap, vUv + dFdy(vUv)).r - center;  // Calculate gradients in the Y directions
    vec3 normalMap = normalize(vec3(dx * scale, dy * scale, 1.0));               // Calculate the normal vector
    normalMap *= strength;                                                       // Apply strength to the normal vector
    return vec4(normalMap * 0.5 + 0.5, 1.0);                                     // Output the resulting normal as a color
  }

Answer 1

so, the solution was to use a CubeTexture to make the normals continues.

solution

compute the normals like so.

   vec4 displacemntNormalCubeMap(samplerCube u_displacementMap, vec3 vUV){
    float scale    = 4.9;   // Adjust this to control the amount of displacement
    float epsilon  = 0.1;  // Small value for calculating gradients
    float strength = 1.;                   
    float center = textureCube(u_displacementMap, vUV).r; // Sample displacement map
    float dx = textureCube(u_displacementMap, vUV + vec3(epsilon, 0.0, 0.0)).r - center;  // Calculate gradients in the X  directions
    float dy = textureCube(u_displacementMap, vUV + vec3(0.0, 0.0, epsilon)).r - center;  // Calculate gradients in the Y directions
    vec3 normalMap = normalize(vec3(dx * scale, dy * scale, 1.0));               // Calculate the normal vector
    normalMap *= strength;                                                       // Apply strength to the normal vector
    return vec4(normalMap * 0.5 + 0.5, 1.0);                                     // Output the resulting normal as a color
  }