Octahedron normal vector encoding

Many rendering techniques benefit from encoding normal (unit) vectors. For example in deferred shading G-buffer space is a limited resource. Additionally it’s nice to be able to encode world space normals with uniform precision. Some encoding techniques work only for view space normals, because they use variable precision depending on normal direction.

World space normals have some nice properties – they don’t depend on camera. This means that on static objects specular and reflections won’t wobble when camera moves (imagine FPS game with slight camera movement on idle). Besides their precision doesn’t depend on camera. This is important because sometimes we need to deal with normals pointing away from camera. For example because of normals map and perspective correction or because of calculating lighting for back side (subsurface scattering).

Octahedron-normal vectors [MSS*10] are a simple and clever extension of octahedron environment maps [ED08]. The idea is to encode normals by projecting then on a octahedron, folding it and placing on one square. This gives some nice properties like quite uniform value distribution and low encoding and decoding cost.

I compared octahedron to storing 3 components (XYZ) and spherical coordinates. Not a very scientific approach – just rendered some shiny reflective spheres. Normals were stored in world space in a R8G8B8A8 render target. Post contains also complete source code (which unfortunately isn’t provided in original paper), so you can paste into your engine and see yourself how this compression looks in practice.

XYZ

float3 Encode( float3 n )
{
return n * 0.5 + 0.5;
}

float3 Decode( float3 f )
{
return f * 2.0 - 1.0;
}

Spherical coordinates

float2 Encode( float3 n )
{
float2 f;
f.x = atan2( n.y, n.x ) * MATH_INV_PI;
f.y = n.z;

f = f * 0.5 + 0.5;
return f;
}

float3 Decode( float2 f )
{
float2 ang = f * 2.0 - 1.0;

float2 scth;
sincos( ang.x * MATH_PI, scth.x, scth.y );
float2 scphi = float2( sqrt( 1.0 - ang.y * ang.y ), ang.y );

float3 n;
n.x = scth.y * scphi.x;
n.y = scth.x * scphi.x;
n.z = scphi.y;
return n;
}

Octahedron-normal vectors

float2 OctWrap( float2 v )
{
return ( 1.0 - abs( v.yx ) ) * ( v.xy >= 0.0 ? 1.0 : -1.0 );
}

float2 Encode( float3 n )
{
n /= ( abs( n.x ) + abs( n.y ) + abs( n.z ) );
n.xy = n.z >= 0.0 ? n.xy : OctWrap( n.xy );
n.xy = n.xy * 0.5 + 0.5;
return n.xy;
}

float3 Decode( float2 f )
{
f = f * 2.0 - 1.0;

// https://twitter.com/Stubbesaurus/status/937994790553227264
float3 n = float3( f.x, f.y, 1.0 - abs( f.x ) - abs( f.y ) );
float t = saturate( -n.z );
n.xy += n.xy >= 0.0 ? -t : t;
return normalize( n );
}

Conclusion

Spherical coordinates have bad value distribution and bad performance. Distribution can be fixed by using some kind of spiral [SPS12]. Unfortunately it still requires costly trigonometry and quality is only marginally better than octahedron encoding.

One other method worth mentioning is Crytek’s best fit normals [Kap10]. It provides extreme precision. On the other hand it won’t save any space in G-Buffer as it requires 3 components. Also encoding uses a 512×512 lookup texture, so it’s quite expensive.

Octahedron encoding uses a low number of instructions and there are only two non-full rate instruction (calculated on “transcendental unit”). One rcp during encoding and one rcp during decoding. In addition quality is quite good. Concluding octahedron-normal vectors have great quality to performance ratio and blow out of water old methods like spherical coordinates.

UPDATE: As pointed by Alex in the comments, interesting normal encoding technique survey was just released [CDE*14]. It includes detailed octahedron normal comparison with other techniques.

UPDATE 2: Added optimized Octahedron decoding by Rune Stubbe.

References

[MSS*10] Q. Meyer, J. Sübmuth, G. Subner, M. Stamminger, G. Greiner – “On Floating-Point Normal Vectors”,  Computer Graphics Forum 2010
[ED08] T. Engelhardt, C. Dachsbacher – “Octahedron Environment Maps”, VMW 2008
[Kap10] A. Kaplanyan – “CryENGINE 3: Reaching the speed of light”, Siggraph 2010
[SPS12] J. Smith, G. Petrova, S. Schaefer – “Encoding Normal Vectors using Optimized Spherical Coordinates”, Computer and Graphics 2012
[CDE*14] – Z. H. Cigolle, S. Donow, D. Evangelakos, M. Mara, M. McGuire, Q. Meyer – “A Survey of Efficient Representations for Independent Unit Vectors”, JCGT 2014