Saturday, November 26, 2005

Normal Map Bug in Max 8

I submited a new normal map bug to Autodesk a few days ago. 3ds Max 8 ships with at least 4 different methods of displaying normal maps in the real-time viewport. These methods are:

1. The "DiffuseBump.fx" DX9 shader
2. The "DX Display of Standard Materials" check box on the Material panel
3. The "RTTNormalMap.fx" DX9 shader
4. The Metal Bump9 Material Plugin

Of these four, all of them except DX Display of Standard Materials have serious visual problems. I believe that the issue is related to the way that Max generates the tangents. Here's an example of the problem:

Max 8 Normal Map Tangent Bug

That big white spot definately shouldn't be there. This is a standard geosphere that's using the "RTTNormalMap.fx" DX9 shader. The normal map that is applied is a simple 127, 127, 255 blue square - basically a blank normal map. I applied a box UV map to the sphere but you get similar artifacts when you use all of the other UV mapping types.

All of the shaders that I have posted on my web site had this problem too. After playing around with lots of different methods, I discovered that the problem happens when the light vector and eye vector are transformed to tangent space in the vertex shader and then used together with the tangent space normal map in the pixel shader. If I transform the eye vector and light vector to world space in the vetex shader instead and then transform the tangent space normal map to world space in the pixel shader (doing all math in world space instead of tangent space), the issue goes away. The drawback of this solution is that it requires more pixel shader instructions - so the shader will run a little slower.

Here's hoping that Autodesk addresses this issue in a service pack! In the mean time, I've updated all of my shaders to use the world space math instead of the tangent space math to give more correct results. You can grab them here.