Saturday, November 26, 2005

Normal Map Bug in Max 8

I submited a new normal map bug to Autodesk a few days ago. 3ds Max 8 ships with at least 4 different methods of displaying normal maps in the real-time viewport. These methods are:

1. The "DiffuseBump.fx" DX9 shader
2. The "DX Display of Standard Materials" check box on the Material panel
3. The "RTTNormalMap.fx" DX9 shader
4. The Metal Bump9 Material Plugin

Of these four, all of them except DX Display of Standard Materials have serious visual problems. I believe that the issue is related to the way that Max generates the tangents. Here's an example of the problem:

Max 8 Normal Map Tangent Bug

That big white spot definately shouldn't be there. This is a standard geosphere that's using the "RTTNormalMap.fx" DX9 shader. The normal map that is applied is a simple 127, 127, 255 blue square - basically a blank normal map. I applied a box UV map to the sphere but you get similar artifacts when you use all of the other UV mapping types.

All of the shaders that I have posted on my web site had this problem too. After playing around with lots of different methods, I discovered that the problem happens when the light vector and eye vector are transformed to tangent space in the vertex shader and then used together with the tangent space normal map in the pixel shader. If I transform the eye vector and light vector to world space in the vetex shader instead and then transform the tangent space normal map to world space in the pixel shader (doing all math in world space instead of tangent space), the issue goes away. The drawback of this solution is that it requires more pixel shader instructions - so the shader will run a little slower.

Here's hoping that Autodesk addresses this issue in a service pack! In the mean time, I've updated all of my shaders to use the world space math instead of the tangent space math to give more correct results. You can grab them here.


Anonymous Anonymous said...

I notice on my machine this problem shows up in the "TangentSpace_2_0" Technique, but not in the "TangentSpace_1_1" Technique of the RTTNormalMap.

November 27, 2005 3:48 AM  
Blogger Ben Cloward said...

Oh, yeah - good point. I forgot to mention that. And for my shaders it doesn't show up in the simple normal map shader (the 1.1 version) but does did show up in all of the normal map shaders that were using pixel shader 2.0 - so pretty much the same thing.

I think that deserves some further investigation...

November 27, 2005 8:30 AM  
Blogger Ben Cloward said...

Ok, so after some experiementing I discovered that the difference between the 1.1 version of the shader and the 2.0 version that's causing the bad lighting to show up is normalizing the tangent space light vector in the pixel shader. In the 1.1 version, the light vector is left as is (un-normalized) - but in the 2.0 version, the light vector is normalized. Normalizing the light vector should create more correct results - so the 2.0 version should be better. For some reason in this case when you normalize the light vector it produces these poor results. If I alter the 2.0 shader so that the light vector remains un-normalized, all of the bad lighting goes away.

Interesting. I need to figure out a way of appending this little clue to my bug report. Thanks for the tip, anonymous!

November 27, 2005 6:39 PM  
Anonymous Anonymous said...

Strange, but that would explain why it effected some of my shaders and not others.

I still learning, and can be pretty slopy about making sure to normalize all the vectors.

November 28, 2005 8:09 PM  
Anonymous Anonymous said...

I came across your site through a google image search and would like to say thank you for having a free texture database for use in indie projects. If it's alright with you, I might end up using a couple. Thanks!

December 03, 2005 3:17 PM  
Blogger Ben Cloward said...

You're very welcome. Feel free to use them for whatever you want - even professional, money-making jobs. They're free!

December 03, 2005 3:20 PM  
Anonymous Anonymous said...

Hi Ben.

Im a student in my last year at high school in Barnsley, England.

I just felt I should tell you that that I found your brick textures very useful for my graphic design work in ICT.

Thank you for letting me use them.

Cya later,


December 08, 2005 3:43 AM  
Anonymous Anonymous said...

In a related note, working on a normal mapped shader for Max 7.1 I found that the results were incorrect when I used the binormal provided by the vertex shader. If I calculate the binormal, cross (tangent, normal), then the results are as expected.

February 03, 2006 6:33 PM  
Anonymous Anonymous said...

thanks for the textures--they're absolute life-savers and much appreciated.
starving artist

April 01, 2006 5:23 PM  
Anonymous Anonymous said...

Thanks for the textures ... I'm attempting to combine them with traditional line drawing ... i'll post the results when i'm done. Cheers mate!

May 27, 2006 7:54 AM  
Anonymous Anonymous said...

Thanks for the textures mate....

May 27, 2006 7:59 AM  
Anonymous Anonymous said...

thank you Ben for being kind wenough to have your textures available for all to use. afgain my gratitude. Be well

July 18, 2006 9:56 AM  

Post a Comment

<< Home