Scott DeWoody - Ambient Occlusion 3ds Max (Mental Ray) Tutorial

Ambient Occlusion 3ds Max (Mental Ray)

Once you're scene is pretty much finished, it is time to set up the occlusion. You want to have your scene pretty much completed when you get to this stage, because if you move the camera, or an object, even a little, it will throw the occlusion off.Ok, so here are the steps I take to set my occlusion renders up in 3ds Max 9 and Mental Ray. So save your scene, and lets get started!

1) First, go to Render > Environment, and change the environment color from what ever it is (default is black) to pure white. Then, we will need to switch the rendering engine from what ever you have. (Most likely its scanline if you havnt changed it before). So go to Render > Render to bring up the Render Globals. You can also click the Render Globals button on the tool bar.

2) On the "Common" tab, scroll all the way to the bottom, where you will see a "Assign Renderer" roll-out tab. Click this to expand it. The first line you'll see is "Production" followed by the Render Engine name, followed by three "...". Click the "..." button, and choose "Mental Ray".

3) Once you click "Mental Ray" you'll notice some of the tabs change at the top. Click the "Renderer" tab. Under samples at the top, change "minimum" to 1. You can leave "maximum" at 4 if you whish. Changing Maximum to a higher value will result in a cleaner render, yet will add to the render time. Normally I'll change it to 16 as a last resort to cleaning up the render.

4) Next to "Sampling" in the "Renderer" tab, is the "Filter Type". Change the type from Box to Mitchell. This will add a nice AA Filter to the image to get rid of some of the grain that the occlusion makes.

5) Leave the Render Globals open for the time being, and open the Material Editor. Change either an exsisting shader "Mental Ray" or make a new shader and make it as a "Mental Ray" shader. (*Note* All Mental Ray shaders and maps have yellow icons. If you see a pink/purplish icon, that is DirectX.)

6) In the Mental Ray shader, click the first map slot at the top which is labled "Surface". The map window will open, and pick the very top one that says "Ambient/Reflective Occlusion".

7) Under the Ambient Occ Map options, change the samples up to 64. I normally just start right off with 64 samples and dont really change it. I cant really tell a difference with a higher sample rate than this, but sometimes higher samples will help, but will add to render time. If you are going to play with samples, it would be best to keep the numbers in multiples of 8. (ie. 8 16 24 32 64 128 ect)

8) Scroll down some to where it says "Max Spread" and "Max Distance". These two values sort of play off of each other. Normally I'll leave the Max Spread as it is (which is .75), and change the "Max Distance" from 1 to 20. This is just a starting value for the "Max Distance" You will need to do lots of test renderings to get the right value that works for you. And it always seems to change for each scene you do. So either increase or decrease this number if the occlusion is coming out to bright or too dark. (I'll show you examples of this when we get to rendering.)

9) Ok,so now with the material created it, dont apply it to anything. Remember, we left the "Render Globals" open. So bring that window to the front, and click on the "Processing" Tab. You should see a little section called "Material Override". Check the little box next to "Enable". Then click and drag your Occ shader from the material library to the "None" map slot in the "Material Override" section of the "Processing" tab. (When you do this, Mental Ray will apply this material to all objects in a scene when it comes time to render, so you dont have to worry about re-texturing everything.)

10) Hit render!
10b)At this point, I'll save an extra copy of my scene just for the occlusion. So in the end you'll have 2 scene files. One for your "diffuse" or "beauty" pass as its called, and then one for your "occlusion" pass. And as long as you dont change the camera or the placement of the objects in the "diffuse" scene, you can keep the occlusion file with you. You can change the textures of objects however, so dont forget that.

11) Ok so we have our occlusion render, but we're not done yet. Save this picture in any format you want. I normally save my occlusion files as J-PEG (.jpg). If you havn't rendered a "normal" pass of your image yet, you'll need to take off the Material Override and render it out, like this one:

12) Time to open photoshop. At this point you'll need a "diffuse" or "beauty" pass rendering. (This is the render with all your textures and lighting, etc) and you'll need your occlusion pass. Once both of these files are open, Copy the Occlusion image to the "Diffuse" image. (You can do this 2 ways in photoshop.) 1) Click and drag, with the move tool, the occlusion onto the diffuse image. 2) Hit ctrl+A and then ctrl+C on the occlusion image, then hit ctrl+V on the diffuse image)

13) In the layer pallte, make sure the occlusion layer is on top of the diffuse. Then select the occlusion layer and change the blending mode from "Normal" to "Multiply".

14) Most likely the occlusion will be really dark for the image at this point. So you have two options. The first option is to lower the opacity. This is the quick and dirty way to blend the occlusion into the image. The second way, is a little more accurate and it looks better. You want to add an adjustment layer > Curves. Then you want to bend the curve up, close to a 2.2 gamma curve. This will brighten the image considerably, but it normally adds a very good touch to it. Plus the occlusion wont be as dark, and should blend well with the diffuse. You can also try lowering the opacity of the occlusion at this point too, for a little more control. If the image is way to bright, make sure the adjustment layer is above the occlusion layer, and turn it into a clipping mask. (right click on the adjustment layer and pick "clipping mask") This will make the adjustment layer only affect the occlusion, leaving the diffuse layer the way it was.

Click to see with and without the occlusion!

Mix and matching these techniques is normally what I do to achive the look I want. Sometimes you wont have to this too much to your image, where sometimes your image will just "pop" when you add these techniques together, so I suggest you experiment with your images to get the look you want.