I’m surprised how often I stumble over general 3D forum posts with the same question of how to avoid jagged edges or dark/ bright thin lines (‘halo’) at the object borders of their rendered 3D image with a transparent background (usually by using PNG or TGA image format for non animated images) when using them to make a composite in Photoshop. This is usually a problem when the pre-multiplied alpha setting is turned on – but that’s not all… I experienced severe troubles with PNG files: with the TGA format it worked fine. Here’s a short ‘how to’.
Given the situation you want to render an object to be used for post-production within Photoshop, then…
- Be sure you set the background to complete black: Rendering > Environment > Background > Colour (click on the colour field and set red, green and blue to 0)
- Set the antialising in your render settings to Blackman - this type of antialising is quite sharp and doesn’t blur the edges. In V-Ray this is located in Renderer > Image sampler > Antialiasing Filter.
- Set the file type in the common rendering dialog to tga, 32-bit, premultiplied and alpha split off, compress on.
- In Photoshop, load the TGA image, double click the layer to convert it from a background layer into a normal layer.
- Go to channels, CTRL-select the alpha channel, invert the selection, go back to layers and hit the delete button.
That’s it. Now you have a totally clean 3D object. No jagged edges or slight ‘halo’ around it, clean shadows (by using matte grounds), no pre-multiplied colours within the semitransparent areas etc. In combustion it’s all dead easy: just use RPF, deselect pre-multiply alpha (but store is on) and all will be fine. Bad luck Photoshop doesn’t support this image format, this would be one hell of a good feature.
Addendum: If you already have rendered your image and your alpha is accidentally pre-multiplied, you can add a 6th step to the 5 I mentioned above. In Photoshop go to Layer > Matting > Remove Black Matte. Voilà, you got rid of the black ‘halo’ again.