In the post before we created  leaf bitmap, either with embedded transparency or with alpha channel. Now I want to point out to some problems that might occur.

Halos or visible fringes
During rendering the leaf bitmaps are filtered and due to oversampling (antialiasing) the background color is mixed with the leaf color. This can lead to visible halos around the leaf edge. In our case with black background this would lead into a dark halo.
Most commercial render applications offer a special option for alpha channels that solves exactly this problem. This option is often called premultiplied or something similar….

What does premultiply mean? Well, I don´t want to go into details – other people already did this much better then I can. I just want to tell you so much: when you use a black background for your leaves (or embedded transparency), then most renderers exactly know what to do. They know the background is black and so then can do their calculations to avoid the unwanted halos.
Timm Dapper has written a nice article on this topic: Alpha Blending for leaves. On Wikipedia you also can find an article about Alpha Channels .

File compatibility problems
There are many formats out there and what format to use often is just a matter of taste.

  • *.tga – very common format, compression possible, alpha channel possible, only 8bit. Works in most cases
  • *.jpg – works in almost all renderers, artifacts due to lossy compression possible, no alpha channel
  • *.png – supports embedded transparency and different bit depths, not compatible with all renderers
  • *.tif – supports alpha, embedded transparency and even layers, different compression algorithms available, also different bit depths. Good format, but be sure to use only options your renderer supports!
  • *.exr – most recent format, not yet supported by all renderers, but could be the new allround format

One pitfall with some of those formats is compression. This is true for *.tif especially, where LZW compression often causes problems. I usually try to save uncompressed. It takes more space on the disk, but for rendering it makes no difference. The contrary is true in many cases, compressed formats sometimes need more RAM during rendering, because they have to be uncompressed on the fly.

Some hints:

  • Sometimes the color channels might be a good place to look at. In some cases you will have a color image with „low contrast“ which makes it hard to use automatic algorithms to do the masking work. But when looking at the color channels it might be, that you will find more contrast in there. If this is the case, then use one of the R G B channels instead of the „regular“ image
  • To create the masks it also might be an idea to create a copy of the original layer, work on contrasts or colors and then start creating the mask on the color corrected layer
  • Do yourself a favor and either scan leaves or at least shoot them in front of an uniform background
  • in case premultiplied alpha does not work in your render application, try to shrink the alpha channel one or two pixels
  • another option would be the „bad“ background color = leaf color approach, but only use this if all other options fail. It´s evil ;-)
https://youtu.be/NAbX026ZS3k