Friday, September 17, 2010

Subsurface Scattering

I was washing my paintbrushes this afternoon, and I realized that I didn't remember having finished a couple of brushes that were sitting clean to one side. This used to happen when I drove across Los Angeles: suddenly I'd be closing in on Santa Monica, and I would have no memory of going through Beverly Hills. I guess I've gotten that good at cleaning paintbrushes.

What I was thinking about, rather than the brushes, was subsurface scattering. Subsurface scattering is a real nightmare for computer generated imagery programmers, but it's not an unproblem for painters.

Subsurface scattering is a significant property of how objects reflect light. Most objects, metals excepted, are at least a little bit translucent. Look at this glass of milk here:

Disclaimer: this is the first example *everyone* gives when talking about this.

The milk overall looks opaque. But if you look carefully at the meniscus - that's the little bit at the top where it touches the glass and curves up with the surface tension - you'll see it's not just white. The milk is very thin there, and some light is passing through it, because it's translucent. It transmits light:

But it's not like clear glass. That light bounces around a bit as it passes through the milk. The meniscus looks a little bluish, because milk likes to scatter blue light the most. That is, light is shining down on that milk. The light scatters in the milk. Where the overall whiteness is not intense enough to deplete your impression of color, you see the preferential scattering of blue light - the other waves move in straighter lines, so fewer of them come leaping out at you.

So much for the meniscus. But consider the body of the milk. You shine some light at some milk, and the milk looks white. Remember - the milk is translucent. Not all of the light that hits it bounces off its surface. Some of it will pass into the milk, get scattered, and keep on going. But some of it will also pass into the milk, get scattered, and come out the same side it went in:

Graphic source:

When you get a lot of light waves going into an object, and some of them come back out the same side after bouncing around a little, they come out jumbled up. We see this jumble as a kind of diffuse glowing quality to the object. Although it is almost universal, this quality is particularly pronounced in a few familiar kinds of thing: milk, quartz, seashells, plastic. And, of course, human skin.

Let's take a look at the structure of human skin:

Graphic source:

The epidermis is the outermost layer (you know, the one you can see). It has relatively low blood content. Apparently only about 6% of light striking skin bounces right off the epidermis. The skin is translucent, and the rest of the light that you see coming back at you from a lit human has undergone subsurface scattering. It has gone into the epidermis or the dermis, bounced around, and popped back out. The dermis is where the blood is - all those little squiggles in the illustration are capillaries. A Caucasian human is not pink. A Caucasian human is a combination of zero-bias translucent material, some skin pigments, and bloody red. You are not seeing pink when you see a pinkish white person. You are seeing a complex phenomenon that includes white, pigments, and rich dark red. And this person seems to glow. Light is not as hard on them as you might think it would be. Because it has become diffused and softened with its transit through their flesh.

So, this is a huge problem for computer effects people. The first movie to make an honest attempt at photorealistic digital characters was Final Fantasy: The Spirits Within (2001). Which is not a very good movie, so don't worry about renting it, please. But let's take a look at some of the characters:

That's the lead hottie, Dr. Aki Ross.

That's Dr. Sid.

And that's Ryan.

Now who's the least convincing? The one with the thick dermis and low-pigment epidermis - Aki. Because computers do not do subsurface scattering very well yet. Subsurface scattering is a complex and not completely understood phenomenon, and it even involves quantum mechanical effects. Which is always exciting. Overall, it's nowhere near as easy as simulating light bouncing off copper.

Interestingly, CG is much better at handling old people and black people. Why? Because their appearances are more strongly characterized by reflection:

In the case of old people, this is because the dermis has thinned. The skin has lost much of its translucency, so the impact of subsurface scattering on light reaching you from an old person is strongly diminished.

In the case of black people, the high concentration of melanin pigment in the skin absorbs a great deal of incoming light:

So a correspondingly larger proportion of light you see coming off of a black person is light reflected at the surface of the skin.

Another good example of the difference between a computer simulating a reflective light situation and a subsurface scattering situation is found in Jurassic Park from, oh my god it was 17 years ago, 1993. The tyrannosaurus rex is pretty convincing here, where hard side-light produces a reflection-dominated scenario:

The brontosaurus is less convincing here, in full light:

This problem still persists. James Cameron, who has a good eye and a pretty strong respect for the limits of his technology, simply decided in the design process that his Na'vi (or whatever they're called) were going to have relatively opaque skin. But this decision simply stylizes, rather than preventing, the inevitable clayish feeling they have:

Notice how much more human-seeming the eyes are, where the simulation involves an opaque surface beneath a transparent lens.

OK, so, finally we're on the same page about subsurface scattering. Maybe you'd like to know how all this relates to painting, the alleged subject of this blog.

Well, painting a person involves painting this phenomenon of subsurface scattering. And we return to something that came up before: a pink person is not pink. A yellow person is not yellow. A black person is not black. Etc. We do not have a color. We have a complex composite phenomenon which we intuitively think of as color. But it is not a color, and no color can represent it.

So how do we make a picture that reads like a person?

Back in the day, they painted in layers. Take this painting of Mars, by Velazquez:

I think I might smite you.

Or this St. Sebastian, by Rubens:

Homoerotic since at least 1618

What is this painting in layers of which I speak? Let me tell you about it! A first layer of paint is applied, generally in a darkish color or set of colors - dark grey, burnt umber, burnt sienna, red.

Once this has dried, you go back over it with a lighter set of colors: your yellow ochres, your oranges and peaches, lighter reds, some white.

Once that's dried, you go back a third time, and put in the highlights: whites, naples yellows...

Or, to phrase it in terms of our discussion here: layer 1, the darks, is the bloody part of the dermis. Layer 2, the medium lights, is the translucent epidermis. And layer 3, the highlights, is the 6% of light that bounces right off the surface of the skin.

These paintings are convincingly fleshy not only because they have these layers that parallel the real structures they represent, but because paint itself is translucent. Oil paint is a mixture of oil and pigment particles. Light scatters inside of it. You can see through it a bit. So that all-important layer 2, the epidermis layer of paint, allows you to see the underlying darkness using precisely the quantum mechanical/physical mechanism which the epidermis does. The layered painting partakes of the same mechanism, and this is why it intuitively reads as having true human structure. The person in the painting has no color, just as a real human has no color. They both have a complex mix of colors that revises itself according to a complex and ongoing process of interaction of light and deep matter.

I did a lot of painting this way when I started painting, because I learned to paint by looking closely at paintings like the ones above and imitating their method of construction. I first thought explicitly about the role of subsurface scattering when I was trying to figure out why this painting of Vadim, done in layers... so much better than this painting of Vadim, done with the exact same colors but without layers:

Aside from that this painting just isn't as good.

Now, this layering technique came before the realism of the mid-19th century. You remember how we talked one time about how the academics and the impressionists were actually very similar, with regard to their treatment of color? Well, they both made the same mistake about color, the excited over-generalization of a new scientific principle. The "true colors" that we see in the impressionist French garden, or the academic Roman bath, both involve color only in terms of reflection from surfaces. And the paint handling itself partakes of this. It is single-layered, called sometimes alla prima, essentially "in one go." The ideal of both movements is to look at a thing - to take the measure of its color - and to lay that color onto the canvas.

Well, you can do some dazzling things with that technique, but you cannot make the viewer feel in their bones that they are looking at a person:

That's the mighty John Singer Sargent, in one of his few nudes. What can you do? The man liked clothes. Anyhow, we see here everything we expect from Sargent: the stunning sense of structure, the adept brushwork, the subtle transitions, the marvelous colors. But the colors are all on the surface. He is representing the glow of flesh without replicating the mechanism of the glow. His glow is not deep, as this glow, for instance, is deep:

Funny story - I was in the Vatican this one time, and I came upon this painting, and I thought, "Holy crap is this good! Who painted this?" Then I looked at the label and I thought, "Well, that figures, it's Caravaggio." Did I tell you that one already?

Anyhow, the alla prima technique, which depends primarily on an opaque application of paint, has this one deep flaw, that the paint is applied thickly enough to suppress its own subsurface scattering, and thus to prevent any use of that scattering as a means of representation. It fails at the ambiguity of the translucent, a failure which is most pronounced in depictions of flesh, because it is the flesh we humans know best.

Where am I as a painter these days? Well, I don't layer anymore the way that the painters of the baroque layered, for the same reason the academics and impressionists gave up on it: it takes for freaking ever. But I have not pursued the solution the alla prima school has pursued. My oil paint is very, very thin, and I build it up in layers while it is wet. There is a trade-off involved in this approach: on the one hand, I am able to produce the fleshiness of the subsurface scattering - fast. On the other hand, I lose the tactility of the baroque painters and the alla prima painters alike. When you build layers properly, or you skip them altogether, you can really slather on the paint, and this builds up a sculptural surface which has its own sensuality. My work lacks that, but I've chosen to live with it, for now. As you know, I am not necessarily consistent in my attitude, and reserve the right to argue tomorrow the exact opposite of the point I am making today.

Here's something I'm working on right now, with the delightful Alley, who could hardly be more translucent of epidermis:


  1. Fascinating analysis, Daniel. This kind of knowledge really deepens an artist's perception.

    The Sargent nude looks pretty good to me, though I haven't seen the original.

    One thing that really changes painting in the 19th century is the development of synthetic pigments, which made a much wider range of colors available. The great colorists of this era, including Degas, Monet, Bonnard and Seurat, used the principle of optical mixing, where colors are interspersed on the surface of the painting instead of being layered. This can produce effects of great beauty and depth.

    Most of the flesh tones I see in alla prima figure painting by non-masters look not only too flat, but too pink or orange. Many of the old master underpaintings are done in tones of green or cool grays, which really helps to neutralize the garish quality of warm hued colors.

    In my own color drawings, which use optical mixing, I nearly always use greens and blues in flesh tones.

  2. So, at the everpresent risk of making a fool of myself, when you take a photograph of a painting and view it digitally, it's made up of pixels. No layering, no scattering, a pixel is a pixel and it has a color.

    This might be related to Fred's comment about optical mixing, now that I think of it, but I'll try not to confuse myself with that and go ahead and ask my question.

    If a layered, glowing painting can be photographed and, hence, converted to pixels of defined colors, can it not be replicated by skillful coloring? I mean, certainly the examples you give of computer generated moving characters are a different issue, as light moves as they do and that scattering needs to be handled for realistic effects. But in a still image, if your eye sees through the translucent upper layers of paint (or skin) and the color you see with your eye is a mixture of those different kinds of light that have traveled different paths, in the end, isn't it still a color? And can't that color then be mixed and applied to simulate what the eye is seeing?

  3. The partial Alley painting is amazing!!!

  4. I'll bet if you could get Alley into a glass you'd get that bluish meniscus effect!

  5. I keep looking at the Alley painting. It's fantastic. I am really looking forward to seeing more of this (and more of Alley, quite frankly).

  6. Hokey, here I am.

    Fred - thank you. I think it's interesting to think about optical color mixing in this context. You could say that this topic involves the limits of the verisimilitude of optical mixing, which can sometimes give you the right *color* but the wrong *sense.* Overall, though, you're right - they got some dazzling colors. Incidentally, I think I wrote the "optical black" series of posts before you started reading the blog - you might enjoy them if you want to go back and find 'em.

    The development of synthetic pigments has always struck me as an interesting topic, which means that maybe I should read something about it sometime so I'll know more than that it widened the possibilities for painting interesting sunsets.

    I know what you mean about the pink and orange problem. The best flesh-tones I've seen in contemporary alla prima come from painters who have a strong sense of the cold neutrals: Adam Miller, Jacob Collins, and Jeremy Lipking come to mind. Miller and Collins, as far as I know, lean on blue-grays, while Lipking likes muted green-grays.

    I like your flesh-tone colds - I myself depend for them almost entirely on Sennelier cool-gray 707, which is a dark bluish-gray, and simple black. I use black to substitute for blue in a lot of a my work.

    Ed - Godfuckingdamnit, I hoped nobody would notice that! I thought about writing about it, but I decided the post was already long enough. You are completely correct. Every single thing I'm talking about is, by its nature, completely unphotographable. Every image I have posted shows, at best, a weak approximation of the effects I am describing. Certainly the complexity of the pixel mix in the Velazquez is greater than it is in the Sargent, but that's not the same thing, categorically, as the presence or absence of subsurface scattering. This post is really only a weak shadow of strolling through a gallery and having the conversation in person. The sense of substantial presence, or absence of substantial presence, really only emerges in confrontation with the physical painting itself.

    To address your last questions, to some extent the color is path-independent, but in some important ways, it is not path-independent, and therefore cannot be simulated. A good subsurface scatter can be mimicked with hard highlights painted over softer light areas, but the real thing involves constant change over time. Each photon is behaving slightly differently as it bounces around inside the material and comes back out. So the character of the diffuse glow, which is visible to the human eye, is in constant, if minute, flux. Add to this the different impressions received by each eye, which slightly contradict one another and produce a kind of pre-cognitive shimmer, and you have the full-bodied presence of the real subsurface scatter.

    The problem the computer programmers are facing is that they cannot even simulate the traces of subsurface scatter visible to a camera lens. It's different from the problem faced by the painter, who must choose between successfully simulating these traces alla prima, or producing them, in layers.

    In case you're wondering, I am largely talking out of my ass, but it all stands to reason, doesn't it? I used to do math and physics this way in school - if you remember the basic principles, you can re-derive the rest yourself on the test without all that tedious "memorizing" part.

    Anyhow, points to you; far from being foolish, your point was devastating.

    And I'm glad you're enjoying the Alley painting. I'll post more as it progresses.

  7. All the color on computer monitors is produced by optical mixing of only three colors.

    A few years ago the Met had a show of Fra Angelico. His paintings are not photorealistic, but they do have a wonderful luminosity. There were some unfinished pieces there, and his underpaintings of flesh tones were in vivid jade green! I believe the originals were egg tempera, so the layering technique may be quite different from oil painting.

  8. "...the real thing involves constant change over time. Each photon is behaving slightly differently as it bounces around inside the material and comes back out. So the character of the diffuse glow, which is visible to the human eye, is in constant, if minute, flux. Add to this the different impressions received by each eye, which slightly contradict one another and produce a kind of pre-cognitive shimmer, and you have the full-bodied presence of the real subsurface scatter."

    Excellent! Fascinating. I learn more from (and am infinitely more interested in) these conversations than I did in college as an "art major."

  9. Fred - Thanks for the clarification on monitors. You see the problem Ed points out. I do think that we respond in subtle ways to the mode of color generation, and that all outputs from a monitor, in this sense, have the same intuitive feel. I didn't know that Fra Angelico painted like that! But it makes perfect sense if egg tempera shows limited transparency, which the underpainting implies it does. The overlayers would dull the green and the green would inform the flesh tones - of course you know that. There are some people working with egg tempera now, and they get those same gorgeous luminous muted colors.

    Ed - I'm glad you get so much from these chats! But please remember one thing: unlike your boring professors, I am not really sourcing or verifying anything. You've taken my claims with a grain of salt so far, and I think that's a good way to go - I'm often wrong, even when it sounds like I know what I'm talking about. One other thing: I really appreciate the contribution you're making to exploring the topics that come up (that goes for you too, Fred; it's great to test my claims against the ideas you guys bring).

  10. As long as there's no test at the end, I'm good. ;) I am sure that when you point out things like the slight contradictions of each eye when viewing something with subtle subsurface scatter, and that those contradictions likely have an impact on the viewing experience as it's processed by the brain, it just all makes sense. I like the things you come up with to discuss as a result of your observations and your just thinking about things.

    And to me, again far removed from any academic 'test' or anything, I'm far better off thinking about something and even coming to (or agreeing with) an inaccurate conclusion*, than I am just going through my day without thinking about these things at all.

    *I'm not saying your thoughts or theories are inaccurate, I'm just making a point that it matters little because the value of my pondering of the topic is greater than giving it no thought at all.

    And thank you for taking my questions seriously (I do fear that they are uninformed) and being so thoughtful in your replies.

    (And finally, my confidence level that my professors in general had sourced or verified anything more than you have is very close to zero.)


    theres my subsurface scattering work... i dont work in layers i just lay tan on black and make sure its blurry enough, you could say my effect is TOO STRONG. :) specular helps to wetten it.

  12. oh sorry i forgot to say yes i do work translucently, of course, just no differing layer of hues, its just continual tanning on tan, but yes, translucency is the key... i figure water colours would be quite successful at it.

  13. just one more thing to add, yes subsurface scattering is the way to become a master, anyone that disagrees cant paint.

  14. [img],550x550,075,f.innocent-young-girl.jpg[/img]
    just like to correct you, when you say computers cant do subsurface scatterin 100% physically correct, you are WRONG.