Thursday, January 28, 2010

Edges and Edge Detection Part 1: Neurology

Well, it's been too long since I've developed an idea that struck me while reading Margaret Livingstone's Vision and Art: The Biology of Seeing. But I've been mulling over the concept of edges, from which the concept of line arises. This is one of the fundamental tools of art, and well worth thinking about a little more.

How do we see edges? As always, the brain avoids using high-level structures for brute information processing of the visual field. You might think, well, receptor cells in the eye respond to light, firing a signal when they are exposed to light and failing to fire a signal when they are exposed to darkness:

That's what you would think, my friend, but you would be leaving most of the story woefully untold. Yes, these cells exist. But it seems that this fairly obvious scheme is not cool enough for the brain. So the raw "light/dark" cells feed this information, while in the eye, to ganglion cells that show center/surround signal processing. Instead of a binary ON for "I'm lit" and a binary OFF for "I'm not lit," they respond to their little flock of receptor cells to produce a continuum of signals (actually, a continuum of frequencies of firing). The simplest scenarios are as follows. The bottom two "high frequency" responses represent, I'm pretty sure, the optimum firing conditions for two different types of center/surround cell, both of which show the first two responses in totally lit and totally unlit conditions:

Keep in mind that we're talking about black and white processing here. We'll get back to that.

But consider this scenario:

That's a classic edge - a sharp contrast over a short region. And the cell is equipped to produce a signal frequency between "medium frequency" and "high frequency" in response.

How does the brain take advantage of all this fancy information that's getting processed in the eye? Well, a couple of Livingstone's colleagues in the Department of Extreme Cleverness, David Hubel and Torsten Wiesel, figured out that these center/surround ganglion cells are feeding their information to cells in the brain that are equipped to compare signals from adjacent ganglion cells. And these higher-level processors show orientation. Several different specialized cells, which are at the point of detecting edges, are wired to each cluster of lower-level center/surround cells, and they fire when the cluster, as a composite, matches their specialized detection function:
On the left are the center/surround ganglia. On the right are the edge detector cells in the brain. Shaded cells are busy firing.

From there, you're off to the races. Further specialization and processing allows detection of continuous edges, broken edges, curved edges, corners, etc. And finally, you have an image that you're conscious of.

What's important to take away here is that, even though you are not aware of it, deeply embedded in your visual perception of the world are these low-level edge data, which are cognitively close to line data. Lines are deeply embedded in your sense of sight. This is a big part of the reason that when you see a line drawing... see a comprehensible visual field, even though virtually all of what you intuitively think of as information in the visual field is missing. The defining characteristics of a critical part of that visual field are present.

This is also why the figure/ground distinction exists in reality even though it does not exist in the raw visual field. The brain never sees the raw visual field. It only becomes aware of the neurologically-processed visual field, into which the edge phenomenon has been encoded.

Now, lots of interesting artistic results arise from all this, and the next few posts will build up a discussion of it. For the time being, let me just say, the same center/edge processing exists in color, but it's lower resolution. This is a big part of the reason that Degas died thinking of himself as a failure. His grand project was to unite the artistic phenomena of color and line. He never thought he found that unity, despite his gorgeous attempts:

Why? Because the line he intuitively recognized as Line is achromatic. It cannot be unified with color. Which is not to say that Degas's attempt was a waste of time. Just look at what he accomplished by pursuing an impossible goal.

A couple notes:

1. Doing all those diagrams in Photoshop? Total pain. Not recommended.
2. I think I'm screwing up the real model of processing as presented by Livingstone. The thematic fundamentals are correct, but I can't vouch for my having related every step of the way correctly. Just read her book, it's awesome.


  1. Maybe in your next few posts you can elaborate on the effects of comic artists such as Moebius and R. Crumb, both of whom have distinct styles and use lines in very different ways.

  2. I'm getting to it man! I was planning on mostly talking about the Dave Gibbons/Frank Miller dichotomy, if that's good enough for you. Most of the world is one or the other, roughly. Xaime = Gibbons. Gilbert = Miller. Moebius = Gibbons. Crumb = Miller. Pete Bagge = retard who can't draw.