Time to time, the subject of subjective experience comes up: Why is red "red"?
The question is universal—Most sighted people see at least one color
in addition to variations in brightness.
Outside of sight, the question of what determines
the content of "qualia", the atoms of subjective experience, is
popular. "What is it like to be a bat?", "Neural plasticity and
consciousness".
Recently, I ask a friend: Why do colors seem as they
do and not some other way? How can we know that two people
experience them similarly?
I do not remember the exact words, but the content of their answer
was thus:
The statistics of color are not symmetric. Blue and
green overlap in a specific way, as do green and red.
We see correlations in color data that are inherent to the
absorption spectra of the cones in our retina, the properties of
light, and the chemical statistics in our environment.
You think that color is mysterious, because it seems you might
exchange red for blue, blue for green, and green for red, and the
world should stay the same. But this is not the case. This would
lead to fundamental differences (and presumably evolutionary
disadvantages) in what the brain can know about colors.
At first, this answer satisfied me: Color qualia were not some
ineffable mystery of the universe, but something inherited by the
content and processing of information in our nervous system. Similar
content and computations should create comparable subjective
experiences. The (closed-loop) processing of information is equivalent to its
experience.
This seemed in line with the loose academic consensus,
with some philosophers adding that the effect of our actions on our
senses is also vital for determining subjective content
(e.g. Hurley and Noë 2003). But, these ideas are not yet quite precise, especially in the
context of color vision.
What is so special (and captivating) about color?
Do we understand the computations underlying its perception?
Very few people frame the problem of qualia as
"Why is left "left"and right "right""?
Something sets color apart. The symmetry of left and right, up and down, seem obvious.
Indeed, some people can hardly tell left from right.
And, if you were to start wearing prismatic glasses that flipped
the world upside down, the brain adapts. In time this adaptation
would restore a subjective experience demonstrably similar to the
right-side-up world of the past.
In these dimensions it seems that subjective experience really is
symmetric.
Left and right are distinguished because you, as an
embodied agent, have a left hand and left eye, a right hand and a
right eye. Their symmetry is broken only by the multitude of
asymmetric objects in our environment, not to mention our numerous
cultural artifacts where left and right must be kept separate (reading
text, driving on the road). Brain lateralization (are you left or
right handed) also breaks this symmetry.
It seems that many qualia are explained by their informational
content (and how this is affected by our actions).
To me the senses of sound, grayscale vision, somatosensation,
proprioception, are all "colorless". It feels intuitive and satisfying
that their subjective content should arise from their relationships
amongst themselves and with my own body.
I cannot say the same for color.
But, there are other senses that
defy "explaining away". Smell, taste, pain come with a preexisting
opinion about how they should feel that lies outside their raw
information geometry. The nervous system contains hard-wired
circuits for processing such sensations.
There are nuclei that trigger salt cravings, dedicated pathways for
pain, and (at least in rodents) innate tagging of certain odors as
terrifying or sexy. Evolution provides hard-wired opinions about
what some sensations mean, and what actions are appropriate upon
their experience. This is a reasonably satisfying explanation of more
visceral sensations, although much remains unaccounted for.
But, back to color. It seems that something "breaks the symmetry"
of color perception so that, yes, red really is "red", and cannot
be anything else. Yet, color is not so visceral that it might be
explained by innate connections.
What possible explanations are there?
- Are color statistics asymmetric in a stable way across time
(and across individuals)?
- Are color channels processed in a way that creates innate quality
(like taste and smell)?
- Can biases in how the early visual system encodes and processes
color explain its subjective quality?
Are color statistics asymmetric in a stable way across time
(and across individuals)?
While we cannot permute red, green, and blue
color channels, color space does have an inherent geometry that
might permit other transformations.
If it were simply the case that red, green, and blue have a
particular correlation structure, then your color quality would
return to baseline after a period of immersion in augmented reality
that provides altered color data with similar correlations.
Red, green, and blue may not be "axes of symmetry", but there would
be other (probably nonlinear and non-uniform) transformations of
color space that *are* symmetric and *do* preserve qualia under
these symmetries.
To some extent this is true. If you look at Bridget Riley's painting "Gather",
you will see how important context and contrast
are for color perception. Red is not so much "red", but rather a very
specific lack of green-ness and blue-ness, in the visual vicinity.
Different (R,G,B) tuples can be perceived as the same subjective
color, depending on context.
But, this is also true for other senses: adjacent (colorless) shapes
can bias perception of size and length.
All this really says is that the relevant informational dimensions
of qualia are relative.
Perhaps the set of symmetries that matter for determining whether
information geometry alone determine qualia are curvature-preserving
isomorphisms within a given sensory space? It is not enough to retain
the 3-D nature of color vision. It is enough, however, to preserve
notions of perceptual distance (similarity vs contrast) across the
space?
The circuitry of the retina and brain are specialized to
filter and compress visual data. Thus, transformations must preserve
color similarity not at the level of rods and cones, but in the high-
level information available to our conscious brains after post-
processing through the retina and V1.
Identifying such a transformation is beyond our current abilities.
We do not understand early visual color encoding that well.
And, I do not think this would be sufficient. I've mentioned that the
changes in sensory input due to our own actions matter for qualia.
More generally, the way that sensory input changes due to other
known factors matters. Colors must also change in a specific way
with illumination levels, and in how they change as an apple ripens
over time.
It seems that even if all of these conditions were
met, it would still not be possible to find a symmetry of color
vision that preserves subjective qualia.
Are color channels processed in a way that creates innate quality
(like taste and smell)?
It seems that different colors have innate behavioral "valences".
Primates that can see red can use it to find ripe fruit or to
communicate social cues. Blushes are red.
And, indeed, different retinal ganglion cells project to different targets. Most visual
data is eventually sent to visual cortex, but more primitive areas
are also targeted.
It seems not-unreasonable to claim that the subjective asymmetry
of color, then, arises in ways similar to taste or smell. Colors mean
something, and have some opinions about what behaviors they
should trigger if you want to survive and reproduce.
Somehow, this explanation remains unsatisfying to me.
Color vision is part of image-processing and image-formation.
If innate valance alone determined the subjective experience of
color, we would need to explain how it couples to visual perception.
We do not simply get a vague feeling of "that apple looks nice to
eat", but a detailed, pixel-for-pixel sense of redness.
My hunch is that innate valence does bias qualia, in the
sense that it tags some color channels with specific and distinct
predictions. But I am not sure this explains everything.
There are other visual inputs which come with an innate valence.
Babies detect faces innately.
Faces are intensely ethologically relevant. Yet, it is the composite
object of the face that is tagged with innate valance. We do not
appreciate "red" only when it takes a form or context that is
appetitive.
Can biases in how the early visual system encodes and processes
color explain its subjective quality?
This seems likely. From the retina, colors are encoded
differently. Contrasts in blue and yellow have coarse spatial
resolution, but good temporal resolution. Contrasts in red and green
have good spatial resolution, but less temporal precision.
There is an optical illusion wherein a rotating disk with black and
white markings gives rise to illusory colors. The specific color
depends on the spatial pattern and rate of movement. (Look up
"Flechner colors").
This shows that different colors couple differently to our
perception of motion and contrast.
Illusory colors experienced when viewing a
white, immersive strobe light (flicker ganzfeld)
further highlight this.
This seems sufficient to ensure that different colors
feel "different". The asymmetry in processing is so severe
that no external transformation of color space will lead to
inputs with comparable correlation structure.
The sub-units of color processing (in cortical columns) are tiled
repeatedly across visual cortex. This is also consistent with the
notion that the it is the "pixels" of vision that are tagged with
subjective aspect, rather than something more abstract.
Assembling it all.
It seems sensible that some aspects of our experience of color
should be innate. A ripe apple and a blushing face are important.
But, innate valence (comparable to smell and taste),
would not necessarily imply "pixel-wise" subjective specificity to
color.
Furthermore, color processing in early vision is intensely asymmetric, likely in
a way that makes it impossible to transform color space while
preserving information geometry after early visual encoding.
This provides an inherent, "pixel-wise" asymmetry to color that could
lend it specific qualitative content.
Is that all, then? The innate structure of early vision provides
specific, oriented, asymmetric "labelled channels". These enforce
the similarity space of color perception. It is a world where "up" is
up and will never be anything else. Simultaneously, other brain
structures provide innate valence that tags these channels in
relation to instinctive behaviors.
Does neural plasticity associate these two distinct sources of
"innateness"? Would this further endow the "pixel-wise" color
perception with vague, hard to articulate sense of innateness?
I think... probably?
Does the weirdness of color comes from this fusion? Color has
innate valence, but also "pixel-like" aspects of vision.
There is no other sense that operates in quite this way.
Somatosensation, which contains many different touch receptors,
may be the closest analogue. And, indeed, feels the closest
(qualitatively) to the content of color vision.
… To me, at least.
