Paper Review : Neural Plasticity and Consciousness

For neuroscientists, treating "The Hard Problem of Consciousness" outside of bar-room speculation is a risky career move. This is why we have true doctors of philosophy, and why the philosophy paper "Neural Plasticity and Consciousness" by Susan Hurley and Alva Noë is a good thing. Hurley and Noë's thesis relates to some recent activity on WeAlone [1,2, maybe 3] , so I will attempt to summarize the article in a language that makes most sense to me.

First Hurley and Noë note that the "hard problem of consciousness" is equivalent to what they call an "absolute gap", i.e. "why should we assume that neural activity is solely responsible for conscious perception at all ?". My interpretation is that Hurley and Noë say "we can't, this is a leap of faith", and for the purposes of the paper accept as an axiom that neural activity corresponds to perception. The meat of the paper then, discusses why some neural activity should take on a particular quality, like seeing, and other neural activity should take on a distinct quality, like hearing.

Lately, I've been throwing around the term "neural topology" and "manifold structure" in an embarrassingly non-rigorous manner. I'd like to say "the topology of qualia acquires the topology of stimuli via learning of the intrinsic statistical structure of the stimuli, and in a sense, the stimulus stimulus model constitutes the nature of qualia", but this is vague. Hurley and Noë express, I believe, a similar sentiment clearly and without abusing terms from mathematics :

It is argued that the different characteristics of input activity from specific sources (visual vs. auditory) generate not just representational structure specific to that source but also source-specific sensory and perceptual qualities.

That is to say, when the brain learns the topology of stimuli ( possibly in union with the topology of motor outputs as they modify stimuli ), the brain acquires the qualia corresponding to said stimuli.

Earlier we talked about the possibility of defining an algebraic structure representing the shape of information coded in the brain. The take-away point was that it might be possible to rigorously say "these two areas have effectively the same abstract structure, since you can relate them by some structure preserving relationship". The Hurley and Noë paper provides anecdotes which suggest that, when two physically distinct neural circuits have the same abstract structure (topology), then the subjective experience (qualia) are also the same. Specifically, they discuss experiments in which blind patients were able to acquire visual qualia through a tactile stimulation device that translates camera images into stimulation of the skin.

After a period of adaptation (as short as a few minutes), subjects report perceptual experiences that are distinctively non-tactile and quasi-visual. … However, Bach-y-Rita emphasizes that the transition to quasi-visual perception depends on the subject’s exercising active control of the camera. … Perceivers can acquire and use practical knowledge of the common laws of sensorimotor contingency that vision and TVSS-perception share. For example, as you move around an object, hidden portions of its surface come into tactile-visual view, just as they would if you were seeing them."

This experiment suggests that giving a system a new topology induces qualia of that topology, and that learning the new topology does not necessarily require expensive and lengthly re-wiring. That camera control was necessary for inducing visual qualia from tactile stimulation suggests that the structure of visual stimuli and the experience of seeing must necessarily incorporate how our actions : movement of the eyes and head, and translation in space, alter the content of visual stimuli. Thus, when we talk about the "topology" of a stimulus, we must also incorporate how our actions change the stimulus (how our motor operators transform the stimulus space).

Hurley and Noë cover a number of other interesting anecdotes, including what happens when the brain fails to adapt its structure to reality, and pointing out that, in a left-right reversal of vision, reversing the interpretation of visual data is topologically equivalent to reversing the coordinates of motor output and proprioception, such that many different possible explanations of neural adaptation may be topologically equivalent.

So, I really do feel like, if we can make this notion of "neural topology*" more rigorous, we will have a satisfying answer to the portion of "the hard problem" that is amenable to scientific and mathematical investigations.

*neural topology : the idea that, in high dimensional sensory spaces, the distribution of probable stimuli occupy a reduced subset of said high dimensional space, and that one can move about this subset in a differentiable manner to transition smoothly between probable stimuli. This is a vague notion. It is related to "statistical structure" and "manifold", although I should note that we don't have enough information to say that the space of probably sensory-motor states is actually a manifold.

Neuroscience, Manifolds

This has been bothering me fore some time, and rather than go through and read the literature I'm just going to dump speculation here.

It seems like it should be possible to derive a general theory for embedding cortical maps. At its simplest, I am referring the to the problem of embedding manifolds with arbitrary topology into the surface of the brain. I understand that "low distortion embeddings" of high dimensional spaces are reasonably well studied, and I think in some instances it might be as simple as naïvely applying mathematical notions of "low distortion embedding" to embedding of manifolds in cortex.

( Side note : in an earlier conversation with Beck, it was noted that, if your space is high dimensional, and your points few, arbitrary embedding is about as good as the optimal low distortion embedding. I think there definitely are high dimensional spaces that only need to encode a relatively sparse set of points that are pretty much randomly organized. Olfactory bulb may be an example : 1000 dimensional vector space, and most attempts to make some sort of map on the surface of the olfactory bulb have failed. However, this could simply mean that high dimensional spaces never embed with low distortion, so random embeddings are getting close to optimal, but optimal is still bad. )

Anyway, places where you typically want to think about low distortion embeddings : primary sensory areas are somewhat obvious, and retinotopic, somatotopic, and tonotopic maps are well studied.

So, what I'm talking about here is more interesting than say, the problem of embedding a spherical globe into a two dimensional map. Visual and somatosensory data have an obvious manifold structure because they are coming from manifold sensory organs. However, the information carried in these sensory streams has a more complex structure, and we ultimately see organization in cortex that reflects this structure.

Lets use the visual system for an example. First, visual information is coming in from the retina, which is to first approximation a hemispherical sheet. Ignore foveal magnification, and just say that this sheet basically ends up being squished, stretched, and flattened onto most primary and secondary visual processing areas. The shape changes, but neighborhoods are preserved.

But, theres also all this natural structure in the information coming from the retina. First of all, you've got brightness, yellow-blue opponancy, and red-green opponancy, so that's three channels effectively forming our familiar three dimensional color space. I'm not sure the brain actually does anything particularly fancy with color information, actually, but basically whats coming into the brain is already this kind of function from a disc to three dimensional color space f:ℝ²→ℝ³.

The really interesting thing about embedding visual space in cortex happens when you start trying to extract low level features. Forget about color for now, its confusing. For now, lets just assume that these low level features are oriented edges. We have to represent a function from the visual field ℝ² (or maybe ℂ would do, or ℝ⁺×𝕋, you know, something two dimensional) to the circle group (apparently called 𝕋). I'm being vague here: something that looks like ℝ²→𝕋.

If you've made it this far and aren't enraged by my bastardized notation, you might have noticed that I'm dropping a component from this visual-orientation space, which is the salience of an oriented edge. The brain represents this as firing rate, but theres no immediately obvious reason why salience should be the component that gets represented in firing rate, and not, say, orientation. Its obvious that firing rate would not work for representing location in the visual field, since it's quite common to have two points in the visual field contain bars of the same orientation, but its impossible for one point in the visual field to contain two bars of the same orientation but different salience.

Incoming visual information on oriented edges takes on the form f:ℝ²→(ℝ⁺×𝕋), so we end up needed to embed a space shaped like ℝ²×(ℝ⁺×𝕋) into cortex, which can be represented simply as a function from a manifold sheet (cortex) to a positive* scalar firing rate** f:ℝ²→ℝ⁺. I'm not sure how to state this formally, but it seems natural that when embedding f:ℝ²→(ℝ⁺×𝕋) in f:ℝ²→ℝ⁺ the ℝ²×𝕋 (orientation) information is going to have to get flattened into ℝ², preferably with minimal distortion.

There is no uniquely optimal way to choose this embedding***. This is evidenced by the fact that orientation selective patches end up forming bands in some animals, and neat little hexagonal "hypercolumns" in others, and sometimes even a mixture of both [citation needed]. The problem is complicated, of course, by the fact that orientation maps aren't the only thing being embedded in the primary visual cortex. In realty, the space you are trying to embed still contains color information, and rather than oriented edges you have this over-complete space of temporally modulated Gabor wavelets****, all of which still needs to get squished into f:ℝ²→ℝ⁺. Oh, also there are two eyes that need to fit into one cortex, hence the ocular dominance columns.

Naïve models of so called "orientation column" formation consider simply the problem of embedding ℝ²×𝕋 in ℝ². These models can reproduce some of the orientation maps we see, but are unsatisfactory. Whenever I run simulations (code lost, hearsay) of this phenomena, I get disorganized columns that eventually converge to stripes if I let the simulation sit long enough. We do see this in some animals, but in many species orientation preference has a crystalline hexagonal packing. At some point in the past, this was thought to indicate a regular periodic organization of cortex. We now know***** that this structure is due simply to the learning rules and the act of embedding ℝ²×𝕋 in ℝ².

Ok, yeah, I'm out of ideas here. I guess I'll leave off with : I'm not sure if anyone has tried to model embedding the space of complex cell receptive fields into ℝ², or tried to construct a nice story about why the space might be embedded as we observe. I'm also not sure if anyone has successfully reasoned about how significantly more complex spaces might end up embedded in cortex. I think in areas like IT, which is supposed to respond to specific objects, the reaction is "well, the space of possible objects is so ridiculously complex theres no way you could flatten it reasonably, so its probably just all mashed in there". Perhaps there are spaces with intermediate complexity that we can look at, perhaps interesting spaces over in parietal lobe that partially represent both visual and motor spaces.

I... guess I'll go try to read more papers.

*its actually non-negative but ℝ∖ℝ⁻ wasn't as stylish. can we just exclude 0 due to "spontaneous spiking" ?

**many simplifications. First of all, I'm not sure we can prove its even firing rate and not something like spike timing that neurons are using to code, second of all neurons have receptive fields that depend on the modulation of a stimulus in time. So, time is another dimension here that I have no idea how to treat formally ( if you can call this nonsense formal ).

***I guess

****I'd say its not completely clear that V1 complex cells _are_ temporally modulated gabor wavelets, but rather that they seem to more or less resemble such wavelets, so we just stopped right there and declared the problem solved.

*****by "we now know" I mean "i assume, I'll look for a reference later"