Neuroscience, Manifolds

This has been bothering me fore some time, and rather than go through and read the literature I'm just going to dump speculation here.

It seems like it should be possible to derive a general theory for embedding cortical maps. At its simplest, I am referring the to the problem of embedding manifolds with arbitrary topology into the surface of the brain. I understand that "low distortion embeddings" of high dimensional spaces are reasonably well studied, and I think in some instances it might be as simple as naïvely applying mathematical notions of "low distortion embedding" to embedding of manifolds in cortex.

( Side note : in an earlier conversation with Beck, it was noted that, if your space is high dimensional, and your points few, arbitrary embedding is about as good as the optimal low distortion embedding. I think there definitely are high dimensional spaces that only need to encode a relatively sparse set of points that are pretty much randomly organized. Olfactory bulb may be an example : 1000 dimensional vector space, and most attempts to make some sort of map on the surface of the olfactory bulb have failed. However, this could simply mean that high dimensional spaces never embed with low distortion, so random embeddings are getting close to optimal, but optimal is still bad. )

Anyway, places where you typically want to think about low distortion embeddings : primary sensory areas are somewhat obvious, and retinotopic, somatotopic, and tonotopic maps are well studied.

So, what I'm talking about here is more interesting than say, the problem of embedding a spherical globe into a two dimensional map. Visual and somatosensory data have an obvious manifold structure because they are coming from manifold sensory organs. However, the information carried in these sensory streams has a more complex structure, and we ultimately see organization in cortex that reflects this structure.

Lets use the visual system for an example. First, visual information is coming in from the retina, which is to first approximation a hemispherical sheet. Ignore foveal magnification, and just say that this sheet basically ends up being squished, stretched, and flattened onto most primary and secondary visual processing areas. The shape changes, but neighborhoods are preserved.

But, theres also all this natural structure in the information coming from the retina. First of all, you've got brightness, yellow-blue opponancy, and red-green opponancy, so that's three channels effectively forming our familiar three dimensional color space. I'm not sure the brain actually does anything particularly fancy with color information, actually, but basically whats coming into the brain is already this kind of function from a disc to three dimensional color space f:ℝ²→ℝ³.

The really interesting thing about embedding visual space in cortex happens when you start trying to extract low level features. Forget about color for now, its confusing. For now, lets just assume that these low level features are oriented edges. We have to represent a function from the visual field ℝ² (or maybe ℂ would do, or ℝ⁺×𝕋, you know, something two dimensional) to the circle group (apparently called 𝕋). I'm being vague here: something that looks like ℝ²→𝕋.

If you've made it this far and aren't enraged by my bastardized notation, you might have noticed that I'm dropping a component from this visual-orientation space, which is the salience of an oriented edge. The brain represents this as firing rate, but theres no immediately obvious reason why salience should be the component that gets represented in firing rate, and not, say, orientation. Its obvious that firing rate would not work for representing location in the visual field, since it's quite common to have two points in the visual field contain bars of the same orientation, but its impossible for one point in the visual field to contain two bars of the same orientation but different salience.

Incoming visual information on oriented edges takes on the form f:ℝ²→(ℝ⁺×𝕋), so we end up needed to embed a space shaped like ℝ²×(ℝ⁺×𝕋) into cortex, which can be represented simply as a function from a manifold sheet (cortex) to a positive* scalar firing rate** f:ℝ²→ℝ⁺. I'm not sure how to state this formally, but it seems natural that when embedding f:ℝ²→(ℝ⁺×𝕋) in f:ℝ²→ℝ⁺ the ℝ²×𝕋 (orientation) information is going to have to get flattened into ℝ², preferably with minimal distortion.

There is no uniquely optimal way to choose this embedding***. This is evidenced by the fact that orientation selective patches end up forming bands in some animals, and neat little hexagonal "hypercolumns" in others, and sometimes even a mixture of both [citation needed]. The problem is complicated, of course, by the fact that orientation maps aren't the only thing being embedded in the primary visual cortex. In realty, the space you are trying to embed still contains color information, and rather than oriented edges you have this over-complete space of temporally modulated Gabor wavelets****, all of which still needs to get squished into f:ℝ²→ℝ⁺. Oh, also there are two eyes that need to fit into one cortex, hence the ocular dominance columns.

Naïve models of so called "orientation column" formation consider simply the problem of embedding ℝ²×𝕋 in ℝ². These models can reproduce some of the orientation maps we see, but are unsatisfactory. Whenever I run simulations (code lost, hearsay) of this phenomena, I get disorganized columns that eventually converge to stripes if I let the simulation sit long enough. We do see this in some animals, but in many species orientation preference has a crystalline hexagonal packing. At some point in the past, this was thought to indicate a regular periodic organization of cortex. We now know***** that this structure is due simply to the learning rules and the act of embedding ℝ²×𝕋 in ℝ².

Ok, yeah, I'm out of ideas here. I guess I'll leave off with : I'm not sure if anyone has tried to model embedding the space of complex cell receptive fields into ℝ², or tried to construct a nice story about why the space might be embedded as we observe. I'm also not sure if anyone has successfully reasoned about how significantly more complex spaces might end up embedded in cortex. I think in areas like IT, which is supposed to respond to specific objects, the reaction is "well, the space of possible objects is so ridiculously complex theres no way you could flatten it reasonably, so its probably just all mashed in there". Perhaps there are spaces with intermediate complexity that we can look at, perhaps interesting spaces over in parietal lobe that partially represent both visual and motor spaces.

I... guess I'll go try to read more papers.

*its actually non-negative but ℝ∖ℝ⁻ wasn't as stylish. can we just exclude 0 due to "spontaneous spiking" ?

**many simplifications. First of all, I'm not sure we can prove its even firing rate and not something like spike timing that neurons are using to code, second of all neurons have receptive fields that depend on the modulation of a stimulus in time. So, time is another dimension here that I have no idea how to treat formally ( if you can call this nonsense formal ).

***I guess

****I'd say its not completely clear that V1 complex cells _are_ temporally modulated gabor wavelets, but rather that they seem to more or less resemble such wavelets, so we just stopped right there and declared the problem solved.

*****by "we now know" I mean "i assume, I'll look for a reference later"


  1. So in some sense, the organizing idea of Algebra is that, to study some type of structure, you should study the morphisms that preserve that structure.

    For instance if you want to study groups, you should study homomorphisms between groups -- this lends itself naturally to definitions like subgroup and normal subgroup, and towards the idea of factoring groups. It also frequently gives you ways to transport your questions about one group to another group where they might be easier.

    That's a pretty small nutshell but it goes pretty far, it seems, and the idea has been enormously influential. Even in areas not so "rigid" as groups, for instance, graph theory, this idea that we should study graphs by studying morphisms of graphs encompasses a tremendous body of work -- paths in graphs are images of the line graph, clique numbers and coloring numbers are questions about homomorphisms between a graph and a clique. More of the work in the graph theory end is probabilistic/combinatorial in nature -- people consider random graphs and can say meaningful things, and can count numbers of homomorphisms of certain types etc., whereas no one really knows how to say anything meaningful or interesting about a random group it seems.

    There's a lot of math that is not algebra. I'm not sure that I really understand the organizing principles behind for instance Dynamical Systems, I believe that the mode of analysis is substantially different. I thought that, mostly, people do local analysis of systems, by linearizing them or something, and considering perturbations. When A says, we only model some of the dynamics because otherwise it is intractable, to be honest I don't really know how you do that or what that means. Does this refer to dropping off higher order terms? And justifying it using Taylor expansions / the related theorem? There's also various arguments along the lines of, this approximation works in this regime, this works in that regime... which ones scale well and which ones scale badly... I don't really know how at a high level people have investigated dynamical systems. I don't think I have been exposed to any of the "really good ideas".

    I saw a talk by Peter Sarnak where he was investigating what he called "Mobius Randomness", which is the inability of certain very simple dynamical systems with some sort of output to correlate at all with the Mobius function. A major theme of the talk was actually using continuous functions to create morphisms of dynamical systems and analyze them with a very algebraic bent. But I think this is very atypical for a study of dynamical systems.

  2. Oh right,

    So I was just trying to say, I (predictably) like this idea of morphisms of brains.

    Even if you aren't sure what you want to call a brain or how you want to limit this class, if you have a pretty good idea what you want to call a morphism (!) then you can probably figure out what are some things you can and cannot prove in such a model.

  3. So, I remember reading about this a while ago. The way I've phrased the problem, I'm squishing vector valued functions-on-manifolds into a low dimensional scalar functions-on-ℝ²

    But, I've seen other papers go backwards where they try to kind of twist up ℝ² to "cover" the space of the complex, higher dimensional manifold. So, there is literature here.

    There are other issues here, like, how the embeddings seem to frequently capture the statistical structure of inputs. Denser parts of a distribution will be given more space, and two units that are correlated on input "want" to be close to each other in cortical space ( if any neuroscientists are reading this, they know what I just said is bullshit, but I'll try to make myself more clear later, hopefully ).

    so, for example, one of the reasons that ℝ²×𝕋 might be embedded ℝ² as hexagonally packed hyper-columns, rather than simple bands ( where I pick a direction in ℝ² and just kind of rotate the phase in 𝕋 over the surface in that direction ) is that oriented edges rarely occur in isolation. Typically, you're observing an extended edge. This means that theres really additional structure, and correlated cells are going to try to be near each other. Well, correlated edge detectors simply lie along lines in cortex. If I were to try to embed my orientation map as simple bands, invariably only one set of edge detectors would end up lining up. So, it turns out that a kind of packed wheel arrangement might be better at impartially wiring together edge detectors to pick up extended edges. I think this might be pure speculation, since obviously not all animals have orientation embedded in such a way. Maybe some other structure in the data dominates over orientation selectivity in these maps, or maybe the column-like embedding really is just an artifact.

    Anyway, I'm not sure "morphism" from group or graph theory is really all that similar to what I'm describing here. I'm trying to take a high dimensional space and just say "whats a good way of shoving this into a brain". So, I guess that action of "shoving" is a morphism, between an abstract representation of information and the real representation in the brain, but I'm not exactly talking about morphisms between brain regions.

    Maybe I can say something like "I've got two bits of brain, and really, both bits of brain are the same, and the information they are representing is similar by this morphism, so I can say that a map for this information should look kinda of like a map for that information in these ways", but I'm not sure how useful that is ( besides, I guess, reducing the overall complexity of thinking about brains by not worrying about the actual nature of the information, just it's shape ), thats more of an immediate corollary to being able to understand how complex spaced are embedded in two dimensional cortex.

  4. I'm fairly certain that the original content of the post is well studied, but I doubt I'll have time to dig into that any time soon.

    A final related thought : Do you remember when I was trying to say that cortical connectivity might be describable by an automorphism from some ridiculously complex manifold to itself ?

    I think this might still be a useful way of thinking about things. First, it requires that you have enough neurons that you can ignore individual units. Instead of talking about neurons in a layer of cortex, you just say "this is a sheet". Instead of talking about neurons in a nucleus, you just say "this is a ball". Petty simple. Then, the general shape of the connections in the central nervous system can be described as a function from this manifold to itself.

    The reason this might actually be useful is that this automorphism might be far simpler to store and evaluate than the complete connectivity matrix of the human brain. When we were doing neural simulations, it was always the connectivity matrix that killed you.

    I figured, changing it from a set of points to a manifold, and changing the directed edges in the graph for a function from the manifold to itself, might simplify things.

    And there is good evidence that properties in cortex are differentiable, in the sense that neurons next to each other have very similar properties, so I do think the structure of the nervous system lends naturally to this simplification.

    Anyway, representing connectivity in this manner might let you say, project to a neighborhood, then you only need to store connection weights within a small neighborhood.

    Or maybe I should get more sleep and less coffee.

  5. p.s. : math people I'm sorry, I don't actually know math, but I'd like to some day.

  6. An interesting mathematical problem that comes out of generalizing this sort of questions is: you have a big data set in some high-dimensional space and you think that actually, this thing is a manifold or other topological space which has far fewer intrinsic dimensions. The tool we use for this is persistent homology (a nice introductory paper by Gunnar Carlsson.) Actually I should keep this in mind as a future post here, once I research it more: a basic explanation of persistent homology and the currently known applications to neuroscience (there are) and ones we can speculate about.

    For now though, the point is that if we have points in a space, we can connect them into a simplicial complex, but how do we know how close the points need to be for us to connect them? We don't (moreover there might not be any one answer since our data might be fractal) but we can vary this distance and structure that stays over a large range of distance is actually probably there. Structure meaning homology, which is not that difficult to compute on simplicial complexes, although it's a rather coarse way of measuring structure.

  7. hmm... seems like the blackboard T character I was using for the circle group only works in chrome, on Ubuntu.

  8. just came across the line "do these high level spaces maintain topology"

    In reference to .. something about neuroscience. I'm not sure what they mean, but this suggests that there might be a whole wonderful world of topological neuroscience out there.