Showing posts with label graphics. Show all posts
Showing posts with label graphics. Show all posts

20111101

Visual analogue of a Shepard tone

A Shepard tone is an auditory illusion that appears to indefinitely ascend or descend in pitch, without actually changing pitch at all.
 
Shepard tones work because they actually contain multiple tones, separated by octaves. As tones get higher in pitch, they fade out. New tones fade in at the lower pitches. The net effect is that it sounds like all the constituent tones are continually increasing in pitch -- and they are, but pitches fade in and out so that, on average, the pitch composition is constant.

Since 2D quasicrystals can be rendered as a sum of plane-waves, it is possible to form the analogue of a Shepard tone with these visual objects. Each plane wave is replaced with a collection of plane waves, at 2,4,8,16... etc times the spatial frequency of the original plane wave.

The relative amplitudes of the plane waves are set so that the spatial frequency stays approximately the same even as the underlying waves are scaled. The result is a quasicrystal that appears to zoom in or out indefinitely, without fundamentally changing in structure.

The infinite zoom effects creates a motion-fatigue optical illusion, which will cause illusory contraction of your visual field after staring at the GIF below:
 
 

 

More quasicrystal zoom GIFs can be found here. You can run and modify the code I used to generate these animation. Copy the following code into a file called QuasiZoom.java. Then, in a terminal, type "javac QuasiZoom.java" in the same directory, and then "java QuasiZoom". Various parameters to tune the output are noted in comments in the code. Then use Gimp to make an animated GIF.

import java.awt.Color;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
import static java.lang.Math.*;

public class QuasiZoom {

    // Defines a gaussian function. We will use this to define the
    // envelope of spatial frequencies
    public static double gaussian(double x) {
        return exp(-x*x/2)/sqrt(2*PI);
    }

    public static void main(String[] args) throws IOException {
        int k = 5;        //number of plane waves
        int stripes = 3;  //number of stripes per wave
        int N = 500;      //image size in pixels
        int divisions=40; //number of frames to divide the animation into
        int N2 = N/2;

        BufferedImage it = new BufferedImage(N, N, BufferedImage.TYPE_INT_RGB);

        //the range of different spatial frequencies
        int [] M=new int[]{1,2,4,8,16,32,64,128,256};
        
    //the main ( central ) spatial frequency
        double mean=log(16);

    //the spread of the spatial frequency envelope
        double sigma=1;

    //counts the frames 
        int ss=0;

    //iterate over spatial scales, scaling geometrically
        for (double sc=2.0; sc>1.0; sc/=pow(2,1./divisions)) 
        {    
            System.out.println("frame = "+ss);

            //adjust the  wavelengths for the current spatial scale
            double [] m=new double[M.length];
            for (int l=0; l<M.length; l++)
                m[l]=M[l]*sc;

            //modulate each wavelength by a gaussian envelop in log
            //frequency, centered around aforementioned mean with defined
            //standard deviation
            double sum=0;
            double [] W=new double[M.length];
            for (int l=0; l<M.length; l++) {
                W[l]=gaussian((log(m[l])-mean)/sigma);
                sum+=W[l];
            }
            sum*=k;

            for (int i = 0; i < N; i++) {
                for (int j = 0; j < N; j++) {

                    double x = j - N2, y = i - N2; //cartesian coordinates
                    double C = 0;                  // accumulator
 
                    // iterate over all k plane waves
                    for (double t = 0; t < PI; t += PI / k){
                        //compute the phase of the plane wave
                        double ph=(x*cos(t)+y*sin(t))*2*PI*stripes/N;
                        //take a weighted sum over the different spatial scales
                        for (int l=0; l<M.length; l++)
                            C += (cos(ph*m[l]))*W[l];
                    }
                    // convert the summed waves to a [0,1] interval
                    // and then convert to [0,255] greyscale color
                    C = min(1,max(0,(C*0.5+0.5)/sum));
                    int c = (int) (C * 255);
                    it.setRGB(i, j, c | (c << 8) | (c << 16));
                }
            }
            ImageIO.write(it, "png", new File("out"+(ss++)+".png"));
        }
        
    }
}


20111024

Animating quasicrystals

This post is inspired by the recent Nobel prize for the discovery of quasicrystals in nature.

We spoke briefly of quasicrystals previously : patterns that appear somewhat periodic but never actually repeat. Two-dimensional quasicrystals can be generated by summing together four or more plane waves. In collaboration with Keegan, we've explored what happens when you make these plane waves travel, and also viewing the quasicrystals in logarithmic coordinates as shown in this earlier post.

 

The Java code below generated sequences of images for animation. Once you have a series of images, you can use Gimp to make an animated GIF. (Convert to black-and-white and 8-color indexed mode before saving to reduce file size.)

import java.awt.image.BufferedImage;  
 import java.io.File;  
 import java.io.IOException;  
 import javax.imageio.ImageIO;  
 import static java.lang.Math.*;  
 public class QUASI1 {  
   public static void main(String [] args) throws IOException {  
     int k=4;     //numer of plane waves  
     int stripes = 27; //number of stripes per wave  
     int N = 800;   //image size in pixels  
     int N2 = N/2;  
     BufferedImage it = new BufferedImage(N,N,BufferedImage.TYPE_INT_RGB);  
     for (double phase=0; phase<2*PI; phase+=2*PI/30) {  
       for ( int i=0; i<N; i++ ) for ( int j=0; j<N; j++ ) {  
         double x = j-N2, y = i-N2; //cartesian coordinates  
         double theta = atan2(y,x); //log-polar coordinates  
         double r = log(sqrt(x*x+y*y));  
         double C=0;        // accumulator  
         for (double t=0; t<PI; t+=PI/k)  
           C+=cos((theta*cos(t)-r*sin(t))*stripes+phase);  
           // use the following line for cartesian crystals:  
           //C+=cos((x*cos(t)+y*sin(t))*2*PI*stripes/N+phase);  
         int c=(int)((C+k)/(k*2)*255);  
         it.setRGB(i,j,c|(c<<8)|(c<<16));  
       }  
       ImageIO.write(it,"png",new File("Test"+(int)(180*phase/PI)+".png")) ;  
     }  
   }  
 }  

To execute this code, copy it into a file names "QUASI1.java". Then, from the terminal, run "javac QUASI1.java", and finally "java QUASI1" to execute it. Or, paste it into your favorite Java IDE.

Changing K will change the degree of symmetry in the crystal. Changing N sets the size of the output images. Changing stripes sets how many wave cycles fit in the rendered image, with larger numbers leading to finer structure.

Since quasicrystals are aperiodic, it is not possible to wrap them around the log-polar "tunnel" ( seen on the left above ) such that two edges of the image meet perfectly. However, for quasicrystals composed of only a few plane waves, you can sometimes get two regions to align well enough to be unnoticeable, especially in black and white.
 
Be sure to check out Keegan's implementation, which doubles as a nice introduction to coding in Haskell.


20110721

Fractals on the Master Boot Record

WeAlone contributor Keegan has adapted the video feedback method of rendering Julia sets to fit in 512 bytes of Intel machine code that runs from the master boot record. This program was created for the IO MBR demo competition.



When a computer starts up, a very small program begins the process of loading and booting up progressively more complex programs, until an entire modern operating system is loaded. With some cleverness and optimization, we were able to make a program that fits in this space, and rather than booting the machine, renders animated Julia set fractals.

The source code is written in assembly, and can be downloaded here. The compiled program image can be downloaded from here.

If you're running Linux, you can try this out yourself using the qemu machine emulator, which can be retrieved from the package manager ( menu → system → administration → synaptic package manager, search for and install 'qemu-kvm' ). Once installed, simply typing "qemu phosphene.mbr" in a terminal should suffice.

You can also create a USB thumb drive that can boot most Intel architecture machines into this fractal rendering mode. Once booted, it is possible to remove the USB stick and leave the machine in a fractal-rendering coma until it is power cycled. Be careful here, if you overwrite the MBR on your own machine, you will trash your partition table and leave your system only able to boot as a fractal.

Assuming your USB thumb drive is /dev/sdb, the following commands will create a bootable USB stick. For the love of humanity do not write to /dev/sda, since this is probably your system boot partition.

$: sudo dd if=phosphene.mbr of=/dev/sdb
$: sync
( wait until IO lights stop blinking and remove the drive )

In one instance, we found that writing to /dev/sdb1 worked while writing to /dev/sdb did not. I'm not sure if this was a fluke, but you can try this if it doesn't seem to work on the first try.

These programs are so small, they can be distributed as plaintext in base64.


with a more flickery, rapidly changing colorscheme :

McCO2I7AuOAHjtC8ABC4EwDNELsPALQOvpl9rM0QhMB1+R5oAKAfMfa/AJ65AArzpR8xwM0QuAFP
uQEBvwB+zRCwQDHS93UEULgCT7sBAc0QusgDMMDuQrkAAVDuiNjuiPjuWAQCgcMDBeLv2+PHBYUA
3wXGBbHfBb0AAWgAEA+hFh+M4IDEEI7AvgAgv8B4sRCs0OgmAAVH4veBxjABgcfwAYH/AJhy5zH2
Mf+M4ID0UI7AiSzfBNjy2cDZ/9nJ2ereydn/2ejYwNz63vnZ7d7puoABuQACYNnqiRTfBNj12OGJ
DN8E2PXY4t3S3MrZwdjI3uveydjA2MPZwt7C2ercwt7B2M3fHIsU2MvfHIscgOcBieiB+oABcxXR
4sDmBIzgAPSO6DD2weIIAdNligcmiAVhR+Kghf91B4zAgMQQjsBKdY9F3tmM4ID0UI7gjthoAKAH
MdLoNgAx9r9AeLuAAbkAAqSF/3UKNgIW/g9T6B0AW+LvgceAAEv2w391CYzYgMQQjtgx9oXbddXp
8P64BU8x280Qw0kDDQppbwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////////////
////////////////////////////////////////////////////////////////////////Vao=



A nice colorscheme with a black background :

McCO2I7AuOAHjtC8ABC4EwDNELsPALQOvp99rM0QhMB1+R5oAKAfMfa/AJ65AArzpR8xwM0QuAFP
uQEBvwB+zRCwQDHS93UEULgCT7sBAc0QusgDMMDuQrkAAVDuiNjuiPjuWAQCgcMDBeLv2+PHBYUA
3wXGBbHfBb0AAWgAEA+hFh+M4IDEEI7AvgAgv8B4sRCs0OgmAAVH4veBxjABgcfwAYH/AJhy5zH2
Mf+M4ID0UI7AiSzfBNjy2cDZ/9nJ2ereydn/2ejYwNz63vnZ7d7puoABuQACYNnqiRTfBNj12OGJ
DN8E2PXY4t3S3MrZwdjI3uveydjA2MPZwt7C2ercwt7B2M3fHIsU2MvfHIscgOcBMMCB+oABcxvR
4sDmBIzgAPSO6DD2weIIAdNligc8/3QC/sAmiAVhR+Kahf91B4zAgMQQjsBKdYlF3tmM4ID0UI7g
jthoAKAHMdLoNgAx9r9AeLuAAbkAAqSF/3UKNgIW/g9T6B0AW+LvgceAAEv2w391CYzYgMQQjtgx
9oXbddXp6v64BU8x280Qw0kDDQppbwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////////////
////////////////////////////////////////////////////////////////////////Vao=


To convert these strings into a usable program, in Linux, use the base64 command. Type "base64 -d > phosphene.mbr" in the terminal, and press enter. Then, paste one of the base 64 encoded programs in the terminal. Press enter, and then control+D ( end of file ). This will convert the text into the compiled machine code for phosphene.mbr. Run it as explained above using qemu or making a bootable USB drive.

$: base64 -d > foo.mbr
McCO2I7AuOAHjtC8ABC4EwDNELsPALQOvp99rM0QhMB1+R5oAKAfMfa/AJ65AArzpR8xwM0QuAFP
uQEBvwB+zRCwQDHS93UEULgCT7sBAc0QusgDMMDuQrkAAVDuiNjuiPjuWAQCgcMDBeLv2+PHBYUA
3wXGBbHfBb0AAWgAEA+hFh+M4IDEEI7AvgAgv8B4sRCs0OgmAAVH4veBxjABgcfwAYH/AJhy5zH2
Mf+M4ID0UI7AiSzfBNjy2cDZ/9nJ2ereydn/2ejYwNz63vnZ7d7puoABuQACYNnqiRTfBNj12OGJ
DN8E2PXY4t3S3MrZwdjI3uveydjA2MPZwt7C2ercwt7B2M3fHIsU2MvfHIscgOcBMMCB+oABcxvR
4sDmBIzgAPSO6DD2weIIAdNligc8/3QC/sAmiAVhR+Kahf91B4zAgMQQjsBKdYlF3tmM4ID0UI7g
jthoAKAHMdLoNgAx9r9AeLuAAbkAAqSF/3UKNgIW/g9T6B0AW+LvgceAAEv2w391CYzYgMQQjtgx
9oXbddXp6v64BU8x280Qw0kDDQppbwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////////////
////////////////////////////////////////////////////////////////////////Vao=
$: qemu foo.mbr


20101121

Fractals

The folks at fractalforums.com have been rendering more of those crazy alien-cyborg-city-spaceship ray-traced fractals. I don't understand their algorithms but their "mandelbulb" software is free for download. Perfect for zoning out for a little brain-massage.









And I thought I was just going to pop over to youtube to grab this little wheel illusion video. The fractals are much more entertaining.


20100331

Quasicrystal

Quasicrystals are aperiodic structures that still have some regularities. Both the spatial and frequency domain representations of quasicrystals are quite beautiful. See also this more recent post for more quasicrystal entertainment. Here is the classic diffraction pattern from a synthesized icosahedral quasicrystal

You can create two-dimensional quasicrystals by summing together more than 3 plane waves. For example, here are some spatial domain two-dimensional quasicrystals :

4-fold :
5-fold :
7-fold :

[overflow gallery]


The picture at the beginning of this post is an x-ray diffraction pattern, which is a little bit like the Fourier transform of a quasicrystal lattice. Here is a simulated 2D Fourier spectrum from a 7-fold 2D quasicrystal :

You can create your own images like this by rendering large two-dimension quasicrystals as described here or here. The frequency domain images can be rendered using ImageJ, or your own personal 2d FFT code. In ImageJ, open an image, and then create a frequency domain image by clicking Process→FFT→FFT. If you rendered a N-fold quasicrystal, you should see 2N points arranged in a circle around the origin, corresponding to your N plane waves. If you applied some nonlinear image operation, like contrast enhancement or thresholding, to the spatial domain image, you will have created some harmonics and overtones of your original N plane waves, which should appear as a constellation of other points that themselves are arranges on a quasi-crystal lattice.

Quasicrystals can be related to more abstract mathematical concepts. For instance, this article finds that some quasicrystals are related to sorting algoritms :
Current research in abstract tiling theory examines tilings of high rotational symmetry in collaboration with Remy Mosseri and co-workers. Possibly the limit of high rotational symmetry may prove easier to analyze than specific finite symmetries (10-fold, for example) of direct physical interest. Surprisingly, rhombus tilings are related to algorithms for sorting of lists. Counting the number of distinct tilings enumerates simultaneously the number of equivalance classes of sorting algorithms, a problem previously considered by computer scientist D.E. Knuth. Our random tiling theory implies an upper bound of log(2) for the tiling entropy per vertex, consistent with a conjecture by Knuth. Click here for a preprint on this research.




20080326

I should be doing work

Two days behind and I'm just pointing the web-cam back at the computer screen.


20080124

a ( e^z+e^(iz) ) + c


Adding a rotation and scaling parameter hes increased the variety of fractals that can be observed in Perceptron. Gradient and coloring methods are important for revealing the structure of the maps, and a gradient control parameter has been added. Functions of the form a ( e^z+e^(iz) ) + c, ( a and c complex ) have been providing much amusement.


20080123

a ( e^z+e^(iz) ) + c


Image Analogies

Some awesome stuff here. Check out the "texture-by-numbers" examples in particular.


20070909

Fractal neurofeedback

There's an article on the Mind Hacks blog that overlaps heavily with the kind of stuff we talk about here. Their output looks Sheep-ish; since my realtime Electric Sheep renderer is working now, maybe I'll build an OpenEEG box and bang out an open source alternative.


20070903

More screenshots

Here's the latest.

Edit: I've added some more screenshots to the gallery, with tasty vertical symmetry imposed by mirroring.


Here I'm trying out some different maps, and also incorporating a camera feed, which is what gives it the more fuzzy, organic look. The geometric patterns with n-way radial symmetry come from z' = z*c, which gives simple scaling and rotation. The squished circles come from z' = sin(real(p) + t) + i*sin(imag(p)), where p = z^2 + c and t is a real parameter.


20070901

More fractal video feedback

I've been working on a new implementation of the fractal video feedback idea. Unlike the previous attempts, the code is nice and modular, so complicated bits of OpenGL hackery get encapsulated in an object with a simple interface. It's still very much a work in progress, but I thought I'd share some results now. Feedback (no pun intended) is very much appreciated.

Video:

Shoving the video through the YouTubes kills the quality. I have some higher quality screenshots in a Flickr gallery. Some of my favorites:




The basic idea is the same as Perceptron: take the previous frame, map it through some complex function, draw stuff on top, repeat. In this case, the "stuff on top" consists of a colored border around the buffer that changes hue, plus some moving polygons that can be inserted by the user (which aren't used in the video, but are in some of the stills). In these examples, the map is a convex combination of complex functions; in the video it's z' = a*log(z)*c + (1-a)*(z2+c). Here z is the point being rendered, z' is the point in the previous frame where we get its color, c is a complex parameter, and a is a real parameter between 0 and 1.

There are two modes: interactive and animated. In interactive mode, c and a are controlled with a joystick (which makes it feel like a flight simulator on acid). The user can also place control points in this (c,a) space. In animated mode, the parameters move smoothly between these control points along a Catmull-Rom spline, which produces a nice C1 continuous curve.

The feedback loop is rendered offscreen at 4096x4096 pixels. Since colors are inverted every time through the loop, only every other frame is drawn to the screen, to make it somewhat less seizuretastic. At this resolution, the system has 48MB of state. On my GeForce 8800GTS I can get about 100 FPS in this loop; by a conservative estimate of the operations involved, this is about 60 GFLOPS. I bow before NVIDIA. Now if only I had one of these...


20070725

Extraction of musical structure

I think my next big project will involve automatically extracting structure from music. Mike and I had some discussions about doing this with machine learning / evolutionary algorithms, which produced some interesting ideas. For now I'm implementing some of the more traditional signal-processing techniques. There's an overview of the literature in this paper.

What I have to show so far is this:


This (ignoring the added colors) is a representation of the autocorrelation of a piece of music ("Starlight" by Muse). Each pixel of distance in either the x or y axis represents one second of time, and the darkness of the pixel at (x,y) is proportional to the difference in average intensity between those two points in time. Thus, light squares on the diagonal represent parts of the song that are homogenous with respect to energy.

The colored boxes were added by hand, and represent the musical structure (mostly, which instruments are active). So it's clear that the autocorrelation plot does express structure, although at this crude level it's probably not good enough for extracting this structure automatically. (For some songs, it would be; for example, this algorithm is very good at distinguishing "guitar" from "guitar with screaming" in "Smells Like Teen Spirit" by Nirvana.) An important idea here is that the plot can show not only where the boundaries between musical sections are, but also which sections are similar (see for example the two cyan boxes above).

The next step will be to compare power spectra obtained via FFT, rather than a one-dimensional average power. This should help distinguish sections which have similar energy but use different instruments. The paper referenced above also used global beat detection to lock the analysis frames to beats (and to measures, by assuming 4/4 time). This is fine for DDR music (J-Pop and terrible house remixes of 80's music) but maybe we should be a bit more general. On the other hand, this approach is likely to improve quality when the assumptions of constant meter and tempo are met.

On the output side, I'm thinking of using this to control the generation of flam3 animations. The effect would basically be Electric Sheep synced up with music of your choice, including smooth transitions between sheep at musical section boundaries. The sheep could be automatically chosen, or selected from the online flock in an interactive editor, which could also provide options to modify the extracted structure (associate/dissociate sections, merge sections, break a section into an integral number of equal parts, etc.) For physical installation, add a beefy compute cluster (for realtime preview), an iPod dock / USB port (so participants can provide their own music), a snazzy touchscreen interface, and a DVD burner to take home your creations.


20070718

OpenCV : open-source computer vision

OpenCV is an open source library from Intel for computer vision. To quote the page,

"This library is mainly aimed at real time computer vision. Some example areas would be Human-Computer Interaction (HCI); Object Identification, Segmentation and Recognition; Face Recognition; Gesture Recognition; Motion Tracking, Ego Motion, Motion Understanding; Structure From Motion (SFM); and Mobile Robotics."

Sounds like some of this could be pretty useful for interactive video neuro-art, or whatever the hell it is we're doing.


20070715

Whorld : a free, open-source visualizer for sacred geometry

From the homepage:

"Whorld is a free, open-source visualizer for sacred geometry. It uses math to create a seamless animation of mesmerizing psychedelic images. You can VJ with it, make unique digital artwork with it, or sit back and watch it like a screensaver."


20070627

Idea: music visualization with spring networks

The basic idea is to connect a collection of springs into an arbitrary graph, then drive certain points in this graph with the waveform of a piece of music (possibly with some filtering, band separation, etc.) This could be restrained to two dimensions or allowed unrestricted use of three.

Spring constants could be chosen so the springs resonate with tones in the key of the piece. Choosing these constants and the graph connectivity to be aesthetically pleasing would likely be an art form in of itself. A good starting point would be interconnected concentric polygonal rings of varying stiffness. Symmetry seems like a must.

For a software implementation, a useful starting point would be CS 176 project 5; a cloth simulator that considers only edge terms is essentially a spring-network simulator. There are many ways to render the output; for example, draw nodes with opacity proportional to velocity, and/or draw edges with opacity proportional to stored energy. Use saturated colors on a black background, and render on top of blurred previous frames for a nice trail effect. Since I've already coded the gnarly math once, I might try to throw this together tomorrow evening, if I don't get distracted by something else.

The variations are really endless. For example, with gravity and stiff (or entirely rigid) edges, you could make a chaos pendulum. By allowing edges to dynamically break and form based on proximity and/or energy, you could get all kinds of dynamic clustering behavior, which might look like molecules forming or something.

A hardware implementation (i.e., actual springs) would be badass in the extreme, although I imagine it would be finicky to set up and tune.


Idea: immersive video with one projector

This is an idea I had while lying in bed listening to Radiohead and hallucinating. (I was perfectly sober, I swear. The Bends is just that damn good.)

Build a frame structure (out of PVC or similar) with the approximate width/depth of a bed, and height of a few feet -- enough that you could comfortably lie on a mattress inside and not feel claustrophobic. Cover every side with white sheets, drawn taut. Mount a widescreen projector directly above the middle of this structure, pointing down. Then hang two mirrors such that the left third of the image is reflected 90 degrees to the left and the right third is reflected 90 degrees to the right (from the projector's orientation), with the middle third projecting directly onto the top of the frame. Then use more mirrors to get the left and right images onto the corresponding sides of the frame. (You'd probably also need some lenses to make everything focus at the same time; this is the only part I'm really iffy on. Fresnel lenses would probably be a good choice. Anyone who knows optics and has any idea how to set this up, please let me know.)

Anyway, the beauty of this setup is that it allows one to control nearly the whole visual field with a single projector and a single video output, thus minimizing complexity and expense. It's not hard to set up OpenGL to render three separate images to three sections of the screen; they could be different viewpoints in the same 3D scene, although as usual I'm more interested in the more abstract uses of this. In particular, you get control over both central and peripheral vision, which has psychovisual importance.

I'm really tempted to build this when I get back to Tech, but there's a high probability that someone else's expensive DLP projector will suffer an untimely demise at the hands of improvised mounting equipment.

Edit: I thought of an even simpler setup that does away with the mirrors and lenses. Make the enclosure a half-cylinder, and project a single widescreen image onto it (orienting left-right with head-feet), correcting for cylindrical distortion in software. The major obstacle here is making a uniformly cylindrical projection surface, but that shouldn't be too hard.


20070423

More on crowd feedback

Everyone has cellphones now, right? If you had a few highly directional antennae you might be able to use the amount of RF activity in a few cellphone bands as an approximation to crowd activity. You could maybe also look for Bluetooth phones and maybe remember individual Bluetooth ID's, although I'm not sure if most phones will respond to Bluetooth probes in normal operation.

Another approach would be suitable for a conference or other event where participants have badges. Simply put an RFID tag in each badge and have a short-range transceiver near each display. Now the art not only responds to aggregate preferences, but it also personalizes itself for each viewer. Effects which have previously held a participant's attention will appear more often when that participant is nearby. This will probably result in overall better evolutionary art -- instead of trying to impress the whole crowd, which is noisy and fickle, the algorithm tries to impress every individual person. While it's impressing one group, other people may be attracted in, and this constitutes more upvotes for that gene.

I think one important feature for this to work effectively is a degree of temporal coherence for a given display. If they're each showing short, unrelated clips (like Electric Sheep does), then people will not move around fast enough for the votes to be meaningful. Rather, each display should slowly meander through the parameter space, displaying similar types of fractal for periods on the order of 10 minutes (though of course there may be rapidly-varying animation parameters as well; these would not be parameters in the GA, though their rates of change, functions used, etc. might be).


20070422

Idea : fractally compressed AR

This is an augmented reality idea I had while walking around looking at trees after Drop Day. Basically, one would wear a VR headset that displays imagery from the outside world, except that occurrences of similar visual objects get replaced with the exact same object, or the same object perturbed in some synthetic way.

So, for example, the leaves of a tree would get replaced with fractals that are generated to look like leaves. As another example, areas of the same "texture" could be identified (basically, areas with little low-frequency spatial component, possibly after a heuristically determined perspective correction). Then a random small exemplar patch is selected and used to fill the entire area with Wei & Levoy / Ashikhmin-style synthetic textures.

The point of all of this is that you're essentially applying lossy compression (by identifying similar regions and discarding the differences between them), then decompressing and feeding the information into the brain (and thus mind). Working on the assumption that consciousness essentially involves a form of lossy compression which selects salient features and attenuates others, you can determine the degree and nature of this compression by determining when a similar, externally applied compression becomes noticeable or incapacitating.

My guess is that there will be a wide range of compression levels where reality is still manageable and comprehensible but develops a highly surreal character. Of course to experiment meaningfully you'd need a good enough AR setup that the hardware itself doesn't introduce too much distortion, although you could also control for this by having people use the system without software distortions.