The structure used by the brain to gather, store/learn, organize, search and use data objects received from parallel modalities. To then recreate the virtual mirror world for modality tracking, modeling and prediction. - How does the brain physically do this? How is the information held and organized? How fast can instantiation occur? How are the actual records searched and modified?

When human vision is presented with the 2D render plane of an object, the wave front is effectively compared to millions of object candidates and orientations - virtually simultaneously. There are no giga hertz clocks in the human brain, only a few dozen parallel steps go from 2D exposure to 3D object recognition and orientation. Then moments later, to a behavior prediction within a scene trial construction and finally an emotional grade.

Silicon based computing generally operates on a serial metaphor, one instruction operation per cycle, and the same for memory systems, one memory location transaction per (several) clock ticks.

But there are silicon technologies that transcend this limit. For instance contents addressable memories (CAM's). These -associative memories have the ability to scan hundreds of thousands of entries - in parallel - on a single clock cycle. This may be just the kind of silicon memory architecture able to solve the massive data searching requirements of visual processing and identification.

Their architecture can be thought of as a bus of input wires feeding millions of memory latches. If a match occurs anywhere across the array, the latch triggers a flag, and the flag is decoded down to an address pointer to a record in regular silicon memory, one clock cycle later

The added flexibility of 'don't care' states along the input data bus would allow 'fuzzy' searches yielding 'degrees of similarity' rather than exact data matches. Again, ideal for visual processing. The major problem with this technology is cost and parallelization. The traditional application area for these classes of memory are for internet packet traffic routers, which have limited memory requirements compared to tradition DRAM memory.

But there are other novel methods of using existing high density silicon memory systems. The challenge is to change paradigms from the serial computational metaphor of CPU and memory to a parallel memory computation metaphor. I.e. the computation occurs by nature of the memory structure alone.

So for example take a 1GB flash memory chip that has 20 input bits (addresses) and 8 output bits (data) Rather than considering the address and data as separate addressing and data domains - consider them both data domains. By paralleling the chips you have a growing input and output bit array. The transform function between the two is programmable and automatic. Without recourse to a serial processor in-between.

Take one example, the recognition or translation of visual data from say the bitmap domain of symbolic language. (But this could equally be applied to a fourier transform bitmap representation of sound). The static input bitmap pattern is applied to the parallelized address bits. the parallelized data bits reflect the translation domain, which may be a pointer, an image, a 3D model etc.

The human mind is able to apply time domain effects and most importantly, manipulate the memory core in coherence with physical animation, so the virtual world of the memory is carved out by the physical animation in reality. These mechanisms are far from clear at present

Reality exists always in a single moment, only computation can link that moment to the past and to the future though the application of simulated time on simulated memory precedents. The memory traces are carved into coherence by the physical engagement with reality with feedback through the modalities.

As you handle a physical object, like a cup, your hands carve out the third dimension from within the vision. Simultaneously as the cup form is being carved by the hands, so the memory traces are carved out too by the virtual hands guided by the modality feedback, the touch texture, the light effects, the weight, the finger positions etc.

To bring these processes back to silicon, there needs to be a method whereby the transform function can be mediated by the mechanical animations in the real world. So as the manipulators describe out volume, texture etc. So the memory traces carve out the virtual representation using the very same memory transfer function route. I.e. physical animation flow through the modalities to the virtual model; physical animation interacting with the real model, transforms to physical animation interacting with virtual model to carve out the existence space of both the real and the imaginary (the virtual)

A more traditional approach might be the linking of animation tools with robotic manipulation. Take a 3D software package, create a default environment of 'existential clay' and a carve tool the shape and performance of a robotic limb. Link the 3D tool animation to the robot arm, with a guiding exploratory protocol driving discovery. As the robot manipulator makes contact with solid objects, the virtual clay begins to fix the external reality. Add a vision overlay, and full visual textures will accompany the process. The external reality is then represented internally by a sensorimotor data trail.