i n c a - research

- 0 -

To say computers are just 1's and 0's is like saying humans are just atoms. Whereas in fact, both humans and computers are just atoms. But what's really intriguing, is that both can process information; but at the moment, we do not speak the same language. That, ultimately, is what this research is all about. Because when computers can understand our language, they will have the means to dwarf our intelligence. But language is just a cunning smoke screen hiding a much deeper structure. And it is that hidden structure I intend to root out.

Everything that really matters in the world has form and behavior predictability. Sure, fancy mathematics can predict the exact arc of a theoretical cannon shell. But the universal language of object form and behavior is not English or math, it is 3D animation. The human brain uses information from the senses to build an inner mirror world into the neuronal memory substrate. These 3D simulations within the brain are intimately 'connected/grounded' to reality through our senses and muscles (the sensorimotor system), whereas data in electronic sytems is almost entirely disconnected.

This research is based on the theory that the human mind contains a detailed 3D simulation of reality, (built from, and connected to reality through the sensorimotor interface). That the processing is integral to the very structure of that interface and the biological memory behind it. And that language is used to 'index' scene scripting. Together with a value hierarchy to grade scene progression (emotion), these processes are sufficient to explain all human motivation and behavior.

Human consciousness is the computational process controlling our physical action (muscles). It originates from biological self assembly, genetically seeded with a survival program that evolved to maximize existence time and reproduction.

It does this by connecting internal biological memory structures with external reality through the sensorimotor interface (senses and muscles), to build internal models of the external environment. As the depth and accuracy of the simulations grow, the potential for prediction, intelligence and conscious action emerges through learned model precedents from reality being made available as animated belief scripts and emotionally graded through a genetically inspired value hierarchy. Yet these simulations still remain connectable to reality through the sensorimotor system. Humans are pre-programmed with motivation to animate movement in alignment with those belief scripted and value graded simulations, to more effectively compete for survival resources and mating opportunities.

The process involves the recognition of objects in reality from the sensory input flow (instantiation) and constructing an internal scene simulation based on those remembered precedents. The 3D simulation exists as collections of memorized precedents, indexed from those recognised objects in reality. The memories extend beyond the fixed time reference of the outside world, because the memorized precedents of the assembled objects contain a series of 3D 'movie script like' scenarios for each of the objects - as learned (recorded into memory) from past experiences. As such, they can be used to make predictions of past and future action. Being virtual, these predictions can beat the time of reality, to allow a human to say, catch a ball in a future moment, and to know the balls origin from the past. I.e. consciousness performs time dilation by building a simulation from memory precedents, which animate through virtual time. The simulated objects can be triggered into initial alignment with reality by the human senses. But the simulation easily yields control to the collections of memory precedents aroused by the scene, which search out emotional value peaks and troughs based on the biologically inspired pleasure/pain value axis.

Sidebar- How we predict the future....

1) Imagine a computer that watches a candle burn itself out.
2) It has software that converts the 2D vision to a memorized 3D animated timeline.
3) A new candle is lit and the computer recognizes the scene from the learned precedent.
4) The computer quickly runs the prior scene precedent and thus 'believes' the candle will grow smaller and disappear in a few hours time.

That oversimplified process is how a human also, is able to predict the future. The brain's data processing currently outperforms electronic computers in this task. Aided by a powerful sensorimotor system to connect memory traces with reality.

The human mind is seeded genetically, with evolutionarily significant 3D object forms and behaviors. This is why we can 'instinctively' recognize sexual beauty and tasty food etc. We have a genetic (and subsequent socially inspired) value hierarchy which classifies the sensory and simulated worlds through emotional beliefs, (e.g. pleasure and pain). Cognitive momentum - 'value seeking', drives the simulation process to experiment with script precedents; to adjust and substitute objects, behaviors and properties within scenes, guided by cost, benefit & risk/reward analyses according to our belief prejudices. The highest value emergent script can be used to direct the physical body animation through the sensorimotor system connecting reality to the virtual simulation memory traces. Control of the actual muscles occur largely through non-conscious, cybernetically trained circuits; to gracefully manage momentum flow & balance.

Self consciousness is triggered by the placement of a self actor object within this simulated reality. This cognitive innovation creates a gateway linking 1) external reality; 2) physical action; 3) the information flow from the senses and 4) the mental simulations in memory. The information flows, achieve 'coherence' through the mechanism of the virtual self actor connected to reality through the sensorimotor system (the musculature, sensory data and feeling states become attached to the simulation of self). The belief in emotional (empathy) states attached to the self actor object, allow the subsequent grading of third party character behavior through metaphor. (So if you see an animal suffer injury, the subsequent instantiation of the vision to create a simulation, can substitute the self actor within the scene script (metaphor), and the emotional grade becomes personal - i.e. empathic). This mechanism is fundamental to social belief programming in humans.

Language creates a shared referencing system between 1) other humans; 2) historic record; 3) the real world of our senses and 4) the simulated world of our mind. It does this by indexing beliefs in objects, behaviors and subjective values with language 'sound and image tags', such that simulations can be built, or interpreted, using serial language narrative. This innovation, allows the construction and communication of shared simulation and emotional states of the same form as those instantiated directly from the senses. It provides a mechanism whereby cultural knowledge can be stored and transferred socially; to be passed on to subsequent generations. Further, the language indexing of emotional states & values (referent states of the mind, rather than of reality), allows for the social re-programming of the value hierarchy used to subsequently grade our internal simulations and beliefs - and thus our actions.

Just as the genes manufacture a unique human body, so they create a unique human mind, with subtle differences in programming. Although everyone is united by a shared reality, they each exist at a unique location in 3D space (perspective) - so even genetic twins will develop unique conscious identities. And just as the genes code for an archetypal physical human form, (two eyes etc.) yet with millions of nuances between individuals (eye color, shape). So they also code for archetypal belief systems; with millions of nuances between those. This is why everyone holds both unique personal and a common shared set of values and beliefs, that must subsequently try to integrate cultural/social programming fashions.

Human consciousness is the best example we have of wilful intelligence, and the description outlined above is the basis for this AI project. But the brain achieves these computations in a very different manner to electronic computers, which act on information serially through a processor pipeline and have a fundamental disconnect between the data and what that data refers to in external reality.

The brain operates on the memory records in situ, in parallel and connected to reality through the sensorimotor system. The best metaphor being, that 3D memory objects (precedents) are struck into resonance (conscious or sub-conscious), by either human sensory perception - the way a tuning fork can catch harmonics from environmental vibrations; by external language narrative; or by internal language or animation narrative driven by value prefereces. These 'resonance's animate various precedents which merge and align into 'scenes' to be graded by our values. Thoughts probably become conscious upon the animations being reflected (rendered) back to the five sensory cortex regions, and/or becoming associated with episodic memory. The brain, with its 100 billion neurons, is the physical substrate for this information processing, which appears to be electro-chemical in nature.

This theory challenges much conventional wisdom about human action, consciousness and artificial intelligence. I.e. the basis of knowledge, logic, intelligence, will, emotions, truth, reality, beliefs - that kind of stuff.

In its simplest form, the theory presents these processes, all of them, by a single paradigm; 3D object computing. That is, all mental activity centers around the processing of virtual 3D objects. Perhaps even 'processing' is really too charitable a word to properly describe what is going on. Juggling might be a more appropriate metaphor.

Language symbols and words are objects themselves; they are extensions of the same simulation processes. But because of their serial communicable and semantic structure, they can act as tools fasioning consciousness, the way physical tools fashion reality (but unlike mechanical tools, internal scenes built from language tools are not limited by either time or the laws of nature).

Further, language has the ability to communicate animation scripts orally and in writing, leveraging the latent power of the brain to inherit wisdom from others and from prior generations. Language indexes and structures memory records, enabling the formation, storage and transmission of deeper and more coherent thoughts.

The gist of this research is that all conscious and intelligent processes center around 3D simulation; with language and symbols used for indexing and scripting. That all knowledge can be understood in terms of 3D model behavior based on precedent assemblies. That software designed to handle 3D models and environments, will be central to AI and commercial software of this class is advancing rapidly.

"When a computer has an animated world going on inside, one that can be aligned to reality through vision; that can be directed or interpreted by language; that 'value' certain outcomes. Then there will be no meaningful or spiritual difference between that computer mind and a humans" - Keith Hoyes Aug 2002

How we create the future

A conceptual & irreverent Introduction

Consider a 3D program, the type that makes animated movies or interactive video games. It can simulate all the laws of physics - gravity, momentum, light refraction etc.

Consider a library of objects, characters and motion behaviors, run, jump, crawl, explode, float etc. These motion vectors can be applied to any particular object character.

Consider the process of motion and shape morph tweening. filling in the missing frames of action between disparate objects or locations.

OK, so you run random simulation scripts and you get chairs jumping, heads exploding, cats boxing, candles dancing through walls etc. Total gibberish.

Now, instead of random scripts, you run script precedents. Say from all the animation movies and games you've ever seen.

Chairs now stay quite still, candles flicker quietly, warriors enter dungeons, sportsmen kick footballs and everyone shoots laser blasters...

OK, so instead of video game precedents, we use real life precedents. Every house, chair, beach, cat, tree, person, factory, machine, or whatever you can remember, placed into a 3D object category with a series of behavior precedents attached.

Draw up any particular scene of characters, and things look pretty normal now. No dancing cats, or anything weird like that.

But what happens next - and who cares.

Say you program a human character to hit a golf ball toward a window. You have a fairly fast computer, within a few milliseconds it has simulated the action and estimates an accurate scene. But it takes about 5 seconds for you to watch the rendered scene in 'human time'.

So in this example, the computer can predict what would happen in the future. Which of course, so could you. But unlike yourself, the computer cannot grade the result. The representation of shattered glass has no meaning.

OK, still pretty boring, you can watch vases fall and smash, Jeeps drive off cliffs, birds fly around, humans sit down, stand up, walk - yeah yeah yeah. no big deal, just a boring computer making silly patterns.

OK, so now, rather than you personally guide the scripts, you run behavior rules on the human, animal and machine characters. For example:

For cat characters...
- purr if you get stroked
- meow if no food for a while
- chase any object that's in view, is small and moves fast.
- run away from objects that are big and noisy.
- etc.

For human characters...
- if a cat is near, stroke it.
- if a cat meows - feed it
- if the door bell rings open the front door.
- if the daylight fades, turn on the lights
- if no food for a while - go eat
- if it's much past midnight - go to bed and sleep
- etc.

Now agreed, this is a pretty sterile environment. No language, no 'free will'. But based on a few simple rules, the computer can now start to statistically predict the action of animate characters as well. Now I don't know for sure what you are going to do next, but I can guess you might read on; you are unlikely to go eat a tennis ball. The computer model could predict that kind of thing, based not on math, but on precedent.

OK, so what's this got to do with AI. And what about creativity, feelings, art and intelligence.

Well first let us at least appreciate how far we have come. A cat, a chimp or even a young child cannot predict the effect of a vase falling, a ball-window collision, or even fading daylight on a human indoors - but our computer can. And that is no trivial thing.

But there are still some big pieces yet missing.

1) First, this is still only a virtual world - it has no link to reality - in or out. Just simple 'cartoons' dancing in a puff of virtual wind.

2) predicting 'some' action is not the same as 'originating' new action.

OK let's say you strap a camera to the side of your head and feed the signal into your computer. As we both know - nothing whatsoever will happen. But what if you start writing software to recognize objects from your library and to orient and place them in the virtual scene to match the vision - Don't tell me - it will never happen - never, not even in a million years it won't. OK - but lets say some smart kid does it - what then?

Well, we'd actually have a simultaneous scene running alongside reality. If the computer, like for the golf ball example, kept running predictions ahead of real time, it could be predicting action based on prior learned behaviors, but now 'of the real world' - before it even happens - 'in the real world'. Pretty cool really!

If every object and action had a name tag. Our computer could then describe the scene ahead and tell you what might happen next, instead of showing it on the screen. Or you could describe a script to the computer, and it could construct the 3D scene and tell you what's likely to happen next - based on character precedents, gravity, momentum etc.

So now our geek has an earpiece and a mic. and his computer can whisper hints about the future. Of course, most of the time he can guess what's going to happen anyway. But it would sure be eerie, a computer whispering - "you're going to pour a coffee; you're going to drink the coffee; you're going to answer the ringing phone; a 90% chance you'll watch Buffy in 5 minutes on TV ..... - "OK, OK, enough already!!"

So how far into the future will this 'thing' see. Not much further really than the fall of an object, or the completion of some human action. After all that work, it sounds pretty pointless to me! So what if our character gets in his Jeep and drives off? Sure, I understand he's not going to drive straight into the neighbors' auto - because of 'precedent' and all that. But where's he going to go, and why? And what if he slips on a banana skin on the way out. No computer's gonna predict that - huh!

Well neither did he, if he slipped on it! But anyway, based on prior experience 80% of the time he goes to work, 10% goes for pizza, 5% to the cinema, so statistically our computer could make a pretty good guess. But sure, he could be going to a new fairground in town - the first time ever. Or maybe he does crash into his neighbors' auto - because he's mad; or to prove he's got free will; or just because he's a teenager. So the computer got it wrong, time to pack our bags I guess, and give up on all this AI stuff.

OK, let's say we're no longer so interested in predicting what everyone does next, our geek decides to program the computer instead, to start grading the predictions. In other words, he writes a program for emotions. Also, he say's to it - "don't just sit their like a moron waiting for camera input, predicting and whispering silly messages to me - do something useful - go enter your virtual world of all those objects and places you have in memory. But wherever you go and whatever you do be sure to grade, grade and then grade some more, according to this schema here. Then come and tell me what you find out!" - Of course he writes this behavior in a very clever and highly advanced computer instruction code:

- WARNING - HIGHLY ADVANCED COMPUTER CODE

FOR
- { always }

IF
- { nothing much happening with vision }

GO
- { explore virtual worlds, grade them and report back to geek! }

- END OF WARNING -

and for emotional schema he codes...
- time is valuable
- money is valuable
- reputation is valuable
- blue eyed blondes in short skirts are valuable
- food is valuable - Papa John's is more valuable
- new knowledge is valuable
- and did I mention blue eyed blondes?
- and if trials fail - start afresh. - cpu cycles are cheaper than real life.

Now for every second of real time, his computer could build about 1000 seconds of script trial segments - what's the cat going to do next; what's the vase going to do next; what's geek going to do next. But he noticed that still left about 99% of the cpu idle, so he says to the computer - "instead of predicting a few seconds ahead, widen the timeline way up. Predict where I'll be in an hour, a day, a year even, and fill in the blanks as best you can. Stick all those memorized prediction segments you have for every object into little life histories that fit together. And also, don't be so boring. Try mixing them up a bit to see what happens. You know, the timelines. Like I wake up in the morning in the office, or in Alaska, and run the behavior script from there. And build metaphor - make the cat act like a human; or a vase like a bird. And instead of morphing the same old object to it's new position, try mixing those up too. When the scene gets boring, give up on it. And don't forget to grade everything, I don't want to know about all that silly nonsense unless I benefit somehow from a script segment - you know, based on those grading schemes I gave you - then come and tell me about it."

So, our geek then waits, and he waits some more. Then finally, one day while at the mall, he here's a faint whisper. "Hey, geek! - Next time you're at the checkout, if you put your hand in the till and grab, you'll get money for free .... and money is valuable right?"

"Oh yeah, now that's a real smart plan!"

"OK, how's this - you see that cute blonde walking this way, the one with the real long hair, the very short skirt, and big blue eyes...."yes, yes! I had noticed her myself thank you very much. ".... "Well she matches your 'most desirable female character model' - "Yes - and I quite realize that too!" - "Well, if you ask her for a date, she might say yes!"

"Listen, girls like that don't date geek's like me with cameras stuck to there face - OK!"

"Well there is a 1.142% chance she'll say yes, but a definite 0% chance if you do nothing, so logically you should go ask her."

"Sure, but there's a 99% chance she's gonna laugh straight in my face - and that sure ain't gonna feel so good. And what if every geek in town acted that way; the poor girl would be hit-on everywhere she goes. - Don't you understand empathy?"

"But you didn't put 'third party character empathy' into my value system!"

"Well maybe I should have?"

So at this point our geek finally decides to give up once and for all on this AI nonsense.