Why people think computers can't
Marvin minsky, mit
First Published in Ai Magazine, Vol. 3 No. 4, Fall 1982. Reprinted in Technology Review, Nov / Dec 1983, And in The Computer Culture, (Donnelly, Ed.) Associated Univ. Presses, Cranbury NJ, 1985
Most people think computers will never be able to think. That is, really think. Not now or ever. To be sure, most people also agree that computers can do many things that a person would have to be thinking to do. Then how could a machine seem to think but not actually think? Well, setting aside the question of what thinking actually is, I think that most of us would answer that by saying that in these cases, what the computer is doing is merely a superficial imitation of human intelligence. It has been designed to obey certain simple commands, and then it has been provided with programs composed of those commands. Because of this, the computer has to obey those commands, but without any idea of what's happening.
Indeed, when computers first appeared, most of their designers intended them for nothing only to do huge, mindless computations That's why the things were called "computers" Yet even then, a few pioneers -.. Especially Alan Turing - envisioned what's now Called "artificial intelligence" - or "ai". The Processes Might Possibly Go Beyond Arithmetic, And Maybe Imitate The Processes That Go ON INSIDE HUMAN BRAINS.
Today, with robots everywhere in industry and movie films, most people think Al has gone much further than it has. Yet still, "computer experts" say machines will never really think. If so, how could they be so smart, and yet so DUMB?
==================================================== We Naturally Admire Our Einsteins and Beethovens, and . wonder if computers ever could create such wondrous theories or symphonies Most people think that creativity requires some special, magical "gift" that simply can not be explained If so, then no computer could create -. since anything machines can do (most people think can BE EXPLAINED.
To see what's wrong with that, we must avoid one naive trap. We must not only look at works our culture views as very great, until we first get good ideas about how ordinary people do ordinary things. We can not expect to guess , right off, how great composers write great symphonies. I do not believe that there's much difference between ordinary thought and highly creative thought. I do not blame anyone for not being able to do everything the most creative people do. I don ' T BLAME THEM for Not Being Able To Explain IT, Either. I do Object To The Idea That, Just Because We can't Explain It now, The No One Ever Could Image How Creativity Works.
. We should not intimidate ourselves by our admiration of our Beethovens and Einsteins Instead, we ought to be annoyed by our ignorance of how we get ideas -. And not just our "creative" ones Were so accustomed to the marvels of the unusual that we forget how little we know about the marvels of ordinary thinking. Perhaps our superstitions about creativity serve some other needs, such as supplying us with heroes with such special qualities that, somehow, our deficiencies seem more excusable.
Do outstanding minds differ from ordinary minds in any special way? I do not believe that there is anything basically different in a genius, except for having an unusual combination of abilities, none very special by itself. There must be some intense concern with some subject, but that's common enough There also must be great proficiency in that subject;. this, too, is not so rare;. we call it craftsmanship There has to be enough self-confidence to stand against the scorn of peers; alone, we call that stubbornness. And certainly, there must be common sense. As I see it, any ordinary person who can understand an ordinary conversation has already in his head most of what our heroes have. So, why can not "ordinary, common sense "- when better balanced and more fiercely motivated -? make anyone a genius, So still we have to ask, why does not everyone acquire such a combination First, of course, it sometimes just the accident of finding a novel way to look at Things. But, Then, There May Be Cer . Tain kinds of difference-in-degree One is in how such people learn to manage what they learn: beneath the surface of their mastery, creative people must have unconscious administrative skills that knit the many things they know together The other difference is in. Why Some People Learn So Many More and Better Skills. A Good Composer Masters Many Skills of Phrase and Theme - But So Does Anyone Who Talks Coherently.
Why do some people learn so much so well? The simplest hypothesis is that they've come across some better ways to learn! Perhaps such "gifts" are little more than tricks of "higher-order" expertise. Just as one child learns to re-arrange its building-blocks in clever ways, another child might learn to play, inside its head, at Fe-arranging how it learns! Our cultures do not encourage us to think much about learning. Instead we regard it as something that just happens to us But learning must itself consist of sets of skills we grow ourselves;..? we start with only some of them and and slowly grow the rest Why do not more people keep on learning more and better learning skills Because it's not rewarded right away, its payoff has a long delay. When children play with pails and sand, they're usually concerned with goals like filling pails with sand. But once a child concerns itself instead with how to better learn, then that might lead to Exponential Learning Growth! Each Better Way to Learn To Learn Would Lead to Better Ways to Learn - And this Could Magnify Itself Into An Awesome, Qualitative Change. Thus, FirstHo "Creativity" Could Be Just The Consesequence of Little CHildhood Accidents.
So why is genius so rare, if each has almost all it takes? Perhaps because our evolution works with mindless disrespect for individuals. I'm sure no culture could survive, where everyone finds different ways to think. If so, how sad, for That Means Genes for Genius Would Need, INSTEAD OF NURTURING, A FREQUENT WEEDING OUT.
==================================================== We can hardly expert to becomle to make Machines DO wonders before we find how to make them do ordinary, sensible things. The earliest computer programs were little more than simple lists and loops of commands like "do this. do that. do this and that and this again until that happens". Most people STILL WRITE PROGRAGINE SUCH LANGUAGES (LIKE BASIC OR FORTRAN) Which Force You to Imagine Everything Your Program Will Do From One Moment To The Next. Let's Call This "do now" programming.
Before long, Al researchers found new ways to make programs. In their "General Problem Solver" system, built in the late 1950's- Allen Newell, JCShaw and Herbert A.Simon showed ways to describe processes in terms of statements like "If the difference between what you have and what you want is of kind D, then try to change that difference by using method M. "This and other ideas led to what we call" means-ends "and" do if needed "programming methods. Such programs automatically apply rules whenever they're needed, so the programmers do not have to anticipate when that will happen. This started an era of programs that could solve problems in ways their programmers could not anticipate, because the programs could be told what sorts of things to try, without knowing in advance which would work. Everyone knows that if you try enough different things at random, eventually you can do anything. But when that takes a million billion trillion years, like those monkeys hitting random typewriter keys, it's not intelligence -. just Evolution The new systems did not do things randomly, but used "advice" about what was likely to work on each kind of problem So, instead of wandering around at random, such programs could sort. Of Feel Around, The Way You'd Climb A Hill in The Dark by Always Moving Up The Slope. The Only Trouble Was a Tendency To Get Stuck On Smaller Peaks, And Never Find The Real Mountain Tops.
Since then, much Al research has been aimed at finding more "global" methods, to get past different ways of getting stuck, by making programs take larger views and plan ahead. Still, no one has discovered a "completely general" way to always find the best method - and no one expects to.Instead, today, many Al researchers aim toward programs that will match patterns in memory to decide what to do next I like to think of this as "do something sensible" programming A.. few researchers - too few, I think - experiment with programs that can learn and reason by analogy These programs will someday recognize which old experiences in memory are most analogous to new situations, so that they can "remember" which methods worked best. On Similar Problems in The Past.
================== CAN computers UNDERSTAND? ========================================================================================================================================================================================
Can we make computers understand what we tell them? In 1965, Daniel Bobrow wrote one of the first Rule-Based Expert Systems. It was called "STUDENT" and it was able to solve a variety of high-school algebra "word problems". LIKE these:
The distance from new york to los angeles is 3000 miles. If The average speted of a jet plane is 600 miles per hour, find the time it takes to traffic from new york to los angeles by jet.
Bill's Father's Uncle Is Twice AS Old As Bill's Father. Two Years From Now I Bill's Father Will Be Three Times AS Old As Bill. The Sum of Their Ages Is 92.Find Bill's Age.
Most students find these problems much harder than just solving the formal equations of high school algebra That's just cook-book stuff -. But to solve the informal word problems, you have to figure out what equations to solve and, to do that, you must understand what the words and sentences mean. Did STUDENT understand? It used a lot of tricks. It was programmed to guess that "is" usually means "equals". It did not even try to figure out what "Bill's fathers' uncle "Means - It Only Notic That this Phrase Rembly" Bill's Father ". It Didn't Know That" Age "and" Old "Refer to Time, But It Took The To Represent Numbers To Be Put in Equations. with a couple of hundred such word-trick-facts, STUDENT sometimes managed to get the right answers.Then dare we say that STUDENT "understands" those words? Why bother. Why fall into the trap of feeling that we must define old words like "mean" and "Understand"? It's Great When Words Help US Get Good Ideas, But Not When The Confuse US. The Ques Tion SHOULD BE: Does Student Avoid The "Real Meanings" by useing tricks?
Or is it that what we call meanings really are just clever bags of tricks. Let's take a classic thought-example, such as what a number means. STUDENT obviously knows some arithmetic, in the sense that it can find such sums as "5 plus 7 IS 12 ". But does it Understand Numbers in Any Other Sense - Say, What 5" IS "- or, for That Matter, What Are" Plus "or" is "? What Would ? What if we built machines that were not based on rigid definitions Wont they just drown in paradox, equivocation, inconsistency Relax Most of what we people "know" already overflows with contradictions;?! Still we survive The best we can do is. Be Reasonably Careful; Let's Just Make Our Machines That Careful, Too. If The Remain Some Chances of Mistake, Well, That's Just Life. =========================== Webs Of Meaning. ================== If every meaning in a mind depends on other meanings in that mind, does that make things too ill-defined to make a scientific project work? No, even when thing go in circles, there still are scientific things to do! Just make new kinds ! of theories - about those circles themselves The older theories only tried to hide the circularities But that lost all the richness of our wondrous human meaning-webs; the networks in our human minds are probably more complex than any other structure Science ever contemplated in. ................................. Let's Go Back to What Numbers Mean. This Time, To Make Things Easier, Well Think About Three. I'm Arguing That Three, For US, Has No One Single, Basic Definition, But Is A Web Of Different Processes That Each Get Meaning From the Others. Consider All the Roles "Three" Plays. One Way We Tell A Three Is To Recite "One, Two, Three", While Pointing to the Different Things. To Do It Right, Of Course, You Have To (i ) touch each thing once and (ii) not touch any twice. One way to count out loud while you pick up each object and remove it. Children learn to do such things in their heads or, when that's too hard, to use tricks like Finger-Pointing. Another Way to Tell A Three Is To Use Some Standard Set of Three Things. THEN BRING Because each trick works in different situations, our power stems from being able to shift from one trick to another To ask which meaning is correct -. To count, or match, or group -. Is foolishness Each has its uses and its ways to support the others. None has much power by itself, but together they make a versatile skill-system. Instead of flimsy links in chain of definitions in the mind, each word we use can activate big webs of different ways to deal of things, to use them, to remember them, to compare them, and so forth. With multiply-connected knowledge-nets, you can not get stuck. When any sense of meaning fails, you can switch to another. The mathematician's way, once you get into the slightest trouble, you're stuck for good Why, then, do mathematicians stick to slender chains, each thing depending as few things as is possible The answer is ironic:!? mathematicians want to get stuck When anything goes wrong, they want! To be the first to notice it. The best way to be surething is having everything collapse at once To them, fragility is not bad, because it helps them find the perfect proof, lest any single thing they think be inconsistent with any other one That's fine for Mathematics;!. in fact, that's what much of mathematics IS. It's Just Not Good Psychology. Let's Face IT, Our Minds Will Always Hold Some Beliefs That Turn Out Wrong. I think it's bad psychology, when teachers shape our children's mathematics into long, thin, fragile, definition tower-chains, instead of robust cross-connected webs. Those chains break at their weakest links, those towers topple at the slightest shove. And that's what happens to a child's mind in mathematics class, who only takes a moment just to watch a pretty cloud go by. The purposes of ordinary people are not the same as those of mathematicians and philosophers, who want to simplify by having just as few connections as can be. In real life, the best ideas are cross-connected as can be. Perhaps that's why our culture makes most children so afraid of mathematics. We think we help them get things right, by making things go wrong most times! Perhaps , INSTEAD, WE OUGHT TO HELP THEM Build More Robust NetWorks in their heads. ================== Castles in the air. =========== ======= The secret of what something means lies in the ways that it connects to all the other things we know. The more such links, the more a thing will mean to us. The joke comes when someone looks for the "real" meaning of anything. For, if Something Had Just One Meaning, That IS, IF IT WERE INLY CONNECTED TO JUST One Other Thing, The IT Wold Scarcely "Mean" At all! That's why I think we should not program our machines that way, with clear and simple logic definitions A machine programmed that way might never "really" understand anything -.. Any more than a person would Rich, multiply-connected networks provide enough different ways to use knowledge that when one way does not work, you can try to figure out why When there are many meanings in a network, you can turn things around in your mind and look at them from different perspectives;. when you get stuck, you can try another view. That's what we mean by thinking! That's why I dislike logic, and prefer to work with webs of circular definitions. Each gives meaning to the rest. There's nothing wrong with liking several different tunes, each one the more because it contrasts with the others There's nothing wrong with ropes -. or knots, or woven cloth - in which each strand helps hold the other strands together -! or apart There's nothing very wrong, in this strange sense, with having all one's mind a Castle in the air! To summarize: of course no computer could understand anything real - or even what a number is - if forced to single ways to deal with them But neither could a child or philosopher So such concerns are not about computers at all, but about.. Our Foolish Quest for meanings That Stand by Themselves, Outside Any Context. Our Questions About Thinking Machines Should Really BE Questions Aboutur Own Minds. ================== Are humans self-aware? ======================================================================================================================================================================================================= Most People Assut, or self-aware; at best the APPEY CAN ONLY COURSE, THISSUMES THAT WE, AS HUMANS, ARE SELF-AWARE. But Are We? I think Not . I know that sounds ridiculous, so let me explain.If by awareness we mean knowing what is in our minds, then, as every clinical psychologist knows, people are only very slightly self-aware, and most of what they think about themselves is guess-work. We seem to build up networks of theories about what is in our minds, and we mistake these apparent visions for what's really going on. to put it bluntly, most of what our "consciousness" reveals to us is just "made Up ". NOW, I don't mean this we're not aware of sounds and hights, or even of some parts of thoughts. I'm only self much what we're not aware of much the goes on inside our minds. When people talk, the physics is quite clear: our voices shake the air; this makes your ear-drums move - and then computers in your head convert those waves into constituents of words These somehow then turn into strings of symbols representing words,. So Now there'S homewhere in your head That "represents" a callnce. What happens next? When light excites your retinas, this causes events in your brain that correspond to texture, edges, color patches, and the like. Then these, in turn, are somehow fused to "represent" a shape or outline of a thing. What happens then ? We all comprehend these simple ideas. But there remains a hard problem, still. What entity or mechanism carries on from there? We're used to saying simply, that's the "self". What's wrong with that idea? Our standard concept of the self is that deep inside each mind resides a special, central "self" that does the real mental work for us, a little person deep down there to hear and see and understand what's going on. Call this the "Single Agent" theory. It is not hard to see why every culture gets attached to this idea. No matter how ridiculous it may seem, scientifically, it underlies all principles of law, work, and morality. Without it, all our canons of responsibility would fall, of blame . or virtue, right or wrong What use would solving problems be, without that myth; how could we have societies at all The trouble is, we can not build good theories of the mind that way In every field, as Scientists we're?. Always Forced to Recognize That What We See As Single Things - Like Rocks or Cl Ouds, or Even Minds - Must Sometimes Be Described as Made Other Kinds of Things. We'll Have To Understand That Self, Itself, IS Not a Single Thing. ============ New theories about minds and machine. ============ It is too easy to say things like, "Computer can not do (xxx), because they have no feelings, or thoughts". But here's a way to turn such sayings into foolishness. Change them to read like this. "Computer can 't do (xxx), because all they can do is execute incredibly intricate processes, perhaps millions at a time "Now, such objections seem less convincing - yet all we did was face one simple, complicated fact:. we really don' t yet know what the limits of computers are Now let's face the other simple fact:. our notions of the human mind are just as primitive.Why are we so reluctant to admit how little is known about how the mind works It must come partly? from our normal tendency to repress problems that seem discouraging. But there are deeper reasons, too, for wanting to believe in the uniqueness and inexplicability of Self. Perhaps we fear that too much questioning might tear the veils that clothe our mental lives. To me there is a special irony when people say machines can not have minds, because I feel we're only now beginning to see how minds possibly could work -. Using insights that came directly from attempts to see what complicated machines can do Of course we're nowhere near a clear and complete theory -.. yet But in retrospect, it now seems strange that anyone could ever hope to understand such things before they knew much more about machines Except, of course, if they believed that minds are not Complex at all. Now, you might ask, if the ordinary concept of Self is so wrong, what would I recommend in its place To begin with, for social purposes, I do not recommend changing anything -?. It's too risky But for the technical enterprise of making intelligent machines, we need better theories of how to "represent", inside computers, the kinds of webs of knowledge and knowhow that figure in everyone's common-sense knowledge systems. We must develop programs that know, say, what numbers mean, instead of just being able to add and subtract them. We must experiment with all sorts of common sense knowledge, and knowledge about that as well.Such is the focus of some present-day Al research. True, most of the world of "Computer Science "is involved with building large, useful, but shallow practical systems, a few courageous students are trying to make computers use other kinds of thinking, representing different kinds of knowledge, sometimes, in several different ways, so that their programs will not get STU ck at fixed ideas. Most important of all, perhaps, is making such machines learn from their own experience. Once we know more about such things, we can start to study ways to weave these different schemes together. Finally, we'll get machines That Think About Themselves And make Up Theories, Good or Bad, OR Bad, of How Themsels Might Work. Perhaps, When Our Machines Get To That Stage, We'll Find It Very Easy To Tell It Has Happened. For, at That Point, .............................. ================== KNowledge and common sense ====================================================================================================================================================================================================================================================== # and literal behavior of computers. They send us silly checks and bills for $ 0.00. They can not tell when we mean "hyphen" from when we mean minus They do not mind being caught in endless loops, doing the same thing over again a billion times. This total lack of common sense is one more reason people think that no machine could have a mind. It's not just that they do only what they're told, it's also that they're so dumb it's almost impossible to tell them HOW to do things right. Is not it odd, when you think about it, how even the earliest Al programs excelled at "advanced" subjects, yet had no common sense A 1961 program written by James Slagle could solve calculus problems at the level of college students;? It even got an A on an MIT exam. But it was not till around 1970 that we managed to construct a robot programs that could see and move well enough to handle ordinary things like children's building blocks and do things like stack them up, take them Down, RearRange Them, And Put Them in Boxes. ? Why could we make programs do those grown-up things before we could make them do those childish things The answer is a somewhat unexpected paradox: much "expert" adult thinking is basically much simpler than what happens in a child's ordinary play It can! Be Harder to Be a novice Than Be An Expert! This is Because, Sometimes, What An Expert Needs To Know and Do Can Be QUITE SIMPLE - ONLY, IT May Be Very Hard To Discover, or Learn, in The First Place. Thus, Galileo had to be smart indeed, to see the need for calculus. He did not manage to invent it. Yet any good student can learn it today.The surprising thing, thus, was that when it was finished, Slagle's program needed .. only about a hundred "facts" to solve its college-level calculus problems Most of them were simple rules about algebra But others were about how to guess which of two problems is likely to be easier; that that kind of knowledge is especially important , Because IT Helps The Program Make Good Judgments About What To Do next. With Such ProGrams Only Thrash About; WITH IY SEEM MUCH More Purposeful. Why Do Human Students Take So Long To Learn Such Rules? We do not know. Today we know much more about making such "expert" programs - but we still do not know much more about making programs with more "common sense" Consider all the different things that children do, when they play with their blocks To.. build a little house one has to mix and match many different kinds of knowledge: about shapes and colors, space and time, support and balance, stress and strain, speed, cost, and keeping track An expert sometimes can get by with deep but. narrow bodies of knowledge - but common sense is, technically, a lot more complicated.Most ordinary computer programs do just the things they're programmed for some Al programs are more flexible; when anything goes wrong, they can back up to some previous. decision and try something else. But even that is much too crude a base for much intelligence. to make them really smart, we'll have to make them more reflective. A person tries, when things go wrong, to understand what's going wrong, INSTEAD OF Just Attempting Somethin . G else We look for causal explanations, or excuses, and, when we find them, add them to our networks of belief and understanding We do intelligent learning Some day programs, too, could do such things -.. But first we ' D NEED A LOT More Research To Find Out How. ================== unconscial fears and phobias. ================== I'll bet that when we try to make machines more sensible, we'll find that learning what is wrong turns out to be as important as learning what's correct. In order to succeed, it helps to know the likely ways to fail. Freud talked about censors in our minds, that keep us from forbidden acts or thoughts And, though those censors were proposed to regulate our social activity, I think we use such censors, too, for ordinary problem solving -. to know what not to do . Perhaps we learn a new one each time anything goes wrong, by constructing a process to recognize similar circumstances, in some "subconscious memory" .This idea is not popular in contemporary psychology, perhaps because censors only suppress behavior, so their activity is invisible on the surface. When a person makes a good decision, we tend to ask what "line of thought" lies behind it. But we do not so often ask what thousand prohibitions might have warded off a thousand bad alternatives. If censors work inside Our Minds, To Kee P US from Mistakes and Absurdities, Why Can't We Feel That Happening? Because, I Suppose, So Many Thousands of Them Work Atate That, IF You Had to Think About Them, You'd Never Get Much Done. Ward Off Bad Ideas Before You "Get" Those Bad Ideas. Perhaps this is one reason why so much of human thought is "unconscious". Each idea that we have time to contemplate must be a product of many events that happen deeper and earlier in the mind. Each conscious thought must be the end of processes in Which it Must Compete With Other Proto-Thoughts, Perhaps by Pleading Little Briefs in Little Courts. But All this we do sense of this area... And how, indeed, could it be otherwise? There's no way any part of the mind could know everything that happens in the rest. Our conscious minds must be like high executives, who can not be burdened with the small details. There's only time For Summaries from Other, Smaller Parts of Mind, That Know Much More About Much Less; The ones That Do The Real Work. ==================================================================================================================================== ================== Then, is it possible to program a computer to be self-conscious? People usually expect the answer to be "no". What if we answered that machines are capable, in principle, of even more and better consciousness than people have? I think this could be done by providing machines with ways to examine their own mechanisms while they are working In principle, at least, this seem possible;. We already have some simple Al programs that can understand a little about how some simpler programs work. (There is a technical problem about the program being fast enough, to keep up with itself, but that can be solved by keeping records.) The trouble is, we still know far too little, yet, to make programs with enough common sense to understand even how today's simple Al problem-solving programs work. But once we learn to make machines that are smart enough to understand such things, I see no special problem in giving them the "self-insight" they would need to understand, change, And Improve themselves. This might not be so wise to do. But what if it turns out that the only way to make computers much smarter is to make them more self-conscious? For example, it might turn out to be too risky to assign a robot to undertake Some Important, Long-Range Task, WITHOUT SOME "Insight" About it's ing it to set to start projects it can't finish, we'd better harness it know what it can do. if we want it versatile enough to solve new kinds of problems, it may need to be able to understand how it already solves easier problems. In other words, it may turn out that any really robust problem solver will to understand itself enough to change itself. Then, if that goes on long enough, why can not those artificial creatures reach for richer mental lives than people have. Our own evolution must have constrained the wiring of our brains in many ways. But here we have more options now, since we can wire machines In Any Way We wish.it Will Be a long time before we learn ENOUGH ABOUT COMMON S ense reasoning to make machines as smart as people are. Today, we already know quite a lot about making useful, specialized, "expert" systems. We still do not know how to make them able to improve themselves in interesting ways. But when we answer such questions, then we'll have to face one, even stranger, one. When we learn how, then should we build machines that might be somehow "better" than ourselves? We're lucky that we have to leave that choice TO FUTURE GENERATIONS. I'm Sure The Want To Build The Things That Well Unless The Find Good Reasons To.
