Chapter Three—
Mind, Self, Society, and Computer

Respect for Machines

For those who work with them, computers can teach the same sense of humility toward human endeavors as work with animals has done for ethologists. Carried away by the extraordinary ability of machines to process information—especially in contrast to the sloppy, irrational, and trial-and-error methods used by humans—researchers in the fields of artificial intelligence (AI), cognitive science, and robotics have, like students of animal behavior, been questioning the hypothesis that there is something special or unique about human abilities.

If anything, the vehemence with which the hypothesis of human distinctiveness is rejected is stronger among those fascinated with the properties of machines than it is among those fascinated with the properties of nonhuman animals. Although the invention of the computer could be taken as a further manifestation of human ingenuity, almost simultaneously with its development has arisen the hypothesis that artificial intelligence will eventually prove itself superior to human intelligence. As two writers in the artificial intelligence tradition have written:

And in all humility, we really must ask: How smart are the humans who've taught these machines? On the evolutionary time scale, thinking animals are relatively recent arrivals. Evolution hasn't had a great deal of time to work on the perfection of human cognition.^[1]

Reflecting that notion, some envision a "post-biological" world, in which machines will carry out most of the work that we used to do.^[2] If indeed "these artificial intelligences will help run society and relieve mankind of the burden of being the leading species," we would be guilty of what Edward Fredkin calls misplaced human vanity to think we occupy a privileged position over other species, including machina sapiens .^[3] The computer surpasses us at precisely the moment that non-human animals have caught up to us.

When research in artificial intelligence is linked to research in sociobiology, as it often is, the resulting perspective on humans is, literally, mind-boggling. The notion that "a body is really a machine blindly programmed by its selfish genes" seems to lead to the notion that machine programming can become the unifying element of all scientific understanding.^[4] The information sciences have become the basis for what its advocates call a unified theory of cognition, linking DNA reproductive codes to animal cognition to human decision-making and finally to machine calculation.^[5] When such a theory is developed, if it ever is, any notion that there is a specifically humanistic dimension to the study of human behavior would become obsolete, for its laws would be reduced by the mathematics of information transmission. The proper study of man, as Herbert Simon puts it, would no longer be man but "the science of decision."^[6]

Humans now have a new challenge to their view of themselves as unique. And so, therefore, do social theorists. For if machines can be developed and programmed in such a way as to emulate, or surpass, human intelligence and dexterity, then not only, pace Simon, would there be no basis for a specific human science; there would also be no basis for assigning any special importance to our cognitive powers. Sherry Turkle describes the computer as an "evocative . . . object that fascinates, disturbs equanimity, and precipitates thought."^[7] One of the thoughts it stimulates is that human selves have no special and distinct features worth singling out for special respect or special study.

As it happens, however, the claims of many enthusiasts for artificial intelligence are now generally viewed as exaggerated.

Computers have accomplished many wonderful things, but they have not yet replaced human intelligence. The question is not really which "machine" plays better chess: the Cartesian or the computer. The claims made on behalf of AI need to be examined, not so that we can praise its possibilities or gloat over its failures but, instead, so that we can learn about ourselves by contrasting the way we think and act to the way machines do. AI is a vast Gedenkenexperiment, a stimulus to reflections about what makes people human. As such, it is fully as important for the last half of this century as Darwinism was for the last half of the previous one. And the experiment appears to show that human thinking differs from machine calculation precisely because humans have interpretative capabilities.

The Human Essence Test

Let us, in the spirit of a thought experiment, accept for the moment the working hypothesis that there are no essential differences between computers and humans. (Reversing what appears to be common sense in order to test common sense is a crucial aspect of the way theorists in artificial intelligence think.) In order to confirm such a point of view, we need to return to questions of philosophical anthropology. We need to ask whether humans are different from any other species and, if they are, what makes them different. Unlike classical social theorists, however, who compared humans with other animals, we need to make comparisons with machines.

Researchers in AI have a test of machine intelligence, called the Turing test (though, it should be added, their version of the test is often different from the version originally proposed by Alan Turing).^[8] To determine whether a machine is intelligent, the Turing test suggests that we imagine a person being given instructions by both a machine and another person. When the person can no longer tell which of them is giving the instructions, intelligence has been modeled by the machine. How, in the same spirit, do we know we are in the presence of a human, rather than machine, intelligence? I propose that in thinking about a "human essence test," we turn

to the one sociological theorist who most addressed questions of mind and intelligent behavior: George Herbert Mead.

Mead's argument is that the difference between human and nonhuman species involves two further distinctions: all animal species have brains, but only humans have minds; and all other species have bodies, whereas only the human has a self. Brains, to take the first distinction, are physiological organs composed of material properties and represented by what in Mead's day was called the central nervous system.^[9] But in contrast to the study of the brain, Mead wrote, "it is absurd to look at the mind simply from the standpoint of the individual human organism." Because "we must regard mind . . . as arising and developing within the social process," in human forms of cognition the social mind complements the biological brain: "The subjective experience of the individual must be brought into relation with the natural, sociobiological activities of the brain in order to render an acceptable account of mind possible at all; and this can be done only if the social nature of mind is recognized."^[10] Mind, therefore, presupposes at least two brains. Mind supplements brain to the degree that an individual incorporates into his or her actions the point of view of another.

Can communication between a human and a machine therefore be considered mindful? Humans can, of course, put themselves in the place of a machine and identify with it, as was the case with Joseph Weizenbaum's ELIZA. Although a very early program, and offered more as a practical joke than an experiment in artificial intelligence, ELIZA is nonetheless helpful in understanding Mead's distinction between mind and brain. ELIZA prompts the respondent to answer the question,

How do you do? Please tell me your problem.

ELIZA will recognize certain key words in the responses given to the question and transform those words into another question. If the response, for example, is

I hate my father,

ELIZA will prompt,

Tell me more about your family,

and so on. One of the questions posed by such a program is not whether ELIZA possesses the Meadian quality of mind—clearly, it does not—but whether the human being interacting with ELIZA does.^[11]

If we take a human individual as the focus of our concern, and ask whether he or she has undergone a transformation in the process of talking with ELIZA, the answer would seem, at first glance, to be yes. After all, the person did exist in what Mead viewed as a triadic relationship: gestures elicited responses, which, in turn, elicited new gestures. Moreover, all this activity took place in the form of language, signs that were recognized by both parties. If our test for the possession of intelligence is the capacity to incorporate into our actions the responses of others as determined by some form of symbolic communication, the human subject interacting with ELIZA exercised qualities of mind. (For similar reasons, the triadic form of communication among birds, discussed in the previous chapter, would also appear, at first, to constitute communication in this sense.) Even more impressively, the human talking to ELIZA experienced growth; Weizenbaum, discussing what he learned from his experience, notes that many people were moved by their interaction with ELIZA, writing to him about how much they learned about themselves—all of which indicates that some people very definitely put themselves in ELIZA's place.^[12] If emotion, transference, catharsis, identification, growth, and transformation can all be experienced by a person through interaction with an artificial other, all the ingredients for mind would seem to be present.

Other programs direct our attention to the question of whether a machine talking to a person displays qualities of mind. Intelligent tutoring systems (ITS) are designed to flag down inappropriate questions from student programmers and check whether the programmer really meant to ask them.^[13] Possibly the machine has assumed too much knowledge (or too little) on the part of the student programmer, or perhaps the options presented to the programmer are too restricted. Under such conditions, ITS systems are capable of spotting errors by, in a sense, substituting themselves for the questioner. GUIDON, for example, was designed to supplement

the medical diagnosis program MYCIN. By comparing a student's questions to those asked by MYCIN, the program can determine when the diagnosis being followed by the student is off-track. GUIDON is also capable of analyzing the discourse patterns of the questions posed of it, to see whether they are consistent with earlier questions.^[14] And, as the limits of GUIDON are reached, other programs—such as NEOMYCIN, HERCULES, IMAGE, and ODYSSEUS—have been developed to refine its possibilities further.^[15] If the Meadian concept of mind is based solely on the ability to take the point of view of another, including another program, these programs would also, like the person communicating with ELIZA, appear to possess mind.

But there is more to the Meadian analysis than reflection by putting oneself in the place of another. The second Meadian distinction is between the body and the self. A physical body becomes a social self when an interaction with another social self has taken place. Since "selves can only exist in definite relationships to other selves," qualities of mind exist when a gesture "has the same effect on the individual making it that it has on the individual to whom it is addressed."^[16] No individual can therefore possess reflective intelligence—that is, be viewed as having a mind—without another individual who also possesses a mind. Mead's formulation is the converse of the Turing test: the other must itself be a self before a self can communicate with it. Human cognition, because it requires that we modify our thoughts by the ways we expect others to react to them, is therefore distinct from any other kind of cognition. By this definition, neither ELIZA nor intelligent tutoring systems would thus qualify.

Here, then, is a relatively parsimonious statement of the argument in favor of human uniqueness: Because we are biological creatures, we possess brains, but because we are social creatures as well, we also possess minds. This distinction between mind and brain is somewhat arbitrary. There are neurobiologists who argue for the existence of a "social brain," in the sense that many activities studied by sociologists, such as religious belief, can be explained by pure neurological functioning.^[17] On the other hand, there are theorists in cognitive science who argue that we have a "cognitive mind," that

thought has its own language, so that qualities of mind do not lie, as Mead argued, in things external to it.^[18] Some work in AI even suggests that there is no distinction between mind and brain at all, but only something called "mind-brain," combining elements of both.^[19] Still, if we accept Mead's distinctions for purposes of analysis, the questions we want to ask are (1) What would a machine need to do to approximate the qualities of the human brain? and (2) What would a machine need to do to approximate qualities of the human mind?

The question of what a machine would have to do to approximate the human brain is technological and biological in nature. Since the brain is composed of a series of neural nets, we should, in principle, be able to develop a machine capable of equaling, or surpassing, the information-processing capacity of human brains (although scientists who have tried to do so generally come away impressed by the information-processing capacities of human brains). If a machine or series of machines were capable of processing information faster and more efficiently than a human, we would have to reject the hypothesis that the brains of humans are superior to those of all other species.

The second question—What would a machine have to do to approximate qualities of mind?—poses a different set of considerations. If we follow Mead's conception of mind, we already have an answer by definition: no machine could ever approximate qualities of mind in interacting with a human, nor could a human show qualities of mind in communicating with a machine, since mind always implies the presence of at least two individuals. Yet, although the "human essence test" is in that sense rigged, it nonetheless helps us understand the importance of social context in the way any particular self understands the world. Real human beings do not make sense out of the world by searching through memory in an application of algorithmic rules. Instead, individuals rely on the social world around them to supply the contexts within which communication takes on meaning. To approximate qualities of mind, then, a machine would have to approximate the social environment that enables human minds to function. The question, in that sense, is not whether machine brains are superior

to human minds or vice versa. Rather, the biological brain and the social mind work in radically different ways: one seeks information as complete and precise as possible; the other does not need hard-wired and preprogrammed instructions—or even trial-and-error learning through the strengthening of neural nets—because it can supply meaning from outside itself. As with the study of animal behavior, we learn from AI that the unique properties of human beings are their interpretative and meaning-producing capacities—this time defined by the presence of other minds, since interpretation and reflection are possible only in a social context.

Software Intelligence

Work in artificial intelligence is generally divided into two kinds: software and hardware approaches. By a software approach is meant one that tries to model what the brain does without entering into the question of how the brain does it. Hardware approaches to AI reject an analogy with the computer and try to model intelligence and learning in machines upon a direct analogy with the neurological structure of the brain.

Early work in AI proceeded along both approaches, but it was not long before the hardware efforts, represented by Frank Rosenblatt and his ideas about perceptrons, gave way to software approaches, which seemed to offer more promising payoffs.^[20] These approaches assumed, as a thought experiment, that the brain resembles the Von Neumann architecture of a digital computer. Somewhere in the brain, according to this point of view, a central processing unit (CPU) stores information in the form of memory. Access to it could be made through instructions causing a search through memory in order to find the correct response. The beauty of this approach was that the human brain did not have to resemble a computer in a physiological sense. If a program could be written that could represent reality, then intelligence lay in the application of instructions or algorithms.^[21] Only one assumption was necessary for this procedure to work, and that was that a complete set of instructions could be provided. As Von Neumann put it,

"Anything that can be exhaustively and unambiguously described . . . is . . . realizable by a suitable finite neural network."^[22] Many researchers in AI are convinced that even the world of everyday life—of metaphor and ambiguity, for example—can be programmed into formal rules that a machine can understand, if only we specify them in all their complexity.^[23]

If memory were infinite, as it could be imagined in the purely abstract theories of Alan Turing and Alonzo Church, machines would simply take as much time as they needed to find the information relevant to an instruction, even if the time to do so was, say, the equivalent of three human lifetimes. But in the real world, researchers designing software approaches to AI soon discovered that the reality outside the computer was more complicated than anyone had realized. They therefore attempted to devise some indirect way of representing reality. One product of these attempts was the creation of "expert systems," programs based on a model of how experts in various areas make decisions.^[24] As long as they confined themselves to relatively limited domains of rule application, expert systems were a success, especially in medical diagnosis with the creation of DENDRAL and MYCIN. Such successes have led researchers within this tradition back to efforts to create more general programs, such as a unified theory of cognition, discussed above; but although such efforts have generated a good deal of excitement, their potential lies in the future.

Another way to get around the problem of representing reality was to take certain shortcuts—to assume that the reality outside the machine could be broken down into smaller and less complicated categories, which then could be combined. Minsky, for example, argued for "frames," while Roger Schank talked about the existence of scripts.^[25] Unwilling to reject the CPU metaphor—but recognizing that to hold information about everything that ever happened to us throughout our lives, such a human CPU would be unwieldy—these researchers assumed that memory is stored by humans in the form of episodes. Each set of generalized episodes could be called a script. In Schank's famous example of a restaurant script, a set

of associations is presumed to exist in the brain about what happens to an individual upon entering a restaurant; within this set of associations, any particular experience in a real-world restaurant can be framed. We may not, the argument ran, be able to represent the whole of reality at any one time, but we can break reality up into all its possible components and represent it that way.

Schank's ideas about scripts were subject to one of the most searing critiques in the literature generated by AI, Searle's effort to prove that machines cannot "understand" the instructions given to them.^[26] The implication of Searle's "Chinese room" critique was that there cannot be any artificial functional equivalent, not of human intelligence, but of human understanding. In the absence of the human capacity to understand, not even frames and scripts can avoid the problem that to know anything, the machine must first know everything.^[27] No wonder that many of the early criticisms of work in the area were directed primarily against software approaches.^[28] Even AI researchers themselves began to look for another approach, one based more directly on the hardware of the human brain.

Yet software efforts to create artificial intelligence did raise one fascinating question. If machines have trouble representing reality outside themselves, how do human brains do it? Humans, like machines, can also be given rules that they are expected to follow. But since our memories are imperfect, it would be difficult to conclude that our rules can also be programmed in detailed specificity if the same technique did not work for computers. Work in AI stimulated neuroscientists to take a closer look at the human brain, and some of them—especially those associated with Gerald M. Edelman—discovered that the whole idea of a CPU in the brain had to be rejected.^[29] The scientists who have taken this position offer instead what Edelman calls a nonalgorithmic understanding of the brain, or, more accurately, of brains; for he argues that different brains develop differently in the form of a selective system, just as certain species are understood by Darwinian theory to evolve in response to new challenges.^[30] Humans have, in other words, what Edelman and Mountcastle called a

"mindful brain," which software approaches, because they are dependent purely on algorithms, could not have.^[31]

But if human memory is not stored somewhere, how do we, in any particular circumstance, know what to do? Putting the question in another form, if it is true, as one neuroscientist argues, that "we are probably much better at recognition than we are at recollection," what we need to understand, in dealing with humans, is not how memory is stored but how it is activated.^[32] The answer may very well lie in what I have been calling the "human essence test" associated with the Meadian distinction between mind and brain. In contrast to the ideas about brains contained in software approaches to AI—ideas emphasizing the brain as a CPU, which, it turns out, may not have been an adequate model of the brain after all—humans have minds, which are capable of interpreting rules and instructions. We do not just search through memories to match a representation to a reality; instead, we fill in the frames or interpret the scripts, because our minds recognize the external reality that our brains cannot.

If this line of reasoning is correct, it would follow that the human brain, unlike machine brains, can be incomplete, that the brain does not need to understand everything with which it is presented, because we also have minds that make sense of the world to our brains. Work in AI unintentionally seems to show that humans are distinct not because their brains store more information than machines but because they can store less and get away with it. Our distinctiveness, in short, lies in the unknowability of the world around us. We gain access to knowledge by interacting with other minds, reflecting on what we learn from that interaction and experiencing growth as a result.

The role played by unknowability in the social world can be illustrated by theorists such as Harold Garfinkel. Although ethnomethodology was formulated in response to Parsonian structural-functionalism, it is highly relevant to the debates about brain and mind stimulated by AI research. For Garfinkel, conversations are interesting not for what is said but for what is not said. Thus, the words

Dana succeeded in putting a penny in a parking meter today without being picked up

might be difficult for a computer to process, because it would not know whether Dana was being lifted up to the parking meter or had not yet been met by his parents in their car. But even if a "natural" language program had anticipated this problem and could reject the incorrect meaning of "pick up" in favor of the correct one, would it be able to interpret the sentence to mean what one of Garfinkel's students assumed it to mean: "This afternoon as I was bringing Dana, our four-year-old son, home from the nursery school, he succeeded in reaching high enough to put a penny in a parking meter when we parked in a meter parking zone, whereas before he has always had to be picked up to reach that high"?^[33]

Research into human conversations stimulated by Garfinkel illustrates the difference between how human minds and machine brains talk. Although some researchers use conversational analysis as a method for understanding breakdowns in communication between machines and humans, the whole point of ethnomethodology is to analyze how people themselves develop the rules that structure what they do, including how they talk.^[34] Thus, to take only one example, Schegloff and Sacks showed that something as seemingly obvious as the closing of a conversation is a socially negotiated process between the speakers. If "there are possibilities throughout a closing, including the moments after a 'final' good-bye, for reopening the conversation," then human agency is always a third party to conversation between two human beings.^[35] Because human conversation is indexical (the meaning of words depends on the context in which they are uttered), computers have shown a remarkable inability to translate from one natural language to another.^[36]

John Bateman has pointed out that nearly all of Alfred Schutz's concepts can be translated into AI language; Schutz's stock of knowledge," for example, is the same as Minsky's frames or Schank's scripts.^[37] Yet despite this similarity, ethnomethodology seems to lead to an appreciation of how plastic our tacit understandings of the world tend to be. Schutzian phenomenology fills in the gaps that a formal analysis of gram-

matical rules can never fill: the everyday world provides the background or tacit knowledge that enables us to act in a contingent world—to act, as Dreyfus puts it, without a theory of how we act. Tacit knowledge, background assumptions, and practical reasoning are all features of mind that enable individuals to be rule-governed creatures, even if they do not know all the possible rules. The Wittgensteinian regress (the notion that the specification of any set of rules always contains a ceteris paribus condition that cannot be understood within the terms of the rules specified), while always a logical problem, rarely becomes a practical human problem.^[38] We can define the situation because the situation is not defined. We can construct meaning because the meaning is not known. Having gone through a period in which they tried to escape from ambiguity, sociological theorists are coming to appreciate it—in part, as Donald Levine states, because of "the recent ascendancy of computerized thoughtways."^[39] Ambiguity is essential to human communication. Because the world is infinite in its possibilities, we will never be able to capture it perfectly in the way we represent it to ourselves. This impossibility forces us to turn to others to share meanings, a social act that binds us together in human communities.

The differences between the knowing brain and the unknowing mind are illustrated by one of the activities that both machines and humans periodically undertake: the playing of games such as chess. As Georg Simmel once pointed out, in a metaphor exceptionally appropriate to the age of artificial intelligence, there are two conditions that would inhibit an individual from playing a game of chess. One is not knowing any moves. The other is knowing all the moves.^[40] Chess-playing programs developed by AI researchers cannot specify all moves; that is why heuristic rules were developed that eliminate nonsensical moves, making it possible for computer programs in the real world to play exceptionally expert chess. Yet let us grant one assumption of science: that if something is theoretically possible, we can imagine it to be practically possible. When the perfect chess program is developed, the result is to stop "playing" chess: when all moves are known, it can no longer be a game. A minimal condition for gaming, as Erving

Goffman once pointed out, is that "a prior knowledge of the players will not render the outcome a foregone conclusion." What makes a game a game is that interaction has taken place: "The developing line built up by the alternating, interlocking moves of the players can thus maintain sole claim upon the attention of the participants, thereby facilitating the game's power to constitute the current reality of its players and to engross them."^[41] Winning games is something our brains do; playing them is something our minds do. (That people both play and play to win only means that they have both minds and brains.)

There would be no need for mind if—not only in the playing of chess but in all other human activities as well—human agents acted with complete knowledge of the consequences of their acts. If the self knows the consequences that will follow from any gesture, speech, act, or form of behavior, it will no longer be a self. Human selves are distinctive not because they are "smarter" than machines but because they are "dumber." Not knowing everything there is to know in advance, they have to rely on social practices, the cues of others, experience, definitions of the situation, encounters, norms, and other ways of dealing with uncertainty that enable mind to develop. One of the leading German philosophical anthropologists, Arnold Gehlen, argues that because humans remain infants far longer than other creatures do, their specific traits develop out of their need to compensate for the lack of what nature has given them.^[42] The same can be said for their brains. Imperfect, trial-and-error, hesitant—the human brain is incomplete in the absence of a social mind. Even if a computer someday surpasses the human brain in its intelligence, it is unlikely to surpass the collective power of assembled minds. It is because humans have minds that we can speak of artificial intelligence, but almost never of artificial wisdom or judgment.

If the human brain and the Von Neumann architecture of the computer cannot model each other, software approaches to AI will never meet the first stage of the "human essence" test necessary to reject the hypothesis of human uniqueness: modeling the brain. It seems virtually impossible, therefore, that these approaches can ever reach the second state: mod-

eling the human mind. We are not sure how we comprehend the world outside our brains; but we are fairly certain that we do not do it through a detailed set of algorithms, specified in software or its functional equivalent, that model reality outside the brain in exact equivalence.

Hardware Intelligence

In part because of the kinds of problems just discussed, the software approach to AI is generally considered to have met a dead end. Indeed, some of the strongest criticisms of efforts to represent reality outside the brain have come not from those hostile to AI itself but from those committed to the kind of "hardware" approach with which AI began—an approach based on the work of Frank Rosenblatt and his notions about perceptrons.^[43] These approaches have been revived, but under a new name, generally called neural networks, parallel distributive processing (PDP), or connectionism.^[44] They represent the latest effort to prove that machines are capable of engaging in humanlike intelligence, thereby disproving the notion that any special characteristics are unique to us.^[45]

Rather than modeling how a brain decides without entering into the way it decides, hardware efforts to model intelligence use certain understandings of neurological behavior to develop analogous data-processing systems. Because the brain works much faster than computers, these thinkers argue, it must be composed of many computational devices working in parallel fashion. And because the brain does not necessarily store its memory in specific locations, waiting to be activated by signals that enter the system, its architecture is better viewed as a series of nets activated by the connections between them. In that sense, PDP approaches circumvent the most conspicuous flaws of earlier efforts: using a Von Neumann machine instead of the brain itself as a model for human intelligence:

One important difference between our interpretation of schemata and the more conventional one is that in the conventional story, schemata are stored in memory. Indeed, they are the major content of memory . In our case, nothing stored corresponds very closely to a schema. What is stored is a set of connection strengths which, when

activated, have implicitly in them the ability to generate states that correspond to instantiated schemata.^[46]

Two important considerations follow from this major shift in emphasis. One concerns rules and scripts. Researchers in the PDP tradition "do not assume that the goal of learning is the formation of explicit rules. Rather, we assume it is the acquisition of connection strengths which allow a network of simple units to act as though it knew the rules."^[47] It follows that the machine—more accurately, in this kind of work, a set of parallel machines—can learn, because it can react to ambiguous or incomplete instructions and furnish the context that can make sense out of them. While researchers in this tradition are cautious about making large claims for their work, they are convinced that machines can reproduce the human capacity to act in particular ways based on past experience.

One example provided by researchers in this tradition helps illustrate what is new about this approach when compared to older forms of AI work. Suppose that a child is in the process of learning the past tense of verbs. The general rule is that we take the present tense and add "ed." Following this rule, a naïve subject would reason as follows:

To respond to such a difficulty, earlier research in AI would have begun a search for all exceptions to the general rule, specifying them as precisely as possible, so that a machine would know how to respond if asked to give the past tense of a verb. PDP works in the opposite way. It begins with what the naïve subject would do, makes a mistake, corrects the mistake, and accumulates in the process enough associations that it eventually comes to learn when an "ed" ought to be added and when some other form of indicating the past tense is correct.^[48] In short, the reasoning here is trial-and-error reasoning, and is in that sense similar to the way humans think.

If software approaches to AI located intelligence in a set of

instructions to a CPU, hardware approaches locate intelligence in a set of procedures that can activate connections. "Under this new view, processing is done by PDP networks that configure themselves to match the arriving data with minimum conflict or discrepancy. The systems are always taming themselves (adjusting their weights). Learning is continuous, natural, and fundamental to the operations" of a system.^[49]

At issue is the way that "learning" organisms relate parts to wholes. One of the root assumptions of work in AI is that intelligence is manifested when enough very small bits of information are assembled together into something called knowledge. Working within a particular philosophical tradition inspired by Descartes and Hume, AI researchers believe that understanding how machines process information enables us to solve what Hume called the problem of the homunculi: that we cannot understand what takes place in the brain by imagining that a little man exists inside of it, giving it instructions; for then we would have to posit a little man inside the brain of the little man, etc.^[50] Their response to the Humean problem is to try to fashion a small machine out of exceptionally dumb components—indeed, the dumber the better. Although writing about AI in general, and not hardware approaches specifically, Daniel Dennett illustrates AI's response to the Humean problem:

Homunculi are bogeymen only if they duplicate entirely the talents they are rung in to explain. . . . If one can get a team or committee of relatively ignorant, narrow-minded, blind homunculi to produce the intelligent behavior of the whole, this is progress. . . . Eventually this nesting of boxes within boxes lands you with homunculi so stupid (all they have to do is remember whether to say yes or no when asked) that they can be, as one says, "replaced by a machine." One discharges fancy homunculi from one's scheme by organizing armies of such idiots to do the work.^[51]

As Douglas Hofstader has put it, the paradox of AI is that "the most inflexible, desireless, rule-following of beasts" can produce intelligence.^[52] But how, exactly, do they do it? Connectionist advocates of AI feel that they have a major advantage over software approaches in the way they produce intelligence. The human mind, they claim, does not work by

accumulating small bits of information, but instead works more holistically. Their approach (involving neural nets), they say, enables them to bypass the problem of dumb components by designing machines that can make associations. Yet the units of such systems are still electrical charges, and it is by no means clear that the associations made by such charges are in any way similar to the ability of human minds to incorporate realities outside the brain into the thinking process.

Even at their most sophisticated, parallel data-processing machines still think of intelligence as parts that somehow add up to a whole. In that sense, some of the problems facing workers in the field of AI are similar to problems that have plagued sociological theory. Durkheim's notion about the division of labor—where each human agent, generally acting in ways unknown to other human agents, nonetheless contributes to the effective overall functional performance of the society—is but one formulation of an age-old problem of how parts and wholes interrelate. Durkheim's division of labor, or any strong form of functionalist sociology, is an effort to understand how a smart organism—civilization or social structure—could emerge out of somewhat limited if not necessarily dumb components—people. In relying on a biological metaphor as the basis for his functionalism, Durkheim envisioned society as composed of hearts, muscles, heads, and other organs—all of them with tasks to perform, but none, save perhaps the head, with much consciousness or awareness of why it is doing what it is doing. It was precisely this sense of homeostatic structures and functions (inherited by Parsons from Durkheim) that led microsociologists, especially Garfinkel and Goffman, to pay more attention to individual human minds.

Just as sociological theorists were led to a greater appreciation of how micro and macro interrelate, researchers in AI are recognizing the limitations of the notion that a focus on the smallest possible parts will tell us something about the behavior of the whole. Marvin Minsky, for example, although once associated with software approaches to AI, has, like his colleague Seymour Papert, become sympathetic to the new approaches associated with PDP and connectionism. Confronting the question of how dumb components can make a smart

machine, he asks us to imagine that intelligence is a "society" composed of agents—such as the comparing agent, the adding agent, the seeing agent—each of which is ignorant of what the other agents are doing: "Each mental agent by itself can only do some simple thing that needs no mind or thought at all. Yet because we join these agents in societies—in certain very special ways—this leads to true intelligence."^[53] The overlap with Durkheim here is striking, and, also as with Durkheim, the question becomes whether PDP approaches can enable us to focus on aggregates—what Minsky calls "societies"—without attributing significant intellectual qualities to the parts that compose those societies.

It is worth noting in this context that Minsky reaches for the metaphor of "society" to talk about the whole, and he also uses the term agent to describe the part. The interaction between parts and whole seems to work for human societies because—among other reasons—human beings clearly possess agency: they can shift their attention back and forth from parts to wholes because they are autonomous agents capable of thinking for themselves. (It is the recognition of the power of human agency that led sociological theory away from an over-determining structuralism.) Can a machine premised upon parts that are dumb replicate the way real human agents operate in the world? Just as software approaches could have manifested intelligence resembling human intelligence if they could have overcome one fatal flaw—the need to specify descriptions of the real world as thoroughly and unambiguously as possible—the ability of the hardware approach to approximate human learning hinges on one point as well: does it make sense to apply the term agency to whatever is operating through a set of essentially dumb microprocedures that activate its states?

The role that agency plays in human intelligence is underscored by the same neuroscientists who reject the CPU model of the human brain. True, they admit, the PDP approach is closer in spirit to what we know about how human brains work; yet they are by no means convinced that these new approaches will enable machines to model the brain either.^[54] The reason has again to do with the social nature of the human mind.

Rosenfield has written that "the world around us is constantly changing, and we must be able to react to it . . . in a way that will take account of the new and unexpected, as well as our past individual experiences." The question, then, is how we come to "take account" of unexpected events. Rosenfield's view is in accord with the Meadian notion that human intelligence is manifested in the ability to interpret the meaning of stimuli outside the self: "Fixed memory stores, we have already seen, cannot accommodate the factors of context and history. Computations—the use of procedures in a limited way—bring us closer to a better solution but still fail to explain a crucial aspect of our perceptual capabilities: how our past affects our present view of the world, and how our coordinated movements, our past and present explorations of the world, influence our perceptions."^[55]

The clear implication of the work described by Rosenfield is that human brains work the way they do because the signs they recognize are not merely representations of parts but also interact with larger wholes in the culture outside of the brain. Human agency, in other words, is a central feature of human intelligence. The unit doing the thinking and learning must be capable of taking in the context of the whole if the parts are going to fit together in any coherent way. If this point of view is correct, then PDP and connectionist approaches to AI cannot reach even the first step of developing an engineering replica of the human brain unless they can demonstrate that their machines in some way model the human agent's capacity to understand the meaning of wholes. Yet it is precisely meaning which, in the PDP view of things, is sacrificed in order to specify microprocedures. As D. A. Norman puts it, "I believe the point is that PDP mechanisms can set up almost any arbitrary relationships. Hence, to the expert, once a skill has been acquired, meaningfulness of the relationships is irrelevant." Because "the interpretation of the process is not in terms of the messages being sent but rather by what states are active," it follows that "in general, there is no single reason why any given cognitive state occurs ."^[56]

PDP approaches, in short, attempt to solve the sociological problem of mind as a problem of engineering. What we get

by using them, even under the best of circumstances, is a machine that may resemble the human brain in an architectural sense, but one still without the capacity of the human brain to rely on mind to supply meaning. These approaches come somewhat closer than purely algorithmic AI methods to what would be needed to reject the hypothesis of human distinctiveness, but are still far from having done so. The hypothesis that what makes humans distinctive is the existence of an interpretative self, still safe from animals, would seem to be safe from computers as well. Humans are distinct because they can, together with those around them, rely on their ability to interpret social contexts, so that what they do not know does not become a hindrance in their ability to negotiate their way through the world. Recent work in artificial intelligence, like the second biological revolution, ought to give social theorists a renewed appreciation of how unique the human mind really is.

Computers, Humans, and Rules

The founders of modern social theory were stimulated to think about the specifically human features of their societies because they shared the intellectual air of the nineteenth century with Darwinian theories of evolution. Contemporary social theorizing, in a similar way, will inevitably be affected by the revolution in computing that marks our own age. As two writers have put it, "artificial intelligence . . . is beginning to tread in waters very familiar to sociologists, while sociologists could soon find [that] some of the methods and concepts of AI provide a novel, but reliable approach to their subject."^[57] Artificial intelligence promises to have the same relationship to social science as sociobiology has. Many argue for its irrelevance.^[58] At the same time, others try to apply its methodologies and principles to such classic sociological topics as Goffmanesque dramatological models, ethnomethodology, sociolinguistics and social cognition, and the sociology of medicine.^[59] And in the work of Niklas Luhmann, which will be discussed in greater detail in chapter 5, we have nothing less than a theory that

uses an analogy with the computer to explain how all systems, including social systems, operate.^[60]

But, as with sociobiology, the question is not whether artificial intelligence constitutes an adequate scientific framework for the study of human beings but, instead, whether the subjects of each science are intelligent in the same way at all times. At the core of the Meadian distinction between brain and mind are two different ways of thinking about intelligence. Brains, understood neurologically, can be imagined as information-processing mechanisms that work by following preprogrammed rules. But minds do not. Powers of mind enable people to incorporate information from social contexts and situations, thereby rendering huge stocks of stored memory unnecessary. What I called in the previous chapter tertiary rules—which enable humans to use their mental powers to alter the rules that govern them—are precisely the kinds of rules that are inappropriate to machines, even the most sophisticated of them. Because human beings have brains, some of the intelligent behavior they display is similar to that of machines (or other animals). But human beings also have minds, which require a unique science for a unique subject.

A summary of the different ways in which computers and real people think is presented in table 1. Each of the two approaches to artificial intelligence discussed in this chapter conceptualizes the problem of intelligence in a different way; and

both types of intelligence are in turn different from human intelligence, which is at least partly the product of qualities of mind. Specifically, these three forms of intelligence relate to rules, communicate, and conceptualize meaning in different ways.

The differences between computer and human intelligence in relation to rules could be illustrated in various ways, but one realm—that of play—is particularly appropriate for questions involving mind and the nature of the self. Charles Horton Cooley's work on the looking-glass self was stimulated by his observations of his own children at play.^[61] Similarly, observations of children playing computer games should indicate how they and their computers develop the rules by which this play takes place.

Sherry Turkle has given a succinct account of children's ordinary play: "In this kind of play children have to learn to put themselves in the place of another person, to imagine what is going on inside someone else's head. There are no rules, there is empathy. There are no dice to roll, there is understanding, recognition, negotiation, and confrontation with others." In computer games, by contrast, precisely defined rule structures make empathy and role playing unnecessary. "You can postulate anything, but once the rules of the system have been defined they must be adhered to scrupulously. Such are the rules for creating 'rule-governed worlds.' They are known to every computer programmer and are now being passed on as cultural knowledge to a generation of children. The aesthetic of rule-governed worlds is passed on through Dungeons and Dragons and science fiction before most children ever meet a computer."^[62] It is as if the rigidity of the rules in the computer compensates for the absence of predetermined rules that marks human play.

These ethnographic observations sharpen our understanding of how machines and human minds relate to rules. The most extreme form of rule adherence is contained in the software approaches to AI, for there the rules are everything. The failure of such programs—or, more precisely, their success only on condition that heuristics, efforts to generalize about rules rather than to specify them, become the dominant way to run

them—indicates that concepts of agency based on the notion that the agent is a rule-follower and nothing else are inappropriate to human beings. Software approaches, being the most rule-bound, are like sociobiology at its most deterministic. Such ways of following rules may be appropriate to some kinds of software programs. But because human beings do not govern their affairs algorithmically, we must, if we are to model how they relate to rules, introduce the possibility that rules can be changed.

It is clearly possible, as hardware approaches have demonstrated, to make machines that will follow rules even when the rules are ambiguous. PDP and connectionist approaches are similar in that sense to sociobiological understandings of culture: they add new elements to a scientific understanding of how organisms behave. By introducing these new elements—elements of mind (or learning)—they avoid purely algorithmic reductionism and take a step closer to approximating how humans think. But the theory of mind that they introduce is minimal. They postulate qualities of mind that allow flexibility in rule-following but not qualities of rule interpretation. PDP and similar approaches remain strongly rule-driven; in the words of Terry Sejnowski, a leading connectionist researcher: "It's not like we're throwing rules away. This is a rule-following system, rather than a rule-based system. It's incorporating the regularities in the English language, but without our having to put in rules."^[63]

If, in contrast to both approaches in AI, we understand the powers of human minds to lie not in how we follow rules, however creatively, but also in how we make them, we need a model of mind capable of interpreting reality outside itself. No machine yet developed is capable of taking the outside world into context, so that it can make the rules it will then follow. A science that is so algorithmic in structure as to negate the possibility of agents following their own rules will not be an especially successful science for mindful human beings.

An emphasis on rules, in turn, raises the question of how they are transmitted. If—following Pagels's terminology^[64] —we distinguish between signs (representations that have only one meaning) and symbols (representations into which other mean-

ings can be read), then machines can manipulate signs, whereas minds can interpret symbols. Thus, a Turing machine recognizes a string of is and os as a set of instructions indicating whether any given switch should be on and off and as an instruction to the next switch; and this property makes possible an extremely impressive set of computations, including those computations that are translated into the signs constituting the letters of this text (and making it possible for me to move portions of this text from one place in the manuscript to another). But no machine, and certainly not the one upon which the signs in this book are being processed, understands what the combinations of signs that form the words I write mean to the reader who is trying to determine whether my argument makes sense or not. In other words, like certain other species found in nature (about fifteen, according to Lumsden and Wilson), computers are capable of reification, of substituting shorter signs for longer signs. But they are not capable of turning signs into symbols.

As long as the instructions given to computers are understood to be signs (and in the PDP version of AI they may not even be signs, but an even smaller electrical charge that can only be called subsigns), little confusion results. But much of the AI literature does refer to machine instructions instead as symbols, thereby forcing some definitional clarity. One way to think about the matter is suggested by Pagels when he notes that symbols are "top down" whereas signs are "bottom up."^[65] That is to say, humans recognize symbols as whole configurations and can disassemble them to account for their parts, while signs are the individual elements that together form a symbol. Given the complexity of their parts, symbols are open to interpretation; given the simplicity of their parts, signs are not. Brains, including artificially created ones as well as complex ones found among primates, can manipulate signs; reification is a purely self-referential process. But because the meaning of a symbol does not exist within the symbol but has to be interpreted by a mind, only a species capable of interpretation can attribute meaning to a symbol. A science of mindful human behavior requires that attention be paid to the ways in which symbols help individual minds make sense out of the

world, whereas a science of machines (like a science of primates) need only consider the ability of one sign to substitute for another.

Because of the difference between signs and symbols, machines and humans respond differently to questions of meaning. Meaning is formal and notational in some kinds of AI research, especially those based on the software model. In such a model, formal modes of expression enable thoughts to be represented by means of syntactical rules (or grammars) that can be rendered into computations. For this very reason, as Jerry Fodor writes, "the machine lives in an entirely notational world; all its beliefs are false."^[66] That is, machines practice "methodological solipsism"; they process data as if there were referents in the real world to be interpreted, without, of course, ever interpreting them. (This feature of AI research is elevated into a methodological principle by Dennett, who argues that we can take an "intentional" stance toward machines, ascribing to them certain features without necessarily making an argument that they possess those features in reality.)^[67] The development of PDP models reinforces the point, for these versions of AI do not, as the software versions sometimes did, claim that they are representing the real world—except the neural structure of the human brain. For them, meaning lies in the strengths of connections between nets and nowhere else. Meaning is always internal to the dynamics of a system.

The notion that meaning is internal to any system has become increasingly popular—for example, among literary critics reluctant to consider anything outside the text (chapter 5 will examine such self-referential theories). In sociology as well, meaning has been redefined to refer only to the ways in which communication takes place, rather than to the message being communicated.^[68] Since some aspects of human behavior are algorithmic, AI research has some, probably minor, direct applicability to the human sciences. But the unique features of human intelligence exist outside the brain; and to grasp those features, we need a science that can add the study of minds to the study of brains. Compared to computers, humans have one minor disadvantage: they calculate more slowly. But

they also have one tremendous advantage: they can bring the whole world into their minds.

The reliance on the computer as a model for the operations of the human mind may well prove to be as limited as an earlier fascination with behaviorism. Psychologists have already begun to recognize the flaws of the mind-as-computer metaphor. Some have turned to cultural anthropology as a way of understanding the factors outside any particular individual's brain that help the individual make sense out of the world.^[69] Others, such as Jerome Bruner, one of the founders of cognitive psychology, find that the computer analogy carries with it an unfortunate shift "from the constuction of meaning to the processing of information."^[70] Bruner's reflections on the limits of the machine model lead him back to a humanistic emphasis on narrative and story-telling as essential ingredients in understanding human cognition, a fascinating intellectual journey that reveals both the power and the limits of computers as guides to the way we think.

Rather than challenging the anthropocentrism with which sociology began, in short, the development of machines that think actually reinforces it. What research into AI seems to show is very similar to what sociobiology inadvertently demonstrates: both fields, originally perceived as a challenge to the notion of a humanistic subject, strengthen the notion that humans require a distinct science because they are a distinct subject. And in looking at what makes them distinct, socio-biology and artificial intelligence lead to a similar conclusion: humans not only add culture to nature, as important as that is; they also add mind to culture. One best appreciates the powers of imagination and interpretation when confronted with a thinking machine that possesses neither.