24—
Subjectivity in the Machine

I—
Thinking and Subjectivity

In this paper I shall propose what I take to be a timely shift of attention from the question of whether a machine can think to the question of who, if anyone, a thinking machine is. But there is life in the old question yet, and one way of looking at it provides a bridge to the new question. This is the strategy of treating "to think" as deponent verb, and it opens up the whole issue of thinking as behavior versus thinking as experience.

Deponent verbs, which in Latin grammar were passive verbs with an active sense, were so called because their original passive sense had been "put aside," deposed as it were, in favor of the new active one. So they involved a history, the history of a transition from something that happened to people to something they did . If this can be traced within a living language it must have been very recent (though the whole history of language, indeed the whole of history, is recent in evolutionary terms). In the case of thinking such a transition does seem to have taken place—indeed, it is still incomplete, the passive sense being preserved in many locutions in current use: "it seems to me," "it occurs to me." We have dropped the archaic passive form "methinks," but it remains in the language as an archaic form; it was current only yesterday, as it were. And we can get a sense of what the transition means by trying it out on another form: take "it seems to me" and transform it into "I seem it to me." That is what "I think" in fact amounts to—from having thought contents occur whether we want them to or not, we move to a position of control, we choose what to

think, we "make up our minds" (consider the transition from the old form 'I have a mind to . . . ," or its current descendant "I've half a mind to . . . ," to "I've made up my mind to . . . ").

Thinking has an authentic etymological connection with seeming or appearing; what I do when I think is to bring something "before" me, in a sense yet to be specified—the expression that comes to mind (the passive position again!) is "before the mind's eye," whose usefulness as a metaphor is however limited because of the tendency of the visual to preempt the field of attention. I will return to this point. The main thing to notice here is that the shift from passive to active takes us from the object position to the subject position. And this is something that happens gradually, ontogenetically as well as phylogenetically, and that requires our participation. Being able to do it is what it means to be a subject in the active sense, to be an agent in the process of thinking rather than a patient. But doing it does not mean the end of thinking in the other sense.

The two possible ways of construing "thinking," then, are as a process that goes on, that happens to us, and as an activity undertaken, that we engage in. Only the latter involves subjectivity essentially. But just what do we understand by this "subjectivity"? And, since understanding and explanation are correlative in the philosophy of science, is there an explanation of subjectivity? To deal with these questions in order: by "subjectivity" in this paper I mean the condition of being a subject, or being in the subject position, in relation to objects known or acted upon—being situated, that is, at one pole of a vector of attention or intention, the pole characterized by noetic consciousness as opposed to the noematic contents of consciousness. I do not mean "subjectivity" to be contrasted with "objectivity" where the latter is used, as it sometimes is, to mean a commendable detachment from affective influences on judgment. If I am to preserve a scrupulous objectivity I cannot allow "subjective" factors to influence my conclusion, so it looks as if I have to keep something called "subjectivity" at bay. But I have to be a subject , hence to have subjectivity in the sense in which I use the term here, if I am to preserve or conclude anything, or know that anything has been preserved or concluded. Another way of putting this is to say that objective knowledge presupposes subjectivity because any knowledge does, knowledge being just such an intentional relation.

Can subjectivity in this sense be explained? There are two main ways of going about the business of explanation, which I will call working out (from an intuited center) and working up (from a postulated base); their paradigm cases are respectively phenomenological and hypothetico-deductive explanations. Ideally these processes meet in the full ex-

planation of a given object, process, or event. This shows not only how the entity in question fits into a causal network but also how we have access to it in experience. But a systematic difficulty arises in any attempt at an explanation of subjectivity: we must start with it in working out, but we can't get to it by working up. This is, again, for the obvious reason that even working up presupposes it: subjectivity is intentional, and because both the explanandum and the explanans are intentional objects it is required if they are to be evoked.

This is a crucial point. I use "intentional" here in Brentano and Husserl's sense, not in Dennett's sense,^[1] which, although it catches admirably how systems with Brentano-Husserl intentionality might behave (or how their behavior might be explained), it doesn't succeed in showing—and to do Dennett justice doesn't try to show—that a system the explanation of whose behavior requires the "intentional stance" needs to have intentionality in the Brentano-Husserl sense. Dennett doesn't think this matters, but I think it makes a tremendous difference. Intentionality is just the feature of subjectivity that directs its awareness towards objects (or is aware of its direction towards objects). "Intentions," in the sense of "purposes," are a familiar case of this, though they account for no more than a small fraction of intentional activity; attention, to which I shall return below, is a special case in which the object is presented (normally in perception), so that its constitution as what it is requires no active contribution from the subject.

Of course the only subjectivity presupposed in this way is my own. Its intentionalities, however, are what make my world and my project and make them meaningful. Or perhaps having a meaningful project in a meaningful world is just what it is to be a subject. Quintilian's definition of the rhetorical form subiectio , as "the suggesting of an answer to one's own question," suggests a possible definition of subjectivity: to be a subject is to be in a position to suggest oneself as the answer to one's own question. One of the things I think is myself. But I think myself differently from the way I think objects, and that is one of the difficulties in the way of a general explanation of subjectivity. Freud to the contrary notwithstanding, the subject can't be made an object; I can't give my own subjectivity as the answer to anyone else's question, nor can anyone else give theirs as the answer to mine. But if we can't produce subjectivity as an explanandum on the basis of an explanans we can perhaps at least locate it, and indicate the kinds of structure and experience that are concomitant with its occurrence.

We have no evidence at the moment of any cases of subjectivity other than our own, and (conjecturally) that of some of the higher mammals. All of them are associated with the advanced development of central nervous systems. But central nervous systems can become ex-

tremely complex without the emergence of subjectivity. What we find we want to say is that there cannot be subjectivity without the full activity of thinking, and that brings us back to the old question. The distinction between thought as process and thought as activity closely parallels the distinction between thought as behavior and thought as experience: any system can behave, whether conscious (or subjective—but the terms are not synonymous) or not, but only a conscious subject can have experience. (The "ex-" of experience," though in its origin a plain "ex-" like any other, shares with the "ex-" of "existence" the sense of a standing-out from something: ex-perience is a coming-out from a going-through, but there has to be some continuity, some substrate, for it to be cumulative.)

II—
Thinking as Behavior

"Thinking as behavior" is itself ambiguous. It has two senses, one relatively straightforward, the other at once more trivial and more profound. The straightforward sense construes the behavior associated with thinking as a behavioral output that is taken to result from thought. It helps in circumscribing the place of thought in the process if there has been an input which has triggered it by offering something to think about. So the paradigm case is the answer to a question, and the classical form of questioning is Turing's imitation game.^[2] A machine that plays the game proficiently, so that its "opponent" thinks it is a human being, has to be admitted to have been thinking, or at any rate doing something that, if a human being did it, would count as thinking.

Turing argued that there could be no reason to deny thought to the machine if it satisfied the test by which we attribute thought to human beings. Is the imitation game such a test? There is clearly something it tests: it is by his or her answers to my questions (or responses to my conversational moves—outright interrogation is not the norm of social intercourse) that I judge whether my interlocutor is awake, English-speaking, intelligent, knowledgeable, witty, thoughtful (which is only one mode of thinking), gifted at languages or mathematics, a compatriot, a fellow-enthusiast, a professional colleague, etc. Mostly of course I assume these things and am disappointed when he or she turns out to be inarticulate, slow-witted, or fraudulent.

The imitation game in fact seems to be less a test of thinking than of the membership of one's interlocutor in one or more of a number of linguistic communities, all the members of all of which are assumed to be capable of thought. But on reflection it does not seem obvious that

if a machine passes the test of admission to such a community, it necessarily follows that this assumption operates in its favor too. What does the assumption rest on? How do people learn the language of thought—not the "language of thought" in Fodor's sense but the ordinary language by which they refer to thought?

They precisely can't learn it by behavioral cues except in the trivial sense: furrowing brows, banging foreheads, etc. But the more profound implications of this sense—the behavioral concomitants of thought, rather than its behavioral consequences—are connected with the realization that sometimes it is just the absence of behavioral cues that indicates thinking. A child who disturbs an immobile and silent parent is rebuked for it—"Can't you see I'm thinking?"—may learn to be silent and immobile too at tricky junctures and may thereby learn to attend to something in itself that would not otherwise have been attended to.

An anecdotal example of the point at issue here was recently provided by a striking scene in a play called "Whose Life Is It Anyway?" Its main character was a hospital patient paralyzed from the neck down. At a climactic point in the play a judge, who had to decide whether to grant the patient's wish to be allowed to die and who was in perplexity about this decision, walked downstage and stood motionless for what was in dramatic terms a very long time, his hand to his chin, obviously deep in thought. It was understood that in this moment he was actively exercising his highest human capacities, his high judicial function; and yet in so doing he was as it were symbolically paralyzed. He wasn't doing anything that the patient couldn't do, and the spectator was led for a moment to reflect that if the highest mode of functioning as a human is compatible with immobility, then the patient's immobility was no bar to his functioning at the highest human level.

That wasn't how the play came out, but it serves my purpose by sharpening the question of what is going on while we are thinking, what attention to the process of thought reveals, and whether attending to it has anything essential to do with the process. It might turn out to be the case that we are thinking beings who happen to be able to observe some of our thinking processes consciously even though there is no need for us to do so, like passengers on a ship who are allowed to go up on the bridge and watch the captain and hear his commands, though the ship would sail on just the same if they stayed below. And it might turn out that machines can't do this, in which case they would be thinking all right but there still might not be anyone there. On the other hand it might turn out that our role on the bridge is as captain, forming an essential link in the causal chain of command. And if there were some

maneuvers that couldn't be done without the captain's intervention, and the machine managed those too, then we would have to ask what in it corresponded to the captain.

Before anyone raises the obvious objection to this metaphor I had better do so myself. It sounds like a homunculus theory, and even Descartes saw that that would not do. He uses much the same image: "I am not only residing in my body, as a pilot in his ship, but . . . I am intimately connected with it, and . . . something like a single whole is produced."^[3] So it is not a question of a part of me observing another part, but of my exercising a reflexive or self-referential capacity that may or may not be essential to the process of thought. This "essential" remains problematic though—it might turn out that the captain himself served a merely decorative function, and that the ship would sail smoothly on even if he stayed below.

III—
Attention and Intenttonality

Certainly there are functions of what we all recognize as thought that we can't attend to even if we try. If someone asks me for the product of six and seven and I say "forty-two," there is just no way in which I can catch any thought-content between the question and its answer. I can of course complicate things so as to provide one—six sevens is the same as three fourteens, which is three tens plus three fours, two-and-a-half of which will already make another ten, and so on. But that is only like climbing the steps at Lourdes on one's knees so as to draw attention to the process.

The concept of attention keeps cropping up and itself needs to be attended to. It makes a pair with intention and the contrast between them is instructive. This has something in common with the contrast between discovery and invention, in that we attend to what is already there but intend precisely what is not yet or is no longer there, or what never was or never could be there (golden mountains, round squares). It is the difference between finding something and creating it. Subjectivity, in the phenomenological sense, is as we have seen intentional by definition: every consciousness is a consciousness of something, and intentionality points along the axis from neoesis to noema. This pointing however is an activity of the knowing subject, which sustains the object of thought; it suggests that the subject is also agent . The standard cases of intentional objects, just referred to, don't we suppose present themselves; they need to be thought of . (Their claim to ontological status consists wholly in the fact that they are objects of thought.)

What if subjectivity were merely attentional ? Consciousness would

still have a content all right, but the question of agency wouldn't arise; we would have to assume that thought proceeded automatically and that the various contents of consciousness associated with it, including the feelings of deliberation, agency, etc., were just given, as we are normally convinced some of them really are (for example, hypnagogic images and phrases). This is in effect the position familiarly known as epiphenomenalism. If it is correct then there can't be any way of knowing whether there could be subjectivity in the machine, but on the other hand it won't matter much.

Epiphenomenalism is about as uninteresting, philosophically speaking, as strict determinism, since if either of them is correct, there's nothing we can do about it; in fact there's nothing we can do period—it all just happens. It may be philosophy, it may be sex, it may be pain, it may be madness—we're just along for the movie. Perhaps someone wrote the script for the movie, perhaps not, it makes no difference, we're strapped into our seats, no climbing out of this cave. And these views might in fact be correct (I know of no argument that can block that possibility in either case), but if so my conjecture that they are or aren't correct has no weight whatever: it's just something that got conjectured in my movie, I can take no credit for it, I 'm not in it. Possibly the machine is watching its movie. If it is, as I shall suggest in a minute, it may or may not be enjoying itself, and that might have implications for us as machine-builders—but not in an epiphenomenalist world; there there aren't any implications of anything for us, just movie-implications at best, as unreal as screen kisses or champagne.

Still once one thinks of it one has to admit that a lot of human experience is merely attentional. So the capacity for conscious attention might be an evolutionary dead-end, a freak of which we happen to be the complacent (or on reflection the astonished) beneficiaries. But it seems just as likely that it conferred some evolutionary advantage, that attention made intention possible. The attention-intention loop constitutes, one might say, a conscious version of the sensorimotor loop, and may indeed be inserted in it, although this is not necessary either to the success of some sensorimotor processes or to the meaningfulness of some intentional responses to attention.

IV—
Sensation and Consciousness

This way of putting it provides a clue as to the emergence of subjectivity out of mere consciousness. First we have peripheral reflex arcs, then sensorimotor coordination through the central nervous system: so

far no consciousness. Then the complexity of sensorimotor activity, its multidimensionality, its necessary swiftness, make it desirable to give the relevant inputs analog form as what we call the "senses," and to represent the state of the ambient world by mapping its features into a visual space into which the position of the body and of its parts can also be mapped. (More accurately, the independent development of such an analog representation makes greater complexity, speed, etc. of sensorimotor processes possible for the organisms that have it.) Visual awareness, let us suppose, is just this mapping. It has its attentional and its intentional aspects: the input is something that as we say "catches" our attention, the output is the directed reach, first of sight (we look rather than just noticing) and then of the appropriate muscular complex. The sum of sensory awarenesses and their derivatives will constitute the content of consciousness.

Any sensorimotor loop, it might be argued, involves representation or mapping of some kind: at the very least the visual (or auditory or olfactory or tactile) space from which the input arrives will map on to the motor space to which the output is directed. However we need our system of representation to recognize the mapping (and the spaces between which it holds) as such—otherwise it's just a bit of causal machinery. In more complex cases, when the loop doesn't put its signals straight through but engages some more complex neural structure, possibly involving time delays that make strategic computations possible, we may find what look to us like internal representations that map into both spaces, sensory and motor. But do they make the input and output spaces "look like" the same space to the organism in question? Is computation here also deliberation of a sort? No simple answer, it seems to me, can be given to these questions. In our case it is so, which suggests that something similar probably holds for other organisms of comparable complexity.

Once the analog representation is in place (let us call it the "sensorium"), the organism can do all sorts of interesting things with it, especially if a good memory is also available. But the development, from the immediate and vivid sensorium (largely preempted by perceptual contents, which when available tend to overwhelm competitors for attention—this explains why the library in which I am writing is quiet, and decorated in muted colors), of the intentional domain required for thinking and subjectivity, must have been long and slow. Once it had learned to attend to the contents of perception (selectively no doubt—at first to movement and change, for example, rather than to constant features), the organism could begin to attend to sensory contents remembered or abstracted from memory. Later on—and here is the transition to agency—it could intend them also (imagine, project, etc.). At

this stage there need be no actual sensorimotor involvement at all: hence the immobility of the thinker. This activity is by definition conscious, and there can be little doubt that many animals share it with us (dreaming dogs, for example). The question is, how much of the activity of thought necessarily goes on there , rather than going on elsewhere and being reported there, or not, as the case may be?

I am suggesting that some forms of complex sensorimotor coordination may have required the insertion of an attentional-intentional segment, that this may have been what made them possible. Or perhaps it is what made learning them possible. In this connection Schrödinger's conjectures as to the evolutionary role of consciousness are relevant. Schrödinger thought that consciousness had a phylogenetic function: "I would summarize my general hypothesis thus: consciousness is associated with the learning of the living substance: its knowing how is unconscious."^[4] But once we know how there seems to be nothing to prevent our programming complex coordinations into the machine without requiring it ever to be conscious; we could give it the appropriate transform between visual space and sensory space instead of making it establish the transform for itself, as seems to be the case with us (although even with us a lot of that seems to happen automatically—binocular vision, for example—only learned refinements requiring conscious monitoring).

V—
The Ascent to Subjectivity

It is tempting to try to establish a developmental sequence, an ascent from the inertia of matter to the reflexive consciousness of the subject. I offer here one possible account of such an ascent, in terms of states (S), representations (R), and an operator (~) which I shall call the "matching" operator. Matching, as I have explained elsewhere,^[5] is not a simple correspondence but has an active sense—among things that match are equal numbers and identical colors, but also left and right gloves, keys and locks, musical phrases and their repetitions or transpositions, bits of jigsaw puzzles and the spaces they fit into, and so on. Above all, in language the signifier matches the signified (indeed I consider matching to be the fundamental phenomenon of signification).

The stages of the ascent are represented as (a) through (f) in figure 1. The states are states of some individual entity, which at stage (c) and above is an organism. At stage (a) the entity is not differentiated from its environment, so that there is no sense to a contrast between internal and external states. At stage (b) there is differentiation but no metabolism, so that nothing about the environment matters to or affects the

Figure 1

entity. Metabolism is the minimum condition for the entity's functioning as an organism. At stage (c) and above, then, the external state S_e is the state of the immediate environment as it affects the organism , ignoring irrelevant conditions even though these might seem, from some other point of view, the most salient features of that environment.

Stage (c) itself represents a situation in which the organism reacts to something in its environment, salinity or temperature, for example, so that it moves in such a way as to bring its internal state S_i into equilibrium with the external state and then stops moving until either S_i or S_e changes with time, in which case movement recommences. The matching takes place across the boundary between the organism and the environment. But at stage (d) I postulate a causal process that forms within the organism a representation of the relevant state of the environment. There is obviously no need for the representation to resemble anything in the environment, as long as changes in the environment produce corresponding changes in the representation. Now the matching becomes internal to the organism, and a mismatch, which leads as before to motion until it is corrected, is no longer a transaction between organism and environment. I suppose that at this stage the organism is sensitive , and that the mismatch, constituting as it now does a state among others of the organism itself, is felt , perhaps in the limit as pain—indeed almost certainly as pain, since presumably the avoidance of gross mismatches is of evolutionary significance.

It has to be admitted that here (as at every point in the ascent) my

assertions are wholly conjectural, but of course just such conjectures, which if true would account for the conditions they are intended to explain, are the stuff of scientific discourse. At stage (e) I envisage a double representation, not only of the state of the environment but of the state of the organism that requires to be brought into equilibrium with it: the monitoring of the match between these representations I take to signal the emergence of conscious awareness, though this is still tied to immediate sensory contents. A centrally important special case at this and the subsequent stage will be the case in which the internal state represented by R (S_i ) is an element of memory ; the availability of large numbers of memory-states is part of what makes consciousness and subjectivity as complex as they are. Finally at stage (f) the dependence on momentary sensory inputs is overcome, and consciousness is free to attend to matchings of pairs of internal representations. This is the condition for subjectivity: consciousness can build up an internal history, can lead a life of its own independently of the external state of affairs, provided the latter does not obtrude in the form of representations on a lower level that demand attention because of painful disequilibria. The number of available internal states can, with time, grow very large, and the life of the subject be almost wholly internal.

VI—
The Place of the Subject

In speaking of "monitoring" or "attending to" matchings of representations there is a risk of misunderstanding that I would like to try to dispel, though since the corresponding form of understanding can at best be metaphorical, this may not be easy. My subjectivity is of course engaged in the writing of this text, as is the reader's in reading it; as we look at the representations of the hierarchical stages, for example, at (e)

the vector of our intentionalities is roughly orthogonal to the plane of the page—we attend to the postulated matching of two states but our own subjectivity is a third element in the situation. But in (e) and (f) I do not mean to suggest that the consciousness or the subjectivity of the organism is a third element alongside the two matched elements; on the contrary, it is their matching . In the perceptual case it will look to the organism as if there is only one state, and that external, perception being of objects in the environment. But all the organism has subjective access to is a representation of the external state, and all representations are internal; what is happening is that the internal representation

(not necessarily resembling the external state but determined by it and responsive to it) is being intended (or attended to) by the organism, which for the purposes of conscious action or reaction is as it were lining it up with, sighting it through, correlating it with, a representation of an internal state, a memory-state for example. All these images of comparison are unsatisfactory because they invoke two entities being manipulated by a third, whereas the subject subsists in the dynamic relation of the two.

The net result of the matching that produces the subject is the intending of objects, of a world. In an earlier text I defined the subject as "what permits the integral, continuous, and possibly repeated apprehension of the object, in the moment of this apprehension and abstracting from purely physiological conditions of perception . . . 'Integral' does not require a total integration of the object in itself . . . and 'continuous' does not require a very long time—but enough. . . . Continuity implies, one might say, a repetition from one moment to the next; the further possibility of the repetition of a whole episode of apprehension, the recognition of the same object after a more or less prolonged absence, implies the 'genidentity' of the subject as an individual and of its own point of view."^[6] So for the subject in the diagram the vector of intentionality lies along the line, in the plane of the page; at stage (e), the first at which talk of intentionality makes any sense, one can imagine its pointing from left to right, sighting the external world through its representation, as it were. In that case we might be tempted to locate the subject itself as a kind of virtual origin of intentionality somewhere to the left. But this would be misleading at best, and by the time we reach (f) the assignment a priori of such directionality makes little sense. In general we might suppose that the vector goes from signifier to signified, but in the ramified network of representations that exists at this level it may become problematic or meaningless to perpetuate that distinction.

The subject in other words is coincident with the matched elements. The somewhat elusive nature of this relationship was anticipated (in what may seem an unlikely quarter) by Jean-Paul Sartre in his doctrine of the "prereflexive cogito."^[7] Sartre's argument in effect was that if I am conscious of something, I am at the same time conscious of being conscious of it. So in the matching of representations I am aware of the matching as well as of what is represented and the form under which it is represented. This double awareness is I think an essential feature of subjectivity. (It can of course be more than double, since the structure of the prereflexive cogito lends itself to recursion, though in practice I suspect that there is a limit to the number of terms in the series that can be attended to at once, and that two terms—the first two—are the

norm.) Saying this however is of no help in explaining how subjectivity is possible. As I suggested at the beginning, I think that such an explanation is unavailable to us in principle. The fact of subjectivity is, in the strict sense, absolute: as a problem it does not admit of solution.

Note that in the ascent described above "representation" is in a somewhat similar situation to "intentionality" in the early part of the paper: a specification of the relation of representation might be drawn up, that would be met by cases in which subjectivity was present, but at the same time the fact that a case met the specification would be no guarantee whatever that subjectivity was in fact present. Indeed, such specifications have been drawn up, by Churchland, Pylyshyn, and others, and they turn out to be nothing more than sophisticated versions of mapping, just as Dennett's intentionality turned out to be a sophisticated version of explanation. It seems to me unlikely that we will understand represent ation in thought until we have understood present ation in perception—and then (perhaps) in thought too.

VII—
The Sensorium as Monitor

What exactly is "present to the mind" in thought? I come now to an even more conjectural part of my paper, which belongs to the higher reaches of the ascent, after all the stages hinted at above. Note, again, that we might go up an ascent like this one (e.g., one that mimicked it behaviorally, with delayed reactions that looked like "allowing time for thought") without activating the sensorium or having genuine cases of subjective presentation or representation. One reason for this is that an organism (or a machine) can have sensors without having a sensorium. The old Turing problem, this time a bit closer to realization, comes up implicitly in Valentino Braitenberg's genial menagerie of "vehicles."^[8] His second simplest vehicle—with two sensors and two motors—already exhibits "fear" or "aggression," depending on whether the sensor-motor connections are parallel or crossed over. This reinforces the view that there is more to thinking than meets the behavioral eye (or less: the immobile parent lost in thought may in fact be merely daydreaming), but again it's we who interpret the behavior as fearful or aggressive.

I will begin this further conjectural development by likening the sensorium to the display of a computer, a very fine-grained display with something on the order of 10⁷ pixels. The display can be used to map a visual field corresponding to perceived features of a world, but it can just as well be used for text. (Of course text can be and often is found in the perceived visual field, since reading and writing are the dominant

sensory and motor activities of organisms above a certain level of acculturation—note that "sensorimotor" would be inappropriate here since there is normally a wide separation, and an extremely complex correlation, if any, between the two activities.)

Text in the sensorium will consist of sound-images (in Saussure's sense) or visual images of letters, etc., and it will coexist there with other sensory elements, with complexes of which textual complexes may be matched, in the first instance just as Saussure says they are in his theory of the linguistic sign. These matchings, achieved by an ability that I have called "apposition,"^[9] might be stored—the mechanism of this storage is for my purposes a matter of indifference, although any attempt to realize this scheme would have to pay careful attention to it—in a memory capable of expansion, in such a way that the appearance of one element in the display might cause the other to be retrieved. The process of retrieval would not be conscious, since the display itself is the analogue of consciousness; the appropriate response would just appear in consciousness, as I suggested above that "forty-two" does when somebody asks for the product of six and seven. We will have to suppose a basic repertoire of possible computational moves to have been genetically installed in apparatus of which we are wholly unconscious.

It is admittedly far from clear just how the contents of the display are presented to consciousness. I mentioned earlier that the use of the sensorium for properly sensory presentation—presumably its earliest use—tends to overwhelm other more con ceptual options, to preempt the space of representation; its use for symbol or text manipulation seems to be secondary and derivative. The history of the familiar computer display, at any rate in its popularly accessible mode, offers an instructive parallel: originally everybody (apart from back-room boys with cathode ray tubes) thought of the display as normally providing visual representations, video images in fact, and only later did it become usual to use it as a monitor for computer operations. But in the case of the "display" in this model of mind it isn't necessarily the case that its contents are obviously sensory at all; they aren't necessarily images in the sense in which that term has been used, e.g., by Kosslyn.^[10] It seems plausible that only after the development of sensory awareness could conscious thought have emerged, but its intentional domain, though as it were derived from the sensorium, isn't properly speaking a sensorium (we might perhaps call the sensory-perceptual domain the primary sensorium, the intentional domain of thought the secondary sensorium, in order to preserve the sense of the suffix as a place where something is going on).

The reason why this display model is so appealing, given what we

now know about computers, is that in fact even the contents of the primary sensorium turn out to be computed rather than just given—perception is the outcome of a computational process, not the mere transmission of data but the construction of a world out of them. This "computational model" is still being argued, though it has been obvious enough for a long time to anyone familiar with the relevant physics and neurophysiology. It is a delicate problem to explicate, to be sure. Consider, for example, Schrödinger's remark at the beginning of the work already quoted: "The world is a construct of our sensations, perceptions, memories. It is convenient to regard it as existing objectively on its own. But it certainly does not become manifest by its mere existence. Its becoming manifest is conditional on very special goingson in very special parts of this very world, namely on certain events that happen in a brain."^[11] Here "world" needs to be disambiguated: the contents of my sensorium are my world, even though I take there to be a world that exists objectively on its own that I will call the world, whose local features I suppose help to determine my world. But the events in my brain are not in the "very world" whose perceptual apprehension they make possible.

The essential point, though—expressed succinctly by the English systems-theoretical eccentric Oliver Wells as: the brain computes the world—is that perception is a computational mechanism whose output to the display is the world we perceive, or in Wells's words, after a discussion of Gibson:

What we propose is that the visual system be considered as a computing device which computes from overlaps of . . . different scenes the stable, continuous, unbounded configuration of the room.—What we "see." Note that this can only be done when there is movement; the head has been turned, and optic information on the retinas has changed. Without this movement there could not be any computation. It is as in mathematics—the computation of an invariant under successive transformations.^[12]

It may be worth noting en passant that Cassirer had this idea of perception as invariance under transformation as early as 1938.^[13] In the generalized or secondary-sensorium case we may say similarly that thought is a computational mechanism whose output to the display is the world we apprehend, or grasp, or understand, in its structure and not merely in its appearances. Much of the input here, while carried by perception, will be purely textual or relational and thus transparent to its mode of embodiment—which is why the same thought content can be conveyed in different words, or different languages, or different symbolic modes.

If this model is plausible, if thought has available to it both display

and memory, both what Churchland^[14] has called "topographic maps" or "state spaces" (among which three-dimensional visual space is the most easily envisaged, although the spaces of higher dimensionality that he describes for other senses—which of course we won't apprehend as spaces in the same way—might well function similarly) and a storage mechanism that assigns addresses to items of structure as they are encoded (cross-connections between experiential items in different state spaces, for example, but also and mainly connections between such items and textual ones, or between textual items) then a great many puzzling facts about brain structure and memory might fall into place and some neurological pathologies be readily explained. The question about the consciousness of thought then transforms into the question of how much thinking goes on in the display and how much is hidden in circuits that draw on stored information, whether learned or innate.

VIII—
The Subject as Agent, The Machine as Subject

What we have to assume (if we are not to fall back into the epiphenomenalist position) is that there is an intentional agency capable, if not of summoning material from storage (although it probably does that too), at least of attending selectively to what happens to be in the display, whether it shows up there on the basis of sensory experience or emerges when sensory experience is to some degree suspended or shorted out (the immobile thinker again). This selective attention will evoke the appropriate connections and thus build intentional structures. Full subjectivity, I argue, requires a reflexive intentional structure that represents on the one hand the genidentity of the agent from his or her past to the present and from the present to at least a proximate future, and on the other the coherence of his or her sensory and textual embodiment (though there are methodological obstacles to the conclusiveness of any such argument). It also requires just such an agency of selection or evocation. Here I think we are going to need a whole new way of looking at action as "letting-happen," as well as a theory of the dynamics of action, with respect to which I find some interesting hints in what I take to be a new reading of Lacan's gloss on Freud's theory of the drive. However, that is another story; I will remark here only that when the computer is turned on it is on and stays on, whether or not one happens to be doing any computing, and in our case when it is off we're dead.

I argue also that the embodiment of subjectivity is transparent to its

structure, which means among other things that although so far there seem to be no cases of subjectivity otherwise embodied than in biologically developed "wetware," nothing we know as yet excludes the possibility that subjectivity might be embodied in hardware. The specific character of any system is presumably to be found in its structure, that is, the complex of relations that it embodies. In principle one might then suppose that the structure is indifferent to its physical embodiment. Thus, for example (at a more primitive level of complexity), the reaching and bearing functions of an artificial joint are in the limit behaviorally indistinguishable from those of its natural counterpart, so that the structure of the patient's behavior is unaffected when a natural element is replaced by a sufficiently sophisticated artificial one. This is what I mean when I say that the embodiment is transparent to the structure. We are thoroughly familiar with numbers of ancillary or corrective or prosthetic devices that are transparent in this way: automobiles and telephones have become just as transparent to the structure of purposive behavior as pockets or eyeglasses. The banality of the examples is significant. It reminds us that intentionality and purpose are everyday matters, not special states we have to work up to deliberately.

If a machine were to develop a reflexive intentional structure of the required kind (and I would want to specify some constraints: the structure is always double, the system of matchings is not, as in the case of language, initially arbitrary, and so on) there would be no reason to deny it subjectivity. However a number of difficult issues need to be surmounted before such a point can be reached. First, consciousness is a necessary precondition of subjectivity, so that the analogue of the interior display has to be provided for the machine. The monitor that we can see won't do, but nor will a monitor that the machine can see—what has to be provided is a way of its seeing, not the monitor, but what the monitor displays. So "interior display" here has to mean "display as seen from within." It is not clear that this condition can ever be known to be met, though we would have to admit that if it were (that is, if we built a machine like us in all relevant respects) there would be no reason to deny the machine the attribute of consciousness.

Second, the interior display has affective components in human beings which have been tuned by millennia of evolutionary selection to be neutral to perception and cognition within normal limits. If we were to make a machine complex enough to rise to consciousness we would have no guarantee that its first experience would not be one of intense pain; it ought therefore to be a matter of course to provide the machine with a mechanism for voluntary anesthesia if not suicide. Third (a point already made eloquently by Turing in 1950), we could not expect the machine to give evidence of subjectivity any earlier in its ontogenetic

development than human beings do—which I believe often to be never, but in any case hardly ever before the age of seven or eight, and that after intense socialization and acculturation.

And when all was said and done we would still be liable to Cartesian scepticism about the reality of the machine's subjective experience, even if it told us elaborate stories about that experience. But then we are liable to this scepticism about one another. And in the end we would have to grant it the same benefit of the doubt that we grant each other, and assume that at the origin of its first person utterances stood an intentionality and an agency. The answer to the question "Who is it?" is essentially: the intentional agent who says "I." But is there any reason to expect that we will understand the relation between this "I" and its embodiment any better in the case of the machine than in our own case?

Perhaps we should say: we will understand both, or neither. The problem is that our understanding anything involves our use of the mechanism of thought, our occupying the subjective standpoint. That is why I find something comical about doctrines like those of Lévi-Strauss and Foucault which claim to have dispensed with the subject (except, in Foucault's case, the subject in the sinister sense of being subjected to social and political forces), and why doctrines like eliminative materialism, for example in Churchland, strike me as perverse. I claim to be a materialist, but there isn't much about me that I want to eliminate, certainly not my feelings and appetites. In the case of eliminative materialism I want to ask: what is eliminated, and from where? If we can have a representation of thought without any elements of "folk psychology," well and good—except that thinking that representation, having it as the content of my intentional domain, brings in my subjectivity again. Churchland writes: "I gradually became comfortable in the idea that there really were quite general ways of representing cognitive activity that made no use of intentional idioms."^[15] One might ask—what does "comfortable" mean here? Doesn't it require elimination in its turn?—and then point out that a representation of cognitive activity which makes no use of intentional idioms makes use of intentionality just the same: that of the thinker, comfortable or otherwise, for whom it is a representation.

For what we are doing now, thinking about machines and about their thinking (and if thinking means computation, or just producing answers to any questions, computational or not, then machines have been doing it all along—that can't ever have been the problem), is all taking place in our individual sensoria, primary or secondary. The question is, could machines think in that way too? And the answer is, why not? Perhaps it will turn out that the structure of thought isn't transparent to its

embodiment, that there's something special and unreproducible (except by biological means) about wetware, but what evidence there could possibly be for such a view is far from clear. And there is something narcissistic in the thought that we are the only machines to have the experience of thinking as distinct from its behavioral manifestations. At all events I think there are likely to be, before too long perhaps, some machines that it would be morally prudent to treat as ends only, and never merely as means. Doing so at least is what it will take to make me feel comfortable. But that opens up a different argument.