Fracking Eyeballs—Asterisk

Our eyes are worth money. We know that, now. It has become a commonplace that our “attention economy” is functionally an eyeball economy. But how did eyeballs come to look like dollar signs? Let’s dig into what we might think of as the original Faustian Bargain by which the sciences of human perception (with their sophisticated technologies of precision monitoring and measurement) cut a deal with those who move the money around. And I propose that we start here:

This puzzling totem face (with its adjacent mini-me) greeted pedestrians on 125th street in Harlem back in the summer of 1925. The curious who meandered over to the shop window for a closer look were, quite without their knowledge, lab rats in an elaborate experiment being conducted by one Howard K. Nixon, a recent Ph.D. in psychology at Columbia University and a pioneer in the new field of “attention science” — specifically as it could be applied to the business of advertising.

Indeed, depending on the day, it might be Howard K. Nixon himself peering out from behind the opaque cloth that constituted the nose, which was in fact a mask for what we might think of as the original “old school” surveillance capitalism. Here’s what was going on behind the scene: 

What we are looking at is the hidden booth where the experimenter sat watching the pedestrians. But not just watching. Also “baiting,” since Nixon had developed various techniques for luring pedestrians to take a closer look at his little “trap window” (bracket these for now; they were goofy). When someone approached the window, the operator threw a lever that dropped a pair of magazine ads into place as the “eyes” in the “face.” Then a recording protocol went into effect, with the observer keeping track, by means of switches in a modified teletype device, of which advertisement held the eyes, and for how long. 

There is much to observe about this experimental set-up, and the series of investigations Nixon and his collaborators undertook with it. For instance, his use of the term “bait” to describe his efforts to get passersby to approach his experiment testifies to the crossing lines of ad-world tactics and behaviorist animal testing. It is impossible not to be struck — at least, if one happens to be a historian of the behavioral sciences — by the similarity between his recording apparatus (a vibrating “time reed,” continuously dipping in and out of a dish of mercury) and the core “kit” developed in Leipzig by Wilhelm Wundt, the progenitor of laboratory psychology in the late nineteenth century. 

At the same time, it is uncanny and odd that Nixon would build the front-facing display to look like a face and place the ads he wished to see people see in the position of the eyes. Nixon manifestly grapples, in laying out his experimental ambitions, with the awkward proximity of his science to the actual low-brow business of trying to advertise new products to a generally indifferent populace. 

After all, mostly, the busy people walking up and down 125th street (about 18 per minute, on average) did not stop to look at his puzzling window display, and hence did not stop to be experimented on. Nixon was reduced to sticking that rather absurd manikin in the window (it can be discerned in the lower right of the first image, its face a scale reduction of that of the window display itself). “This mannikin held a placard which announced in very small type that this was ‘The Mystery Man,’ with some ambiguous remarks as to the purpose of the display,” Nixon explained in his publication on the experiments.

Upshot: being an advertising experimentalist required learning to advertise for your experimentalism. And that could be a rather demoralizing business. Nixon eventually even tried putting a picture of the Polish silent-screen bombshell Pola Negri in the window. Even so, most people still hustled past. And while the small number of folks who did stop and look did tend to get a bit more interested when the hidden experimenter threw the lever and dropped into view the two ads to be tested, this led Nixon to other worries: yes, testing people in a lab meant they knew they were being tested, but doing street-corner window displays with unusual changing placards probably left many of the onlookers thinking, as Nixon put it, “this is just another advertising stunt.”

Which, in some sense, it was. After all, Nixon had gotten his Ph.D. in experimental psychology, but he got his first job in the new Business School at Columbia. And in this way he was part of a new kind of synergy between the laboratory sciences of human sensory physiology and experimental psychology and the burgeoning academic specialization in making money.

He was a relatively small part of a large movement. All over the United States academic psychologists were finding their way to the practical investigation of human attention.

This marriage of convenience between American psychologists and American ad-men was a cozy connection warmed across the early decades of the twentieth century. It was out of this marriage that the concatenation of attention and value first emerged  As early as 1895, one of the mainstays of the Leipzig-trained progenitors of academic psychology in the United States, Edward Wheeler Scripture, who held a post at Yale, had written a book chapter on attention that suggested ad-men were ahead of laboratory psychologists in the practical understanding (and manipulation) of eyeballs. Scripture felt the psychologists needed to do more lab work in order to be able to give something back to those who had deep lay expertise in attention capture. 

It seems likely that Scripture’s thinking inspired the younger Yale- and Leipzig-trained psychologist Harlow Gale, who in the closing years of the nineteenth century (while based at the University of Minnesota), undertook what seems to be the earliest documented laboratory experiments on print ads by configuring an incandescent flash chamber in which subjects were exposed to a momentary glimpse of a magazine spread, and then asked to report on what they noticed.
He and his colleagues conducted more than three thousand iterations of this test, on more than a dozen subjects, under conditions set up (as Gale explained) “to obtain as nearly as possible in an experiment the conditions of rapidly turning the leaves of a magazine to see what would catch one’s attention.”
Already, then, by 1900, laboratory scrutiny of eyes and minds had turned to what Gale called the “attention value” of particular parts of a page, of particular kinds of font, and of various hues. 

One very early adopter of Gale’s scientific approach to advertising was the “attention wave” psychologist Walter Dill Scott, yet another Leipzig-trained experimentalist. Sidling up to the booming world of the Chicago admen circa 1900, Scott gathered a set of his columns on “The Psychology of Advertising,” which ran in the industry rag Mahin’s Magazine, and published them as The Theory and Practice of Advertising (1903). If Scott was steeped in the experimental psychological tradition that had worked so hard to time and test human attention, his book pivoted from the laboratory “slicing” and “dicing” attention to the marketplace problem of pricing this new and valuable social commodity.  

The longest chapter of The Theory and Practice of Advertising was titled, predictably, “Attention.” In it, Scott aligned the primary research problem of early twentieth-century psychology with the essential stuff of the commercial advertiser. Regardless of the various ways one might characterize the world of business, he explained, the “aim of every advertisement is to attract attention.”
And attention was, he proceeded to note, a topic on which psychology (“the newest of the experimental sciences”) had been hard at work for some time. What could the scientists of the mind offer to the practical money-men working up new shills for their clients? 

Scott put his finger on the basic problems: attention was a limited resource, because people could only pay attention to a relatively small number of things (Scott cited experiments that suggested ordinary people can attend to about four words, presented at a flash); and attention to any one thing came at the expense of attention to another.  Perhaps most importantly, attention, as the psychologists had parsed it, could be voluntary or involuntary. And this was key. Because the kind of attention that mattered for the admen was the involuntary kind. Here is how Scott put it:

What we discern in these sentences (and in the whole book of which they are a part) is the way that the rapidly institutionalizing science of human attention adroitly seconded itself to a vast and dynamic emerging commercial program: psychology, Scott promised, was the science that could assist the ad-men in the work of forcing things to our attention — because the psychologists understood the characteristics of involuntary attention in an intimate and technical way.

As to the specifics, one rather suspects that the admen nodded along, feeling that all this was rather commonsensical. But the words of Dr. Scott were, nevertheless, the blessing of science upon their from-the-hip practices. Moreover, Scott did actually lay out a number of ads and subject them to a “psychological” critique:  Why put a big picture of a slimy frog in a magazine pitch for White Star Coffee? Frogs don’t drink coffee. And coffee isn’t made from frogs either. No connection. Scott thought that such “irrelevant cuts” were a mistake. Sure, a frog on its hind legs, looking debonair, might (in its very oddness) attract the eye, but, in Scott’s view, such tripe made the ad itself harder to understand. (Interestingly, actual experiments, done a bit later, by another advertising psychologist, demonstrated that this was probably false — remember this point, since we’ll return to it.) 

In a 1912 review of The Theory and Practice of Advertising, the psychologist Edward K. Strong said that it had made an “enormous change” in the attitudes of businessmen towards the field of psychology and its practical value.
And he should know:  Strong had  completed his PhD at Columbia a year earlier, with a dissertation titled “The Relative Merit of Advertisements: A Psychological and Statistical Study” — a work that was built out of close experimental comparisons of different kinds of print advertisements, with a view to providing systematic advice to admen on optimizing their solicitations.  At Columbia, Strong found a number of senior psychologists who encouraged and facilitated his research into the science of commercialized attention. His direct advisor, James McKeen Cattell, had been one of the first American psychology researchers to be cited in Germany in connection with research on “Attention Waves.” And Cattell had also pioneered, just a few years before Strong came to campus, a set of statistical techniques for “order of merit” analyses that later became the basis of modern attitude scaling in the quantitative social sciences.
 

Strong used the same statistical approach to evaluate the pooled rankings of various tar soap ads (i.e., dandruff shampoos), which had been shown to experimental subjects. His aim? To segment consumers (by sex, in particular) and to understand the way different combinations of ad size, white space, image, and text could seize and hold attention. In so doing, he was pioneering in a new kind of psychology that was finding a very new kind of home — in the emergent “business schools” being founded in exactly these years (NYU’s in 1900, Harvard’s in 1908, Columbia’s in 1916, and Stanford’s in 1925). 

At this point, Howard K. Nixon re-enters our story. Nixon was another Columbia-trained psychologist, one of many influenced by Gale, Cattell, and Scott. But Nixon’s work in the 1920s demonstrates a new phase of academic interest in attention — a new experimental interest in the literal human eye. 

In Nixon’s earliest experiments, he would hide  in an observation booth, accompanied by a metronome beating half seconds, and then watch closely as two ads were revealed to a  seated subject. Ticking his pencil along the rungs of a fascinating little ladder-pad with each beat of the metronome (marking the left side when the eyes were to the left; marking the right side when the eyes were to the right), Nixon created a graphical record of the subject’s visual encounter with the two stimuli. Using this and other  forms of what we might think of as “analog eye tracking,” Nixon and his collaborators tallied hundreds of hours of subjects’ perusals of stacks and stacks of advertisements, configured to test a host of hypotheses about what grabbed attention, for whom, and for how long. They tested ads with borders and without. They tested color ads against black and white. They tested ads with pictures of people against ads with pictures of objects. And they tested these differentia against some basic segmentation of the pool of subjects:

In this context, it will be interesting to linger for a moment on one of Nixon’s actual findings, a result that puzzled him a bit, and to which I have already alluded above: his discovery that, pace Scott and others (like Gale) who believed that attention was bound up with relevance and conceptual coherence, involuntary attention seemed to work better when things made less sense.

Nixon ran a series of tests to investigate this outstanding issue, on which there was disagreement within the community of active ad-copy writers and graphic designers. If you were advertising a car, was it better to show a picture of a car, or of a pretty woman? If one was selling shirt collars, was it better to do so under the headline “Murder!” or the headline “Quality Shirt Collars”? To make a long story short (and to skip over the proto-pataphysical techniques by which Nixon established what we might call a “science of irrelevancy”), Nixon found that “irrelevant illustrations do attract more attention and hold interest longer than do relevant ones.”
Which is to say: systematically doing violence to sense raised revenue. This could be proven in his laboratory. Would it be wrong to say that we live, in many respects, in the world evolved from that finding? 

In the mid 1920s, Nixon and his generation of advertising psychologists could only dream of the precision and accuracy with which human attention would be measured in the decades to come. The technoscience of eyeball fracking was just getting underway. For a quick sense of how fast it would accelerate, consider this advertisement, which ran in the Des Moines Register on the 15th of September 1941:

It is an image worth lingering on. Though after looking at it for a few moments, you might legitimately begin to wonder about the path that your own eyes took in that perusal. Did you go first to the headline? “HOW A WOMAN LOOKS AT A MAN”? Or did your eye alight immediately on the man in the gray flannel suit? How long did you stay with him, and the mesh of reconstructed optical trajectories that web his dowdy form? And what about the little figure in the lower left? The woman who appears to peer down the double barrel of some peepshow console equipped with gatling-gun canisters?

This amazing ad represents a perfect little punctum in the world of midcentury attentional surveillance, a moment in which the scientific advertisers advertised scientific advertising as…an advertisement. Which is to say, what we have here is an ad for menswear that deploys the datasets developed by an actual eye tracker that had been developed in order to measure the attention value of advertisements! 

We tend to think of eye tracking as a distinctly internet-age phenomenon, but in the 1930s such systems were in use — not just in sophisticated laboratories (where the devices could be used to study the “saccades” in human reading, or the optical reflexes under various forms of cuing and stimulations), but on Madison Avenue too, where they offered the promise of unprecedented access to the visual life of consumers.

Indeed, the “Brandt Eye Camera” (depicted in the lower left-hand corner of figure 4, and in figure 5, below) offers an exemplary instance of the alliance between laboratory psychology and those who would commodify human attention. 

How did it work? Different readers may have different appetites for the inner workings of the device, so this paragraph and the next are for those who love the nitty gritty. They are skippable! (And you are invited to imagine the track of your eyes as you drop down to the stuff to follow.) 

Herman Francis Brandt, the psychologist who developed this device, borrowed cutting edge techniques from several contemporaries as he refined his portable and relatively robust system. It had been known for decades that it was possible to shine a bright beam of light obliquely at a human eye, and to translate the reflection of this light into spatial information about the direction of the gaze. If the front of the eye were perfectly spherical, this would not work. But it isn’t, and what Brandt called the “protuberance” of the front of the cornea meant that the physical surface of the eye effectively “pointed” in the direction of the sightline.

It was by no means easy to get accurate information about this vector from a tiny spot of reflected light, but it could be done. Instrument designers in this period could not, of course, make use of laser-light. Instead, the light of an arc lamp, condensed via a series of lenses to pencil-point fineness, was used to create a corneal reflection sufficiently bright to be recorded on slow-moving film-stock, mounted in a motion picture camera off to the side of the subject’s line of sight.

Brandt’s device made use of this established technique, but he was able to make a number of improvements on the exceedingly finicky and expensive systems of corneal reflection that had hitherto been reserved for highly specialized laboratory work. Systems like these depended on the head of the subject remaining perfectly stationary throughout the recording process. There was no way to distinguish between deflections of the corneal reflection caused by positional readjustment of the actual eyeball (moving your eyes), and those resulting from general motion of the head or body. Brandt incorporated into his system a clever technique for getting around this problem: a small reflecting bead was positioned  on the forehead of the subject, and a secondary beam of oblique light recorded the physical displacement of this bead, which thus served as an index for the physical movements of the head itself. Inscribing those movements made it possible to correct the eye-tracking data for extraneous physical movements. 

Here is Brandt’s system in use with a human subject, one of about 200 assayed in the course of research he published in 1940:

Figure 7 shows one result of this work: the circuit of optical scrutiny effected by a subject looking at this set of images. As Brandt put it, “Each of the 200 ocular patterns obtained in this study tell a story in terms of fixations and excursions, thus indicating location, duration, and sequence of fixations, as well as direction, distance and frequency of excursions.”

Over the course of his career Brandt would explore many potential applications of his technology: he examined how people looked at works of art; he tracked the eyes of experts against those of amateurs; he compiled information about how those adept at mathematics looked at problems as they tried to solve them; he explored the forensic value of eye tracking, endeavoring to see if ocular patterns could reveal when suspects were lying about what they did and did not know (a use that brought technical eye tracking into conversations about mind reading).

But it is essential to specify that none of these interesting matters was what drove the actual development of this system and directed Brandt’s substantive research program across the late 1930s and 1940s. What concerned Brandt was advertising and the commercial application of eye tracking to the determination of “attention value.” 

Indeed, all the images reproduced above were part of a series of experiments to ascertain the attention value of size in magazine advertising, a subject on which a series of experimental psychologists had come to disagree. Everyone understood that larger ads attracted more attention than smaller ones. But could the relationship be quantified? It needed to be, of course, from an advertiser’s perspective, because different ad sizes had different prices. At what price was a larger ad a good deal? 

And here, then, in Figure 8 is a schematic representation of that spatio-temporal valuation: a summation and collation of those hundreds of subjects’ visual perusal of the image array:

Using his powerful camera, Brandt made his way in (and beyond) academic psychology. In 1938, he was hired as a professor of psychology at Drake University, in Des Moines, just a hundred miles west of the Iowa State in Ames. He would hold a prominent position there for a decade, founding and directing the “Visual Research Laboratories” that centered on the Brandt eye camera. From this post, he rapidly produced a suite of articles in psychology journals that evaluated the attention value of a host of features of print advertisements, contributions gathered and substantially reprinted in his book, The Psychology of Seeing, published in 1945 — a book largely about advertising that was itself brightly advertised in The New York Times and elsewhere. 

This relatively popular volume helped cement Brandt’s reputation, so that by 1946 he could be identified in the Science Newsletter as the “inventor of the famous Brandt Eye Camera.” Knockoff versions of his device were sold from the back of Popular Mechanics in the early 1950s, by which time Brandt’s work had received notice in a wide range of mainstream publications, including Newsweek, Life, The Saturday Evening Post, Science, and Billboard — not to mention extensive coverage in the advertising trade journals. 

The story of Brandt’s camera and its applications within the world of advertising across the period from 1935 to 1955 reveals the close connections between laboratory psychology and the early life of our eyeball-fracking economy. Eye tracking became a way of conceptualizing and measuring the money value of human attention, and the application of new techniques of optical surveillance to consumers gave rise to familiar features of what we now identify as the “attention economy.” Brandt’s personal trajectory — from psychology teacher (and part-time traveling salesman) to prominent research psychologist and leader in the hybrid academic/corporate field of “marketing” — exemplifies a remarkable development in the scientific study of attention in general. But it also has implications for those who think about the changing relationship between universities and the private sector — and the perennial imbrications of scientific knowledge and capitalist endeavor.

And it will be fitting, in conclusion, to spell out those entanglements a little more clearly. If we return, for a moment, to Brandt’s original 1937 patent for his eye camera and scrutinize the headnote on the filing, we will discern that the intellectual property in question already participated in a wider (sublated) commercial network of relations and obligations (Figure 9):

It turns out that Herman Brandt was, in this filing, assigning a financial interest in his system to “Look, Inc.” of Des Moines. That company was indeed the recently-founded Cowles brothers company, which in February of that very year had launched the first issue of the large-format, highly visual Look magazine — a publication that would give the market-dominant Life a run for its money across the decades that followed, reaching nearly three million readers with biweekly issues in the 1940s. A little sleuthing reveals that Professor Brandt’s “Visual Research Laboratories,” notionally a Drake University institution, was effectively an internal corporate R&D facility at Look magazine.

Brandt left Drake in the late 1940s, and it appears he followed commercial leads out of the academy: in 1952 he incorporated the “Institute of Visual Research” in Chicago and shortly thereafter, in 1953, struck up a commercial relationship with the C. H. Stoelting Corporation in that city, a major manufacturer of testing systems, laboratory instruments, and precision industrial equipment. Two years later, in 1955, he passed away at the age of 60 in Oak Park, Illinois. But his company lived on: the Institute of Visual Research was taken over by a talented account executive from the Leo Burnett advertising agency, Edmund W. J. Faison, who himself held a PhD in psychology from George Washington University and had worked with Brandt for several years. 

Faison took the reins of a successful corporation, secured its intellectual property and business relationships, and rapidly expanded the enterprise. By 1961 he had incorporated a consortium of consumer behavior research companies in Switzerland and could boast of more than a thousand formal studies of consumer products (on “visibility, content communication, psychological connotations,” etc.) for a huge number of suppliers and retailers, sellers of “detergents, facial soaps, breakfast cereals, baby foods, paper napkins, facial and toilet tissue, candies, salad dressings, cold meats, frozen foods, canned foods, beers, scouring pads, starches, pet foods, and the like.” More than three thousand people worked with Faison’s research projects in this area. This was Brandt’s legacy.

At the start of the twentieth century, Walter Dill Scott concluded his many-times republished book The Theory of Advertising with a friendly gesture directed at his practical readers — the ad-men and business folks he hoped would pick up his volume. “The successful advertiser,” he explained, “must be a psychologist.” Half a century later, the careers of men like Faison and Brandt rather suggested the obverse: to be a truly successful psychologist, one had to be an advertiser. Or at least in advertising. Across those fifty years, the technical study of human attention in psychological laboratories came to be operationalized in the quest for the essential commodity of a consumer culture that took shape across the rise of mass media: attention.