Who Framed Augmented Reality?
The history of an age-old novelty
Art by J. R. Bee o
r
d
F
a
c
t
o
r
y
LAST MONTH?Facebook premiered its vision for the future at its development conference, F8. The camera-app technology Mark Zuckerberg calls augmented reality (or AR) borrows heavily from the social network Snapchat, which enables users to layer animated digital content onto photos on the fly. On stage, Zuckerberg promoted this collaging as social media’s first steps toward modish virtual screen manipulations. “This will allow us to create all kinds of things that were only available in the digital world,” Zuckerberg bubbled effusively. “We’re going to interact with them and explore them together.” Taken in, USA Today?repeated this claim to innovation, elaborating on the digital mogul’s Jules Verne-like promise: “We will wander not one, but two worlds—the physical and the digital,” For my part, I was particularly delighted by Facebook’s proposal to animate bowls of cereal with marauding cartoon sharks, savoring, perhaps, the insouciant violence I associate with childhood adventure.
Yet watching marine life encroach on a bowl of Frosted Flakes reminded me less of Tom Cruise’s information-management Mr. Miyagi in Minority Report than of the Michael Jordan–Looney Tunes face-off in Space Jam, during which live-action and animated characters join forces to whiz and bang their way through a cosmic game of interspecies b-ball. A rehabbed history of augmented reality suggests that our desire to physically mingle with art objects—to “interact with them and explore them together,” in Zuckerberg’s words—is more basic than its current status as social media curio implies. It’s embedded in the history of animation and its modernist, realist literary ancestors. Augmented reality is neither an innovative social media magician’s trick nor science fiction descending onto the present, but a basic artistic fantasy refracted through new technology.
Gertie Rex
Winsor McCay’s Gertie the Dinosauris generally considered the first animated cartoon ever, and it made use of the same trope of mixing reality and man-made art when it premiered all the way back in 1914. McCay was a cartoonist famous for the Freudian, surrealist comic Little Nemo in Slumberland, which was published in weekly instalments in the New York Heraldand New York American—though its material is more frequently compared to Bosch than to Garfield. McCay, already two hits deep into his career in the first decade of the twentieth?century, purportedly decided to animate a comic strip in 1909 on a dare from friends griping about his daunting productivity. Following a brief stint with an animated Nemo, McCay developed Gertie the Dinosaur, an amiable brontosaurus with a stoner grin, and took her on a vaudeville roadshow across America.
https://www.youtube.com/watch?v=TGXC8gXOPoU
The cartoon begins as a live-action sequence depicting McCay and two?other cartoonists dreaming Gertie up when a deflated tire leads the trio to kill time at the American Museum of Natural History. In a fictionalized version of the dare, McCay bets his dinner that he can “make the Dinosaurus live again by a series of hand-drawn cartoons.” After laboring for six months to produce thousands of dinosaur outlines, McCay unveils Gertie at a black tie dinner, proclaiming, “Gertie—yes, her name is Gertie—will come out of that cave and do everything I tell her to do.” Live-action McCay gesticulates at cartoon Gertie. “Come out Gertie, and make a pretty bow!” The camera zooms in, the formerly sessile cartoon takes up the entire screen, and a now-animate Gertie unfixes herself from the cave where she’d been in hiding, walks forward, and begins to gulp down rocks and trees. She waggles her head while following McCay’s commands, like, “bow” and “raise your foot.” The audience is informed that Gertie loves to dance. Real life McCay’s vaudeville act begins to play music and Gertie soft-shoes enthusiastically. Finally, real-life McCay disappears behind the screen and reappears in cartoon form in his black dinner duds. He climbs up Gertie’s spine and the pair ride off screen.
Live-action McCay gesticulates at cartoon Gertie.
Perhaps Gertie’s allure rests in the way McCay picked apart cultural threads and rebraided them into a new modernist mythos. Watching Gertie I’m reminded of Freud’s analysis of automata in E.T.A. Hoffmann’s The Sandman, in which a living doll represents a return to consciousness of repressed fears and fantasies. The doll, Olimpia, is eventually revealed as a mechanical simulation of a girl, who strikes fear into the heart of the narrator by confirming his fear of Olimpia’s omnipotent creator. Automata, in Freud’s parable and as artifacts, spike playfulness with the uncanny. The same can be said of robots in science fiction stories (and in alarmist articles about futurism), who stage rebellions to avenge themselves for having been shackled to human purpose. Yet here is Gertie, a large, two-dimensional version of an automaton with a ragtime soundtrack who gives off very little menace. Like many other children’s products using dinosaur imagery, Gertie’s comforts are proportionate to her source material’s awesome mandibles. This produces a palliative quality that seems baked into animation itself. In Gertie’s case, the danger of putting human creations on the same plane as us is tempered by the friendliness of the cartoon, and the end effect is of wonder, fear, adventure, and protection against the elements. I do not think the vitality of a cartoon, including its potential to do us mortal harm in the vein of Hoffmann’s Sandman or Asimov’s robots, is quite so salient without showing humans and cartoons interacting. The same themes of paleontological readiness and protection against the elements came through with Zuckerberg’s reference to “adventure” and “exploration,” when he promoted his iteration of augmented reality.
Though it was initially marketed as a novelty, Gertie the Dinosaur became hugely influential. Two of McCay’s most notable artistic descendants were Dave and Max Fleischer, brothers behind the Fleischer Bros animation studio that produced Betty Boop and Popeye. Wishing to economize the animation process by tracing over real photographs to create frame-by-frame animations, the siblings experimented with animation meeting live-action ?by inventing the rotoscope, whose first creature was the ne’er-do-well Koko the Clown. To animate Koko, Dave dressed in a black clown outfit while Max cranked a camera before the photos were printed and traced over. Separated during the war, Max and Dave reunited to form Out of the Inkwell Productions in 1921. Out of the Inkwell’s signature opener was a black and white Koko animation jumping out of his page to harass live-action Max Fleischer. The pair interact when live-action Max draws new details on the page or handles the cartoon as if it were real, such as when he squeezes a bubble Koko is trapped in to distort Koko’s waistline.
Zoom Out
Since then, the humans and animations have appeared together?in every decade, in franchises both obscure and megawatt, from the Theodore Roosevelt parody franchise “Colonel Heeza Liar” (1910s and 1920s) to Disney’s Fantasia (1940), Mary Poppins(1964), Bedknobs and Broomsticks(1971), Who Framed Roger Rabbit (1988), and Space Jam (1996), among many others. ?Animation historian Steven Kanfer traces Out of the Inkwell’s conceit to the Jewish golem and adds that it was a well established trope in both animation and literature: “The idea of a creature rising up against its maker was a part of a long tradition that embraced the ancient Jewish tale of the Golem, Frankenstein’s monster, The Sorcerer’s Apprentice, H.G. Wells’s The?Island of Dr. Moreau, and scores of folk stories. It was not even a new cartoon idea.” Zuckerberg’s augmented reality, which encourages interaction between what are basically rebranded digital animations and photographed people is merely this trend’s present, cooler form.
The human/drawing interaction trope that Zuckerberg is rebranding as Facebook’s own innovation even predates animated cartoons.
Perhaps it should not be surprising, then, that Facebook and Snapchat are delivering augmented reality through a camera instead of the glasses, contacts, or implants we were promised by science fiction. Not only does the smartphone camera allude (if probably accidentally) to AR’s animation roots, it also jibes with the content of the human/cartoon intercourse trope in which reality-on-cartoon action?is made possible by imagery reminiscent of probing camera lenses and digital and movie screens. In general, humans and cartoons traverse ontological boundaries by whipping through tunnels reminiscent of camera lenses or piercing a film or screen—sometimes both at once. In Mary Poppins, for example, Julie Andrews and Dick Van Dyke infiltrate the cartoon world by jumping into illustrated panels of chalk art; the flat, screen-like surface flummoxing expectation by giving way to three dimensions. And in Who Framed Roger Rabbit and Space Jam, private investigator Eddie Valiant and sad golfer Michael Jordan traverse the real life/cartoon boundaries by zipping through elongated tunnels and penetrating Looney Tunes–branded screens.
The human/drawing interaction trope that Zuckerberg is rebranding as Facebook’s own innovation even predates animated cartoons. One type of scrapbook, the paper dollhouse, played with the appeal of mixing real-life and an invented world. It was most popular from 1875-1920, and over forty years its form remained consistent: A dollhouse unfolded theatrically to create illusions of progress and depth. The first page of the album was always a porch or entrance; the next page, a front hall; subsequent pages were designated dining halls, living rooms, dressings, pantries, or backyards. They mixed representational and real materials, inked-in curtains alongside wallpaper scraps, silver leaf, crepe, net, lace, buttons, and dried flowers. Some books included peepholes that enabled readers to view sequential spaces simultaneously, much like Ub Iwerks’s multiplane camera, invented in 1933, which made use of layered planes of glass to provide depth of field to cartoons. Scrapbook historian Beverly Gordon explains, “If the books were collections of stage sets that formed a mini theater, then the scrapbook house maker was the director, set designer, costumer, and stage manager. If she later actually played with the book, she was also the author and producer of ongoing or ever-changing plays. Most books show enough wear to indicate that they were indeed pored over and played with repeatedly.” Many paper dollhouse books also included prosceniums, much like those featured in Who Framed Roger Rabbit, to cushion the collision of illustrated and real worlds.
The Scrap Heap of History
It is clear that human/cartoon interaction, whether planted in this century on our cell phones or in the last on movie screens, or even in Victorian paper books, is not a passing fancy. As a final note, I wonder how this fantasy relates to another late-nineteenth-century idea: modernism and its leisure-class desire for art to swallow life.
“One is tempted to extrapolate from this early interest in the relationship between word and image a later embrace of imagism, or what Ezra Pound would deride as ‘Amygism,’” writes one historian of modernism about modern poet Amy Lowell’s childhood scrapbook, which included both poems and comics. Other modern poets to keep scrapbooks include Edna St. Vincent Millay, H.D., Marianne Moore, e.e. cummings, and Hart Crane. (Before scrapbooking became feminized, it was considered an intelligent pastime by the upper and middle classes who wished to collect clippings from essays and news in their “extract books.”) An article in The Henry James Reviewhas also compared Henry James’s literature to a paper dollhouse, explaining, “James’s late diction and the paper doll house both share what we now think of as a modernist vision of the self as an arranged collection of images . . . They develop through a collage aesthetic that yokes composition and theatricality, arranging found elements into a stage-set display.”
Social network sites like Facebook have clear genealogy in the nineteenth?century’s friendship book, a type of scrapbook that aggregated inscriptions, illustrations, and signatures from friends and which were circulated among social groups. In broader strokes, scrapbooks also predicted social media for their value “not so much to serve the memory as to enact rituals of consumption and the hoarding of treasure”—a media-rich poor man’s cabinet of curiosities. Both scrapbooks and social media evolved with the growth of news media and publishing industries. Scrapbookers used periodicals and catalogues as sources of text and imagery, or even simply as paper stock for cutouts. As with social media, corporations began to pitch their advertisements for inclusion in scrapbooks by printing chromo-lithograph cards of their brand names along with more ornate illustrations to entice consumers to paste their ads into their scrapbooks in the equivalent of today’s native advertising.
“Be one?of the people on?whom nothing is lost,” advised James in his essay on literary realism, “The Art of Fiction.” It is this modernist impulse to collect and merge the frame of reality with that of art that fueled animation and now defines social media’s augmented reality. In other words, Zuck’s vision for the future is centuries old.