Superhero shows are a weird subgenre of sci-fi. The super-powers and how the superheroes use them in pursuit of their world-saving goals are often the point, and so often skimp on the sci part of sci-fi. The Amazon original The Boys is no different, where the core novum is a chemical (compound V) that gives people superpowers.
I love the show. Though it’s definitely for adults with its violence and psychopathy and depravity, I think it’s closer to what would happen if humans had superhuman powers in a world of late-stage capitalism, enshittification of everything, and wannabe fascists. I’ve been a fan since it first aired. (And can’t wait to dive into the comics after the show wraps.)
It hasn’t really had many interfaces of note across the series. And the one I’m going to talk about in this post isn’t a “big” interface. But it was bad, so I’m coming out of my hiatus to talk about it, and then to make an appeal similar to what I did when I reviewed Idiocracy in 2019.
In the Season 4 finale—hastily renamed “Season 4 Finale” instead of “Assassination Run” after the alleged July 13 assassination attempt of Donald Trump—co-founders of The Boys, Grace Mallory and Butcher, invite the young supe Ryan to an underground bunker with three goals in mind.
Give him some time with Butcher who, as a kind of stepfather to Ryan, wants to see him before he dies. (Butcher is dying from a “sentient tumor” that developed from his overuse of “Temp V”.)
Convince Ryan to turn against his father, Homelander.
Entrap Ryan if he refuses.
It’s this last goal that involves the interface, because sure enough, Ryan is highly conflicted at the idea of killing his father after Butcher explains “You’re the only one who can stop him.”
“You’re the only one who can stop him.” —Butcher
As Ryan tries to leave to think things through, Grace blocks his way, saying “You can’t leave.” Ryan uses his super vision to observe that the walls of the room they’re in are 6 feet thick. Grace tries to explain, “This is the CIA Hazlet Safehouse, designed to hold people like you. I could seal us in here, flood the room with halothene, and we’d all take a nice, long nap.” As Ryan gets more agitated and threatens to leave anyway, she reaches out to a big, red momentary button mounted to the concrete wall beside her, presumably to release the aerosolized anesthesia.
And that’s it. That’s the interface. Because in a show that is very compellingly written, this is bad design.
It’s obvious
Being a big, red panic button, it might as well have a spotlight on it and a neon sign blinking “Press here to suppress.” Any supe worth their salt will recognize it as a threat and seek to disable it. I trust it would have a Normally Closed circuit, so that ripping the button out of the wall or severing the conduit would trip it, but a supe with Ryan or Homelander’s x-ray vision could just follow the circuit back to discover the nature of the halothane system and work from there. Much better is a system that wouldn’t call attention to itself.
It’s hard to get to
It’s hard to tell the complete room layout from the scene. It looks half hospital recovery room, half storage room, and I suspect is a converted supe prison cell (with windows, though?) The button appears to be just inside…the bathroom? Out of sight of the main part of the room, sure, so kind of hidden unless the supe needs to ever pee, but also harder to get to. A single button at around elbow-height works when a near-average-height person is upright and able to reach out to press it. But if you’ve just been knocked down, or had your arm laser-severed, or I don’t know, been body slammed across the room away from that button, you’re screwed. Even a ceiling-to-floor crash bar doesn’t work because it still requires your being within arms reach of that one spot. Better is a system that does not depend on where anyone is in the room for activation.
It works at human response speed
This is world with fast and mind-control supes. It doesn’t make sense to rely on human response times to activate it. Better is a semi-automated system that monitors everything and can respond in microseconds when data trends suspiciously.
Between its being obvious, hard to get to, and requiring manual activation I think nearly every single supe in the show would find it trivial to stop that button from being pressed if they wanted.
The scene could have been written more smartly—without sacrificing the efficiency of the beat—with something like this…
Grace
This is the CIA Hazlet Safehouse, designed to hold people like you. If you try to leave…
Cut to an arc shot of a supe-monitoring display. On the side, a live transcript of the conversation types out Grace’s words as she speaks them. In the center, infrared video of them in the room with overlays for each of them labeled SUPE or human, live vital signs, and a line showing their AI-predicted movements.
Grace (voiceover)
…or any of our vital signs crash…
Cut back to the actors
Grace
…the room is flooded with halothane and we all take a nice, long nap.
Zoom in to Ryan’s face as his eyes dart around and his breathing intensifies.
Cut to interface reading “escape prediction” and a number rising to 75, 80, 85. At 90 it turns red and a soft alarm goes off.
Cut to an extreme close up of Ryan’s ear to show he hears this alarm.
This isn’t obvious to the supe, works faster than a human could, and doesn’t rely on a human being in a specific spot.
Now instead of this, we could have Ryan brag about what a bad-ass he is and escape before the system can react, but this moment is constructed in the original to show that Ryan isn’t just an arrogant mini-Homelander. He’s a conflicted adolescent with an adolescent’s poor impulse control, and he panicked seeing her reach for the button. Having an alarm sets that same stage for him to panic. Note that I don’t think it’s good design for a system to tip its hand before it enacts control measures—as this does with the alarm—but it would be more forgivable than the dumb button, which just paints the CIA as incompetent and undermines the diegesis.
OK, that said, this next bit goes out to my fellow Americans:
One of the reasons I have wanted to talk about this show is not just the fascism of the villains, but how it illustrates the corrupting effect of power, and that’s directly related to the coming American election.
With Biden dropping out of the race yesterday, and the Democratic National Convention a month away, I can’t yet formally lean on the merits of the Democratic candidate to make a case for weeks to come. (Though, go go go, Kamala!) But the case against the Republican party almost makes itself.
What we are facing as a nation with this election is existential. The Supreme Court has outrageously ruled that a president is unaccountable for his actions while in office. A dictator’s wet dream. And Trump has declared publicly that he will be a dictator “on day one,” but it’s easy to see that he means “as of day one”. What malignant narcissist willingly gives up power once he has it? His many ties to the wretched Heritage Foundation and its deeply, deeply disturbing Project 2025 (see this video and this one where he directly praises this group and their plan) tell us that if he is elected and his cronies have their way, we fall towards an extremist religious-nationalism that puts The Boys to shame and spells the end of the ideals and institutions that were the reason the United States was invented in the first place. The American Experiment is on the brink.
But to quote the ACLU, despair and resignation are not a strategy. We have to America-up and enact a strategy. Please, please…
Expose the Extremism
Get familiar with the extremist plans (the Christianization and militarization of public school, cutting overtime protections for 4.3 million people, banning labor unions, privatizing Medicare, replacing a million experts with loyalist lackies, putting the DOJ under presidential control, close NOAA and end free weather reports, categorizing LGBTQ+ folks as pederasts and instating a death penalty for it, trying to pass a constitutional amendment to make abortion illegal, and much more) and share those often and loudly on your social media platforms of choice. Especially reach out to anyone on the fence, in a swing state (Arizona, Georgia, Michigan, Pennsylvania, and Wisconsin), or who thinks they should just sit this one out because the (current) candidates are so old or not doing enough of what they want. We cannot afford “protest votes.”
Volunteer
If you don’t have money to spare (and with the current income inequality plaguing the nation that’s likely to be most of us) you can donate time and effort. If you’re in a solidly-colored state, you can join texting and letter-writing campaigns to those in swing states. If you’re in a swing state (Arizona, Georgia, Michigan, Pennsylvania, and Wisconsin), you can help canvas directly to voters still deciding. (How they’re still undecided is utterly alien to me, but here we are.) Here are just a few places you can opt to volunteer.
If you do have money to spare, spare it. Give to progressive and Democratic causes that will use that buying power to get ads, get the word out, and support the vote. Dig deep because I know we’ve heard it before, but this one is critical.
Most importantly, have a plan to vote. Register if you’re not. If you are, double-check your voter registration status because they are purged just before elections, often bumping democrats for the most trivial of reasons. Vote by mail if you are overseas or if getting time off on the day of might be a problem. Find your polling location. Make a plan with others to go vote together. Charge your phone and bring water in case there are long lines. (And many bastards have worked very hard to ensure there will be long lines.) Get calendar reminders for voting deadlines sent directly to you.
If everyone gets out there and activates the vote, we can avoid giving the absolutely wrong people the power they should not have. You’re the only one who can stop him.
Like so much of the tech in Black Panther, this wearable battle gear is quite subtle, but critical to the scene, and much more than it seems at first. When Okoye and Nakia are chasing Klaue through the streets of Busan, South Korea, she realizes she would be better positioned on top of their car than within it.
She holds one of her spears out of the window, stabs it into the roof, and uses it to pull herself out on top of the swerving, speeding car. Once there, she places her feet into position, and the moment the sole of her foot touches the roof, it glows cyan for a moment.
She then holds onto the stuck spear to stabilize herself, rears back with her other spear, and throws it forward through the rear-window and windshield of some minions’ car, where it sticks in the road before them. Their car strikes the spear and get crushed. It’s a kickass moment in a film of kickass moments. But by all means let’s talk about the footwear.
Now, it’s not explicit, the effect the shoe has in the world of the story. But we can guess, given the context, that we are meant to believe the shoes grip the car roof, giving her a firm enough anchor to stay on top of the car and not tumble off when it swerves.
She can’t just be stuck
I have never thrown a javelin or a hyper-technological vibranium spear. But Mike Barber, PhD scholar in Biomechanics at Victoria University and Australian Institute of Sport, wrote this article about the mechanics of javelin throwing, and it seems that achieving throwing force is not just by sheer strength of the rotator cuff. Rather, the thrower builds force across their entire body and whips the momentum around their shoulder joint.
Ilgar Jafarov, CC BY-SA 4.0, via Wikimedia Commons
Okoye is a world-class warrior, but doesn’t have superpowers, so…while I understand she does not want the car to yank itself from underneath her with a swerve, it seems that being anchored in place, like some Wakandan air tube dancer, will not help her with her mighty spear throwing. She needs to move.
It can’t just be manual
Imagine being on a mechanical bull jerking side to side—being stuck might help you stay upright. But imagine it jerking forward suddenly, and you’d wind up on your butt. If it jerked backwards, you’d be thrown forward, and it might be much worse. All are possibilities in the car chase scenario.
If those jerking motions happened to Okoye faster than she could react and release her shoes, it could be disastrous. So it can’t be a thing she needs to manually control. Which means it needs to some blend of manual, agentive, and assistant. Autonomic, maybe, to borrow the term from physiology?
So…
To really be of help, it has to…
monitor the car’s motion
monitor her center of balance
monitor her intentions
predict the future motions of the cars
handle all the cybernetics math (in the Norbert Wiener sense, not the sci-fi sense)
know when it should just hold her feet in place, and when it should signal for her to take action
know what action she should ideally take, so it knows what to nudge her to do
These are no mean feats, especially in real-time. So, I don’t see any explanation except…
An A.I. did it.
AGI is in the Wakandan arsenal (c.f. Griot helping Ross), so this is credible given the diegesis, but I did not expect to find it in shoes.
An interesting design question is how it might deliver warning signals about predicted motions. Is it tangible, like vibration? Or a mild electrical buzz? Or a writing-to-the-brain urge to move? The movie gives us no clues, but if you’re up for a design challenge, give it a speculative design pass.
Wearable heuristics
As part of my 2014 series about wearable technologies in sci-fi, I identified a set of heuristics we can use to evaluate such things. A quick check against those show that they fare well. The shoes are quite sartorial, and look like shoes so are social as well. As a brain interface, it is supremely easy to access and use. Two of the heuristics raise questions though.
Wearables must be designed so they are difficult to accidentally activate. It would have been very inconvenient for Okoye to find herself stuck to the surface of Wakanda while trying to chase Killmonger later in the film, for example. It would be safer to ensure deliberateness with some mode-confirming physical gesture, but there’s no evidence of it in the movie.
Wearables should have apposite I/O. The soles glow. Okoye doesn’t need that information. I’d say in a combat situation it’s genuinely bad design to require her to look down to confirm any modes of the shoes. They’re worn. She will immediately feel whether her shoes are fixed in place. While I can’t name exactly how an enemy might use the knowledge about whether she is stuck in place or not, but on general principle, the less information we give to the enemy, the safer you’ll be. So if this was real-world, we would seek to eliminate the glow. That said, we know that undetectable interactions are not cinegenic in the slightest, so for the film this is a nice “throwaway” addition to the cache of amazing Wakandan technology.
Black Georgia Matters and Today is the Day
Each post in the Black Panther review is followed by actions that you can take to support black lives.
Today is the last day in the Georgia runoff elections. It’s hard to overstate how important this is. If Ossoff and Warnock win, the future of the country has a much better likelihood of taking Black Lives Matter (and lots of other issues) more seriously. Actual progress might be made. Without it, the obstructionist and increasingly-frankly-racist Republican party (and Moscow Mitch) will hold much of the Biden-Harris administration back. If you know of any Georgians, please check with them today to see if they voted in the runoff election. If not—and they’re going to vote Democrat—see what encouragement and help you can give them.
If their absentee ballot has not been registered, they can go to the polls and tell the workers there that they want to cancel their absentee ballot and vote in person. Help them know their poll at My Voter Page: https://www.mvp.sos.ga.gov/MVP/mvp.do
Back to Blade Runner. I mean, the pandemic is still pandemicking, but maybe this will be a nice distraction while you shelter in place. Because you’re smart, sheltering in place as much as you can, and not injecting disinfectants. And, like so many other technologies in this film, this will take a while to deconstruct, critique, and reimagine.
Description
Doing his detective work, Deckard retrieves a set of snapshots from Leon’s hotel room, and he brings them home. Something in the one pictured above catches his eye, and he wants to investigate it in greater detail. He takes the photograph and inserts it in a black device he keeps in his living room.
Note: I’ll try and describe this interaction in text, but it is much easier to conceptualize after viewing it. Owing to copyright restrictions, I cannot upload this length of video with the original audio, so I have added pre-rendered closed captions to it, below. All dialogue in the clip is Deckard.
Deckard does digital forensics, looking for a lead.
He inserts the snapshot into a horizontal slit and turns the machine on. A thin, horizontal orange line glows on the left side of the front panel. A series of seemingly random-length orange lines begin to chase one another in a single-row space that stretches across the remainder of the panel and continue to do so throughout Deckard’s use of it. (Imagine a news ticker, running backwards, where the “headlines” are glowing amber lines.) This seems useless and an absolutely pointless distraction for Deckard, putting high-contrast motion in his peripheral vision, which fights for attention with the actual, interesting content down below.
If this is distracting you from reading, YOU SEE MY POINT.
After a second, the screen reveals a blue grid, behind which the scan of the snapshot appears. He stares at the image in the grid for a moment, and speaks a set of instructions, “Enhance 224 to 176.”
In response, three data points appear overlaying the image at the bottom of the screen. Each has a two-letter label and a four-digit number, e.g. “ZM 0000 NS 0000 EW 0000.” The NS and EW—presumably North-South and East-West coordinates, respectively—immediately update to read, “ZM 0000 NS 0197 EW 0334.” After updating the numbers, the screen displays a crosshairs, which target a single rectangle in the grid.
A new rectangle then zooms in from the edges to match the targeted rectangle, as the ZM number—presumably zoom, or magnification—increases. When the animated rectangle reaches the targeted rectangle, its outline blinks yellow a few times. Then the contents of the rectangle are enlarged to fill the screen, in a series of steps which are punctuated with sounds similar to a mechanical camera aperture. The enlargement is perfectly resolved. The overlay disappears until the next set of spoken commands. The system response between Deckard’s issuing the command and the device’s showing the final enlarged image is about 11 seconds.
Deckard studies the new image for awhile before issuing another command. This time he says, “Enhance.” The image enlarges in similar clacking steps until he tells it, “Stop.”
Other instructions he is heard to give include “move in, pull out, track right, center in, pull back, center, and pan right.” Some include discrete instructions, such as, “Track 45 right” while others are relative commands that the system obeys until told to stop, such as “Go right.”
Using such commands he isolates part of the image that reveals an important clue, and he speaks the instruction, “Give me a hard copy right there.” The machine prints the image, which Deckard uses to help find the replicant pictured.
This image helps lead him to Zhora.
I’d like to point out one bit of sophistication before the critique. Deckard can issue a command with or without a parameter, and the inspector knows what to do. For example, “Track 45 right” and “Track right.” Without the parameter, it will just do the thing repeatedly until told to stop. That helps Deckard issue the same basic command when he knows exactly where he wants to look and when doesn’t know what exactly what he’s looking for. That’s a nice feature of the language design.
But still, asking him to provide step-by-step instructions in this clunky way feels like some high-tech Big Trak. (I tried to find a reference that was as old as the film.) And that’s not all…
Some critiques, as it is
Can I go back and mention that amber distracto-light? Because it’s distracting. And pointless. I’m not mad. I’m just disappointed.
It sure would be nice if any of the numbers on screen made sense, and had any bearing with the numbers Deckard speaks, at any time during the interaction. For instance, the initial zoom (I checked in Photoshop) is around 304%, which is neither the 224 or 176 that Deckard speaks.
It might be that each square has a number, and he simply has to name the two squares at the extents of the zoom he wants, letting the machine find the extents, but where is the labeling? Did he have to memorize an address for each pixel? How does that work at arbitrary levels of zoom?
And if he’s memorized it, why show the overlay at all?
Why the seizure-inducing flashing in the transition sequences? Sure, I get that lots of technologies have unfortunate effects when constrained by mechanics, but this is digital.
Why is the printed picture so unlike the still image where he asks for a hard copy?
Gaze at the reflection in Ford’s hazel, hazel eyes, and it’s clear he’s playing Missile Command, rather than paying attention to this interface at all. (OK, that’s the filmmaker’s issue, not a part of the interface, but still, come on.)
The photo inspector: My interface is up HERE, Rick.
How might it be improved for 1982?
So if 1982 Ridley Scott was telling me in post that we couldn’t reshoot Harrison Ford, and we had to make it just work with what we had, here’s what I’d do…
Squash the grid so the cells match the 4:3 ratio of the NTSC screen. Overlay the address of each cell, while highlighting column and row identifiers at the edges. Have the first cell’s outline illuminate as he speaks it, and have the outline expand to encompass the second named cell. Then zoom, removing the cell labels during the transition. When at anything other than full view, display a map across four cells that shows the zoom visually in the context of the whole.
Rendered in glorious 4:3 NTSC dimensions.
With this interface, the structure of the existing conversation makes more sense. When Deckard said, “Enhance 203 to 608” the thing would zoom in on the mirror, and the small map would confirm.
The numbers wouldn’t match up, but it’s pretty obvious from the final cut that Scott didn’t care about that (or, more charitably, ran out of time). Anyway I would be doing this under protest, because I would argue this interaction needs to be fixed in the script.
How might it be improved for 2020?
What’s really nifty about this technology is that it’s not just a photograph. Look close in the scene, and Deckard isn’t just doing CSI Enhance! commands (or, to be less mocking, AI upscaling). He’s using the photo inspector to look around corners and at objects that are reconstructed from the smallest reflections. So we can think of the interaction like he’s controlling a drone through a 3D still life, looking for a lead to help him further the case.
With that in mind, let’s talk about the display.
Display
To redesign it, we have to decide at a foundational level how we think this works, because it will color what the display looks like. Is this all data that’s captured from some crazy 3D camera and available in the image? Or is it being inferred from details in the 2 dimensional image? Let’s call the first the 3D capture, and the second the 3D inference.
If we decide this is a 3-D capture, then all the data that he observes through the machine has the same degree of confidence. If, however, we decide this is a 3D inferrer, Deckard needs to treat the inferred data with more skepticism than the data the camera directly captured. The 3-D inferrer is the harder problem, and raises some issues that we must deal with in modern AI, so let’s just say that’s the way this speculative technology works.
The first thing the display should do it make it clear what is observed and what is inferred. How you do this is partly a matter of visual design and style, but partly a matter of diegetic logic. The first pass would be to render everything in the camera frustum photo-realistically, and then render everything outside of that in a way that signals its confidence level. The comp below illustrates one way this might be done.
Modification of a pair of images found on Evermotion
In the comp, Deckard has turned the “drone” from the “actual photo,” seen off to the right, toward the inferred space on the left. The monochrome color treatment provides that first high-confidence signal.
In the scene, the primary inference would come from reading the reflections in the disco ball overhead lamp, maybe augmented with plans for the apartment that could be found online, or maybe purchase receipts for appliances, etc. Everything it can reconstruct from the reflection and high-confidence sources has solid black lines, a second-level signal.
The smaller knickknacks that are out of the reflection of the disco ball, and implied from other, less reflective surfaces, are rendered without the black lines and blurred. This provides a signal that the algorithm has a very low confidence in its inference.
This is just one (not very visually interesting) way to handle it, but should illustrate that, to be believable, the photo inspector shouldn’t have a single rendering style outside the frustum. It would need something akin to these levels to help Deckard instantly recognize how much he should trust what he’s seeing.
Flat screen or volumetric projection?
Modern CGI loves big volumetric projections. (e.g. it was the central novum of last year’s Fritz winner, Spider-Man: Far From Home.) And it would be a wonderful juxtaposition to see Deckard in a holodeck-like recreation of Leon’s apartment, with all the visual treatments described above.
But…
Also seriously who wants a lamp embedded in a headrest?
…that would kind of spoil the mood of the scene. This isn’t just about Deckard’s finding a clue, we also see a little about who he is and what his life is like. We see the smoky apartment. We see the drab couch. We see the stack of old detective machines. We see the neon lights and annoying advertising lights swinging back and forth across his windows. Immersing him in a big volumetric projection would lose all this atmospheric stuff, and so I’d recommend keeping it either a small contained VP, like we saw in Minority Report, or just keep it a small flat screen.
OK, so we have an idea about how the display would (and shouldn’t) look, let’s move on to talk about the inputs.
Inputs
To talk about inputs, then, we have to return to a favorite topic of mine, and that is the level of agency we want for the interaction. In short, we need to decide how much work the machine is doing. Is the machine just a manual tool that Deckard has to manipulate to get it to do anything? Or does it actively assist him? Or, lastly, can it even do the job while his attention is on something else—that is, can it act as an agent on his behalf? Sophisticated tools can be a blend of these modes, but for now, let’s look at them individually.
Manual Tool
This is how the photo inspector works in Blade Runner. It can do things, but Deckard has to tell it exactly what to do. But we can still improve it in this mode.
We could give him well-mapped physical controls, like a remote control for this conceptual drone. Flight controls wind up being a recurring topic on this blog (and even came up already in the Blade Runner reviews with the Spinners) so I could go on about how best to do that, but I think that a handheld controller would ruin the feel of this scene, like Deckard was sitting down to play a video game rather than do off-hours detective work.
Special edition made possible by our sponsor, Tom Nook. (I hope we can pay this loan back.)
Similarly, we could talk about a gestural interface, using some of the synecdochic techniques we’ve seen before in Ghost in the Shell. But again, this would spoil the feel of the scene, having him look more like John Anderton in front of a tiny-TV version of Minority Report’s famous crime scrubber.
One of the things that gives this scene its emotional texture is that Deckard is drinking a glass of whiskey while doing his detective homework. It shows how low he feels. Throwing one back is clearly part of his evening routine, so much a habit that he does it despite being preoccupied about Leon’s case. How can we keep him on the couch, with his hand on the lead crystal whiskey glass, and still investigating the photo? Can he use it to investigate the photo?
Here I recommend a bit of ad-hoc tangible user interface. I first backworlded this for The Star Wars Holiday Special, but I think it could work here, too. Imagine that the photo inspector has a high-resolution camera on it, and the interface allows Deckard to declare any object that he wants as a control object. After the declaration, the camera tracks the object against a surface, using the changes to that object to control the virtual camera.
In the scene, Deckard can declare the whiskey glass as his control object, and the arm of his couch as the control surface. Of course the virtual space he’s in is bigger than the couch arm, but it could work like a mouse and a mousepad. He can just pick it up and set it back down again to extend motion.
This scheme takes into account all movement except vertical lift and drop. This could be a gesture or a spoken command (see below).
Going with this interaction model means Deckard can use the whiskey glass, allowing the scene to keep its texture and feel. He can still drink and get his detective on.
Tipping the virtual drone to the right.
Assistant Tool
Indirect manipulation is helpful for when Deckard doesn’t know what he’s looking for. He can look around, and get close to things to inspect them. But when he knows what he’s looking for, he shouldn’t have to go find it. He should be able to just ask for it, and have the photo inspector show it to him. This requires that we presume some AI. And even though Blade Runner clearly includes General AI, let’s presume that that kind of AI has to be housed in a human-like replicant, and can’t be squeezed into this device. Instead, let’s just extend the capabilities of Narrow AI.
Some of this will be navigational and specific, “Zoom to that mirror in the background,” for instance, or, “Reset the orientation.” Some will more abstract and content-specific, e.g. “Head to the kitchen” or “Get close to that red thing.” If it had gaze detection, he could even indicate a location by looking at it. “Get close to that red thing there,” for example, while looking at the red thing. Given the 3D inferrer nature of this speculative device, he might also want to trace the provenance of an inference, as in, “How do we know this chair is here?” This implies natural language generation as well as understanding.
There’s nothing from stopping him using the same general commands heard in the movie, but I doubt anyone would want to use those when they have commands like this and the object-on-hand controller available.
Ideally Deckard would have some general search capabilities as well, to ask questions and test ideas. “Where were these things purchased?” or subsequently, “Is there video footage from the stores where he purchased them?” or even, “What does that look like to you?” (The correct answer would be, “Well that looks like the mirror from the Arnolfini portrait, Ridley…I mean…Rick*”) It can do pattern recognition and provide as much extra information as it has access to, just like Google Lens or IBM Watson image recognition does.
*Left: The convex mirror in Leon’s 21st century apartment. Right: The convex mirror in Arnolfini’s 15th century apartment
Finally, he should be able to ask after simple facts to see if the inspector knows or can find it. For example, “How many people are in the scene?”
All of this still requires that Deckard initiate the action, and we can augment it further with a little agentive thinking.
Agentive Tool
To think in terms of agents is to ask, “What can the system do for the user, but not requiring the user’s attention?” (I wrote a book about it if you want to know more.) Here, the AI should be working alongside Deckard. Not just building the inferences and cataloguing observations, but doing anomaly detection on the whole scene as it goes. Some of it is going to be pointless, like “Be aware the butter knife is from IKEA, while the rest of the flatware is Christofle Lagerfeld. Something’s not right, here.” But some of it Deckard will find useful. It would probably be up to Deckard to review summaries and decide which were worth further investigation.
It should also be able to help him with his goals. For example, the police had Zhora’s picture on file. (And her portrait even rotates in the dossier we see at the beginning, so it knows what she looks like in 3D for very sophisticated pattern matching.) The moment the agent—while it was reverse ray tracing the scene and reconstructing the inferred space—detects any faces, it should run the face through a most wanted list, and specifically Deckard’s case files. It shouldn’t wait for him to find it. That again poses some challenges to the script. How do we keep Deckard the hero when the tech can and should have found Zhora seconds after being shown the image? It’s a new challenge for writers, but it’s becoming increasingly important for believability.
Though I’ve never figured out why she has a snake tattoo here (and it seems really important to the plot) but then when Deckard finally meets her, it has disappeared.
Scene
Interior. Deckard’s apartment. Night.
Deckard grabs a bottle of whiskey, a glass, and the photo from Leon’s apartment. He sits on his couch and places the photo on the coffee table.
Deckard
Photo inspector.
The machine on top of a cluttered end table comes to life.
Deckard
Let’s look at this.
He points to the photo. A thin line of light sweeps across the image. The scanned image appears on the screen, pulled in a bit from the edges. A label reads, “Extending scene,” and we see wireframe representations of the apartment outside the frame begin to take shape. A small list of anomalies begins to appear to the left. Deckard pours a few fingers of whiskey into the glass. He takes a drink before putting the glass on the arm of his couch. Small projected graphics appear on the arm facing the inspector.
Deckard
OK. Anyone hiding? Moving?
Photo inspector
No and no.
Deckard
Zoom to that arm and pin to the face.
He turns the glass on the couch arm counterclockwise, and the “drone” revolves around to show Leon’s face, with the shadowy parts rendered in blue.
Deckard
What’s the confidence?
Photo inspector
95.
On the side of the screen the inspector overlays Leon’s police profile.
Deckard
Unpin.
Deckard lifts his glass to take a drink. He moves from the couch to the floor to stare more intently and places his drink on the coffee table.
Deckard
New surface.
He turns the glass clockwise. The camera turns and he sees into a bedroom.
Deckard
How do we have this much inference?
Photo inspector
The convex mirror in the hall…
Deckard
Wait. Is that a foot? You said no one was hiding.
Photo inspector
The individual is not hiding. They appear to be sleeping.
Deckard rolls his eyes.
Deckard
Zoom to the face and pin.
The view zooms to the face, but the camera is level with her chin, making it hard to make out the face. Deckard tips the glass forward and the camera rises up to focus on a blue, wireframed face.
Deckard
That look like Zhora to you?
The inspector overlays her police file.
Photo inspector
63% of it does.
Deckard
Why didn’t you say so?
Photo inspector
My threshold is set to 66%.
Deckard
Give me a hard copy right there.
He raises his glass and finishes his drink.
This scene keeps the texture and tone of the original, and camps on the limitations of Narrow AI to let Deckard be the hero. And doesn’t have him programming a virtual Big Trak.
So while the world is in the grip of the novel COVID-19 coronavirus pandemic, I’ve been thinking about those fictional user interfaces that appear in pandemic movies that project how quickly the infectious-agent-in-question will spread. The COVID-19 pandemic is a very serious situation. Most smart people are sheltering in place to prevent an overwhelmed health care system and finding themselves with some newly idle cycles (or if you’re a parent like me, a lot fewer idle cycles). Looking at this topic through the lens of sci-fi is not to minimize what’s happening around us as trivial, but to process the craziness of it all through this channel that I’ve got on hand. I did it for fascism, I’ll do it for this. Maybe this can inform some smart speculative design.
Caveat #1:As a public service I have included some information about COVID-19 in the body of the post with a link to sources. These are called out the way this paragraph is, with a SARS-CoV-2 illustration floated on the left. I have done as much due diligence as one blogger can do to not spread disinformation, but keep in mind that our understanding of this disease and the context are changing rapidly. By the time you read this, facts may have changed. Follow links to sources to get the latest information. Do not rely solely on this post as a source. If you are reading this from the relative comfort of the future after COVID-19, feel free to skip these.
Get on a boat, Hongkongers, you can’t even run for the hills! Contagion (2011)
And yes, this is less of my normal fare of sci-fi and more bio-fi, but it’s still clearly a fictional user interface, so between that and the world going pear-shaped, it fits well enough. I’ll get back to Blade Runner soon enough. I hope.
Giving credit where it’s due: All but one of the examples in this post were found via the TV tropes page for Spreading Disaster Map Graphic page, under live-action film examples. I’m sure I’ve missed some. If you know of others, please mention it in the comments.
Four that are extradiegetic and illustrative
This first set of pandemic maps are extradiegetic.
Vocabulary sidebar: I use that term a lot on this blog, but if you’re new here or new to literary criticism, it bears explanation. Diegesis is used to mean “the world of the story,” as the world in which the story takes place is often distinct from our own. We distinguish things as diegetic and extradiegetic to describe when they occur within the world of the story, or outside of it, respectively. My favorite example is when we see a character in a movie walking down a hallway looking for a killer, and we hear screechy violins that raise the tension. When we hear those violins, we don’t imagine that there is someone in the house who happens to be practicing their creepy violin. We understand that this is extradiegetic music, something put there to give us a clue about how the scene is meant to feel.
So, like those violins, these first examples aren’t something that someone in the story is looking at. (Claude Paré? Who the eff is—Johnson! Get engineering! Why are random names popping up over my pandemic map?) They’re something the film is doing for us in the audience.
The Killer that Stalked New York (1950) is a short about a smallpox infection of New York City.
Edge of Tomorrow (2014) has this bit showing the Mimics, spreading their way across Europe.
The end of Rise of the Planet of the Apes (2011) shows the fictional virus ALZ-113 spreading.
The beginning of Dawn of the Planet of the Apes (2014) repeats the fictional virus ALZ-113 spreading, but augments it with video overlays.
There’s not much I feel the need to say about these kinds of maps, as they are a motion graphic and animation style. I note at least two use aposematic signals in their color palette and shapes, but that’s just because it helps reinforce for the audience that whatever is being shown here is a major threat to human life. But I have much more authoritative things to say about systems that are meant to be used.
Before we move on, here’s a bonus set of extradiegetic spreading-pathogen maps I saw while watching the Netflix docuseries Pandemic: How to Prevent an Outbreak, as background info for this post.
The five examples in this section are spread throughout the text for visual interest, but presented in chronological order. They are The Andromeda Strain (1977), Outbreak (1995), Evolution (2001), Contagion (2011), and World War Z (2013). I highly recommend Contagion for the acting, movie making, the modeling, and some of the facts it conveys. For instance, I think it’s the only film that discusses fomites. Everyone should know about fomites.
Since I raise their specter: As of publication of this post the CDC stated that fomites are not thought to be the main way the COVID-19 novel coronavirus spreads, but there are recent and conflicting studies. The scientific community is still trying to figure this out. The CDC says for certain it spreads primarily through sneezes, coughs, and being in close proximity to an infected person, whether or not they are showing symptoms.
Note that these five spreading pathogen examples are things that characters are seeing in the diegesis, that is, in the context of the story. These interfaces are meant to convey useful information to the characters as well as us in the audience.
Which is as damning a setup as I can imagine for this first example from The Andromeda Strain (1971). Because as much as I like this movie, WTF is this supposed to be? “601” is explained in the dialogue as the “overflow error” of this computer, but the pop-art seizure graphics? C’mon. There’s no way to apologize for this monstrosity.
This psychedelic nonsense somehow tells the bunkered scientists about how fast the eponymous Andromeda Strain will spread. (1971) Somehow the CRT gets nervous, too.
I’m sorry that you’ll never get those 24 seconds back. But at least we can now move on to look at the others, which we can break down into the simple case of persuasion, and the more complex case of use.
The simple case
In the simplest case, these graphics are shown to persuade an authority to act. That’s what happening in this clip from Outbreak (1995).
General Donald McClintock delivers a terrifying White House Chief-of-Staff Briefing about the Motaba virus. Outbreak (1995)
But if the goal is to persuade one course of action over another, some comparison should be made between two options, like, say, what happens if action is taken sooner rather than later. While that is handled in the dialogue of many of these films—and it may be more effective for in-person persuasion—I can’t help but think it would be reinforcing to have it as part of the image itself. Yet none of our examples do this.
Compare the “flatten the curve” graphics that have been going around. They provide a visual comparison between two options and make it very plain which is the right one to pick. One that stays in the mind of the observer even after they see it. This is one I’ve synthesized and tweaked from other sources.
This is a conceptual diagram, not a chart. The capacity bar is terrifyingly lower on actual charts. Stay home as much as you can. Special shouts out to Larry West.
There is a diegetic possibility, i.e., that no one amidst the panic of the epidemic has the time to thoughtfully do more than spit out the data and handle the rest with conversation. But we shouldn’t leave it at that, because there’s not much for us to learn there.
More complex case
The harder problem is when these displays are for people who need to understand the nature of the threat and determine the best course of action, and now we need to talk about epidemiology.
Caveat #2:I am not an epidemiologist. They are all really occupied for the foreseeable future, so I’m not even going to reach out and bother one of them to ask their opinions on this post. Like I said before about COVID-19, I really hope you don’t come to sci-fi interfaces to become an expert in epidemiology. And, since I’m just Some Guy on the Internet Who Has Read Some Stuff on the Internet, you should take whatever you learn here with a grain of salt. If I get something wrong, please let me know. Here are my major sources:
Kate Winslet, playing epidemiologist Dr. Erin Mears in Contagion (2011), is probably more qualified than me. Hey, Kate: Call me. I have questions.
Caveat #3: To discuss using technology in our species’ pursuit of an effective global immune system is to tread into some uncomfortable territory. Because of the way disease works, it is not enough to surveil the infected. We must always surveil the entire population, healthy or not, for signs of a pathogen outbreak, so responses can be as swift and certain as possible. We may need to surveil certain at-risk or risk-taking populations quite closely, as potential superspreaders. Otherwise we risk getting…well…*gestures vaguely at the USA*. I am pro-privacy, so know that when I speak about health surveillance in this post, I presume that we are simultaneously trying to protect as much “other” privacy as we can, maybe by tracking less-abusable, less-personally identifiable signals. I don’t pretend this is a trivial task, and I suspect the problem is more wicked than merely difficult to execute. But health surveillance must happen, and for this reason I will speak of it as a good thing in this context.
Epidemiology is a large field of study, so for purposes of this post, we’re talking about someone who studies disease at the level of the population, rather than individual cases. Fictional epidemiologists appear when there is an epidemic or pandemic in the plot, and so are concerned with two questions: What are we dealing with? and What do we need to do?
Part 1: What are we dealing with?
Our response should change for different types of threat. So it’s important for an epidemiologist to understand the nature of a pathogen. There are a few scenes in Contagion where we see scientists studying a screen with gene sequences and a protein-folding diagram, and this touches on understanding the nature of the virus. But this is a virologists view, and doesn’t touch on most of what an epidemiologist is ultimately hoping to build first, and that’s a case definition. It is unlikely to appear in a spreading pathogen map, but it should inform one. So even if your pathogen is fictional, you ought to understand what one is.
“We’ve sequenced the virus and determined its origin, and we’ve modeled the way it edges the cells of the lung and the brain…” —Dr. Hextall, Contagion (2011)
A case definition is the standard shared definition of what a pathogen is; how a real, live human case is classified as belonging to an epidemic or not. Some case definitions are built for non-emergency cases, like for influenza. The flu is practically a companion to humanity, i.e., with us all the time, and mutates, so its base definition for health surveillance can be a little vague. But for the epidemics and pandemics that are in sci-fi, they are building a case definitionfor outbreak investigations. These are for a pathogen in a particular time and place, and act as a standard for determining whether or not a given person is counted as a case for the purposes of studying the event.
Case definition for outbreak investigations
The CDC lists the following as the components of a case definition.
Clinical criteria
Clinical description
Confirmatory laboratory tests
These can be pages long, with descriptions of recommended specimen collections, transportation protocols, and reporting details.
Combinations of symptoms (subjective complaints)
Signs (objective physical findings)
Source
(Sometimes) Specifics of time and place.
There are sometimes different case definitions based on the combination of factors. COVID-19 case definitions with the World Health Organization, for instance, are broken down between suspect, probable, and confirmed. A person showing all the symptoms and who has been in an area where an infected person was would be suspect. A person whose laboratory results confirmed the presence of SARS-CoV-2 is confirmed. Notably for a map, these three levels might warrant three levels of color.
As an example, here is the CDC case definition for ebola, as of 09 JUL 2019.
n.b. Case definitions are unlikely to work on screen
Though the case definition is critical to epidemiology, and may help the designer create the spreading pathogen map (see the note about three levels of color, above), but the thing itself is too text-heavy to be of much use for a sci-fi interface, which rely much more on visuals. Better might be the name or an identifying UUID to the definition. WHO case references look like this: WHO/COVID-19/laboratory/2020.5 I do not believe the CDC has any kind of UUID for its case definitions.
While case definitions don’t work on screen, counts and rates do. See below under Surveil Public Health for more on counts and rates.
Disease timeline
Infectious disease follows a fairly standard order of events, depicted in the graphic below. Understanding this typical timeline of events helps you understand four key metrics for a given pathogen: chains of transmission, R0, SI, and CFR.
For each of the key metrics, I’ll list ranges and variabilities where appropriate. These are observed attributes in the real world, but an author creating a fictional pathogen, or a sci-fi interfaces maker needing to illustrate them, may need to know what those numbers look like and how they tend to behave over time so they can craft these attributes.
Chains of Transmission
What connects the individual cases in an epidemic are the methods of transmission. The CDC lists the following as the basics of transmission.
Reservoir: where the pathogen is collected. This could be the human body, or a colony of infected mynocks, a zombie, or a moldy Ameglian Major flank steak forgotten in a fridge. Or your lungs.
Portal of exit, or how the pathogen leaves the reservoir. Say, the open wound of a zombie, or an innocent recommendation, or an uncovered cough.
Mode of transmission tells how the pathogen gets from the portal of exit to the portal of entry. Real-world examples include mosquitos, fomites (you remember fomites from the beginning of this post, don’t you?), sex, or respiratory particles.
Portal of entry, how the pathogen infects a new host. Did you inhale that invisible cough droplet? Did you touch that light saber and then touch your gills? Now it’s in you like midichlorians.
Susceptible host is someone more likely than not to get the disease.
A map of this chain of transmission would be a fine secondary-screen to a spreading pathogen map, illustrating how the pathogen is transmitted. After all, this will inform the containment strategies.
Variability: Once the chain of transmission is known, it would only change if the pathogen mutated.
Basic Rate of Reproduction = How contagious it is
A famous number that’s associated with contagiousness is the basic reproduction rate. If you saw Contagion you’ll recall this is written as R0, and pronounced “R-naught.” It describes, on average, how many people an infected person will infect before they stop being infectious.
If R0 is below 1, an infected person is unlikely to infect another person, and the pathogen will quickly die out.
If R0 is 1, an infected person is likely to infect one other, and the disease will continue through a population at a steady rate without intervention.
If R0 is higher than 1, a pathogen stands to explode through a population.
The CDC book tells me that R0 describes how the pathogen would reproduce through the population with no intervention, but other sources talk of lowering the R0 so I’m not certain if those other sources are using it less formally, or if my understanding is wrong. For now I’ll go with the CDC, and talk about R0 as a thing that is fixed.
It, too, is not an easy thing to calculate. It can depend on the duration of contagiousness after a person becomes infected, or the likelihood of infection for each contact between a susceptible person and an infectious person or vector, and the contact rate.
Variability: It can change over time. When a novel pathogen first emerges, the data is too sparse and epidemiologists are scrambling to do the field work to confirm cases. As more data comes in and numbers get larger, the number will converge toward what will be its final number.
It can also differ based on geography, culture, geopolitical boundaries, and the season, but the literature (such as I’ve read) refers to R0 as a single number.
Range: The range of R0 >1 can be as high as 12–18, but measles morbillivirus is an infectious outlier. Average range of R0, not including measles, of this sample is 2.5–5.2. MEV-1 from Contagion has a major dramatic moment when it mutates and its predicted R0 becomes 4, making it roughly as contagious as the now-eradicated killer smallpox.
Serial interval is the average time between successive cases in a chain of transmission. This tells the epidemiologist how fast a pathogen stands to spread through a population.
Variability: Like the other numbers, SI is calculated and updated with new cases while an epidemic is underway, but tend to converge toward a number. SI for some respiratory diseases is charted below. Influenza A moves very fast. Pertussis is much slower.
Range: As you can see in the chart, SI can be as fast as 2.2 days, or as slow as 22.8 days. The median in this set is 14 days and the average is 12.8. SARS-CoV-2 is currently estimated to be about 4 days, which is very fast.
The case fatality rate is a percentage that any given case will prove fatal. It is very often shortened to CFR. This is not always easy to calculate.
Variability: Early in a pandemic it might be quite low because hospital treatment is still available. Later in a pandemic, as hospital and emergency rooms are packed full, the CFR might raise quite high. Until a pathogen is eradicated, the precise CFR is changing with each new case. Updates can occur daily, or in real time with reports. In a sci-fi world, it could update real time directly from ubiquitous sensors, and perhaps predicted by a specialty A.I. or precognitive character.
Range: Case fatality rates range from the incurable, like kuru, at 100%. to 0.001% for chickenpox affecting unvaccinated children. The CFR changes greatly at the start of a pandemic and slowly converges towards its final number.
So, if the spreading pathogen map is meant to convey to an epidemiologist the nature of the pathogen, it should display these four factors:
Mode of Transmission: How it spreads
R0: How contagious it is
SI: How fast it spreads
CFR: How deadly it is
Part 2: What do we do?
An epidemiologist during an outbreak has a number of important responsibilities beyond understanding the nature of the pathogen. I’ve taken a crack at listing those below. Note: this list is my interpretation of the CDC materials, rather than their list. As always, offer corrections in comments.
Surveil the current state of things
Prevent further infections
Communicate recommendations
Epidemiology has other non-outbreak functions, but those routine, non-emergency responsibilities rarely make it to cinema. And since “communicate recommendations” is pretty covered under “The Simple Case,” above, the rest of this post will be dedicated to health surveillance and prevention tools.
Surveil the current state of things
In movies the current state of things is often communicated via the spreading pathogen map in some command and control center. The key information on these maps are counts and rates.
Counts and Rates
The case definition (above) helps field epidemiologists know which cases to consider in the data set for a given outbreak. They routinely submit reports of their cases to central authorities like the CDC or WHO, who aggregate them into counts, which are tallies of known cases. (And though official sources in the real world are rightly cautious to do it, sci-fi could also include an additional layer of suspected or projected cases.) Counts, especially over time, are important for tracking the spread of a virus. Most movie goers have basic numeracy, so red number going up = bad is an easy read for an audience.
Counts can be broken down into many variables. Geopolitical regions make sense as governmental policies and cultural beliefs can make meaningful distinctions in how a pathogen spreads. In sci-fi a speculative pathogen might warrant different breakdowns, like frequency of teleportation, or time spent in FTL warp fields, or genetic distance from the all-mother.
In the screen cap of the John Hopkins COVID-19 tracker, you can see counts high in the visual hierarchy for total confirmed (in red), total deaths (in white), and total recovered (in green). The map plots current status of the counts.
Rates is another number that epidemiologists are interested in, to help normalize the spread of a pathogen for different group sizes. (Colloquially, rate often implies change over time, but in the field of epidemiology, it is a static per capita measurement of a point in time.) For example, 100 cases is around a 0.00001% rate in China, with its population of 1.386 billion, but it would be a full 10% rate of Vatican City, so count can be a poor comparison to understand how much of a given population is affected. By representing the rates alongside the counts you can detect if it’s affecting a subgroup of the global population more or less than others of its kind, which may warrant investigation into causes, or provide a grim lesson to those who take the threat lightly.
Counts and rates over time
The trend line in the bottom right of the Johns Hopkins dashboard helps understand how the counts of cases are going over time, and might be quite useful for helping telegraph the state of the pandemic to an audience, though having it tucked in a corner and in orange may not draw attention as it needs to for instant-understanding.
These two displays show different data, and one is more cinegenic than the other. Confirmed cases, on the left, is a total, and at best will only ever level off. If you know what you’re looking at, you know that older cases represented by the graph are…uh…resolved (i.e. recovery, disability, or death) and that a level-off is the thing we want to see there. But the chart on the right plots the daily increase, and will look something like a bell curve when the pandemic comes to an end. That is a more immediate read (bad thing was increasing, bad thing peaked, bad thing is on the decline) and so I think is better for cinema.
At a glance you can also tell that China appears to have its shit sorted. [Obviously this is an old screen grab.]
In the totals, sparklines would additionally help a viewer know whether things are getting better or getting worse in the individual geos, and would help sell the data via small multiples on a close-up.
Plotting cases on maps
Counts and rates are mostly tables of numbers with a few visualizations. The most cinegenic thing you can show are cases on geopolitical maps. All of the examples, except the trainwreck that is The Andromedia Strain pathogen map, show this, even the extradiegetic ones. Real-world pathogens mostly spread through physical means, so physical counts of areas help you understand where the confirmed cases are.
Which projection?
But as we all remember from that one West Wing scene, projections have consequences. When wondering where in the world do we send much-needed resources, Mercator will lie to you, exaggerating land at the poles at the expense of equatorial regions. I am a longtime advocate for alternate projections, such as—from the West Wing scene—the Gall-Peters. I am an even bigger big fan of Dymaxion and Watterman projections. I think they look quite sci-fi because they are familiar-but-unfamiliar, and they have some advantages for showing things like abstract routes across the globe.
If any supergenre is here to help model the way things ought to be, it’s sci-fi. If you only have a second or less of time to show the map, then you may be locked to Mercator for its instant-recognizability, but if the camera lingers, or you have dialogue to address the unfamiliarity, or if the art direction is looking for uncanny-ness, I’d try for one of the others.
What is represented?
Of course you’re going to want to represent the cases on the map. That’s the core of it. And it may be enough if the simple takeaway is thing bad getting worse. But if the purpose of the map is to answer the question “what do we do,” the cases may not be enough. Recall that another primary goal of epidemiologists is to prevent further infections. And the map can help indicate this and inform strategy.
Take for instance, 06 APR 2020 of the COVID-19 epidemic in the United States. If you had just looked at a static map of cases, blue states had higher counts than red states. But blue states had been much more aggressive in adopting “flattening the curve” tactics, while red states had been listening to Trump and right wing media that had downplayed the risk for many weeks in many ways. (Read the Nate Silver post for more on this.) If you were an epidemiologist, seeing just the cases on that date might have led you to want to focus social persuasion resources on blue states. But those states have taken the science to heart. Red states on the other hand, needed a heavy blitz of media to convince them that it was necessary to adopt social distancing and shelter-in-place directives. With a map showing both cases andsocial acceptance of the pandemic, it might have helped an epidemiologist make the right resource allocation decision quickly.
Another example is travel routes. International travel played a huge role in spreading COVID-19, and visualizations of transportation routes can prove more informative in understanding its spread than geographic maps. Below is a screenshot of the New York Times’ beautiful COVID-19 MAR 2020 visualization How the Virus Got Out, which illustrates this point.
Other things that might be visualized depend, again, on the chain of transmission.
Is the pathogen airborne? Then you might need to show upcoming wind and weather forecasts.
Is the reservoir mosquitoes? Then you might want to show distance to bodies of still water.
Is the pathogen spread through the mycelial network? Then you might need to show an overlay of the cosmic mushroom threads.
Whatever your pathogen, use the map to show the epidemiologist ways to think about its future spread, and decide what to do. Give access to multiple views if needed.
How do you represent it?
When showing intensity-by-area, there are lots of ways you could show it. All of them have trade offs. The Johns-Hopkins dashboard uses a Proportional Symbol map, with a red dot, centered on the country or state, the radius of which is larger for more confirmed cases. I don’t like this for pandemics, mostly because the red dots begin to overlap and make it difficult to any detail without interacting with the map to get a better focus. It does make for an immediate read. In this 23 MAR 2020 screen cap, it’s pretty obvious that the US, Europe, and China are current hotspots, but to get more detail you have to zoom in, and the audience, if not the characters, don’t have that option. I suppose it also provides a tone-painting sense of unease when the symbols become larger than the area they are meant to represent. It looks and feels like the area is overwhelmed with the pathogen, which is an appropriate, if emotional and uninformative, read.
The Johns-Hopkins dashboard uses a proportional symbol map. And I am distraught at how quaint those numbers seem now, much less what they will be in the future.
Most of the sci-fi maps we see are a variety of Chorochromatic map, where color is applied to the discrete thing where it appears on the map. (This is as opposed to a Cloropleth map, where color fills in existing geopolitical regions.) The chorochromatic option is nice for sci-fi because the color makes a shape—a thing—that does not know of or respect geopolitical boundaries. See the example from Evolution below.
Governor Lewis watches the predicted spread of the Glen Canyon asteroid organisms out of Arizona and to the whole of North America. Evolution (2001)
It can be hard to know (or pointlessly-detailed) to show exactly where a given thing is on a map, like, say, where infected people literally are. To overcome this you could use a dot-distribution map, as in the Outbreak example (repeated below so you don’t have to scroll that far back up).
Outbreak (1995), again.
Like many such maps, the dot-distribution becomes solid red to emphasize passing over some magnitude threshold. For my money, the dots are a little deceptive, as if each dot represented a person rather than part of a pattern than indicates magnitude, but a glance at the whole map gives the right impression.
For a real world example of dot-distribution for COVID-19, see this example posted to reddit.com by user Edward-EFHIII.
COVID-19 spread from January 23 through March 14th.
Often times dot-distribution is reserved for low magnitudes, and once infections are over a threshold, become cloropleth maps. See this example from the world of gaming.
In Plague, Inc., you play the virus, hoping to win against humanity.
Here you can see that India and Australia have dots, while China, Kyrgyzstan, Tajikistan, Turkmenistan, and Afghanistan (I think) are “solid” red.
The other representation that might make sense is a cartogram, in which predefined areas (like country or state boundaries) are scaled to show the magnitude of a variable. Continuous-area cartograms can look hallucinogenic, and would need some explanation by dialogue, but can overcome the inherent bias that size = importance. It might be a nice secondary screen alongside a more traditional one.
On the left, a Choropleth map of the 2012 US presidential election, where it looks like red states should have won. On the right, a continuous cartogram with state sizes scaled to reflect states’ populations, making more intuitive sense why blue states carried the day.
Another gorgeous projection dispenses with the geographic layout. Dirk Brockman, professor at the Institute for Theoretical Biology, Humboldt University, Berlin, developed a visualization that places the epicenter of a disease at the center of a node graph, and plots every city around it based on how many airport flights it takes to get there. Plotting proportional symbols to this graph makes the spread of the disease radiate in mostly- predictable waves. Pause the animation below and look at the red circles. You can easily predict where the next ones will likely be. That’s an incredibly useful display for the epidemiologist. And as a bonus, it’s gorgeous and a bit mysterious, so would make a fine addition in a sci-fi display to a more traditional map. Read more about this innovative display on the CityLab blog. (And thanks, Mark Coleran, for the pointer.)
How does it move?
First I should say I don’t know that it needs to move. We have information graphics that display predicted change-over-area without motion: Hurricane forecast maps. These describe a thing’s location in time, and simultaneously, the places it is likely to be in the next few days.
National Hurricane Center’s 5-day forecast for Hurricane Florence, 08 SEP 2018. Image: NHC
If you are showing a chorochromatic map, then you can use “contour lines” or color regions to demonstrate the future predictions.
Not based on any real pathogen.
Another possibility is small multiples, where the data is spread out over space instead of time. This makes it harder to compare stages, but doesn’t have the user searching for the view they want. You can mitigate this with small lines on each view representing the boundaries of other stages.
Not based on any real pathogen.
The side views could also represent scenarios. Instead of +1, +2, etc., the side views could show the modeled results for different choices. Perhaps those scenario side views and their projected counts could be animated.
To sing the praises of the static map: Such a view, updated as data comes in, means a user does not have to wait for the right frame to pop up, or interact with a control to get the right piece of information, or miss some detail when they just happened to have the display paused on the wrong frame of an animation.
But, I realize that static maps are not as cinegenic as a moving map. Movement is critical to cinema, so a static map, updating only occasionally as new data comes in, could look pretty lifeless. Animation gives the audience more to feel as some red shape slowly spreads to encompass the whole world. So, sure. I think there are better things to animate than the primary map, but doing so puts us back into questions of style rather than usability, so I’ll leave off that chain of thought and instead show you the fourth example in this section, Contagion.
MEV-1 spreads from fomites! It’s fomites! Contagion (2011), designed by Cory Bramall of Decca Digital.
Later stages of the map are interspersed with viewer-reaction shots.
Prevent further transmissions: Containment strategies
The main tactic for epidemiological intervention is to deny pathogens the opportunity to jump to new hosts. The top-down way to do this is to persuade community leaders to issue broad instructions, like the ones around the world that have us keeping our distance from strangers, wearing masks and gloves, and sheltering-in-place. The bottom-up tactic is to identify those who have been infected or put at risk for contracting a pathogen from an infected person. This is done with contact tracing.
Contain Known Cases
When susceptible hosts simply do not know whether or not they are infected, some people will take their lack of symptoms to mean they are not infectious and do risky things. If these people are infectious but not yet showing symptoms, they spread the disease. For this reason, it’s critical to do contact tracing of known cases to inform and encourage people to get tested and adopt containment behaviors.
Contact tracing
There are lots of scenes in pathogen movies where scientists stand around whiteboards with hastily-written diagrams of who-came-into-contact-with-whom, as they hope to find and isolate cases, or to find “patient 0,” or to identify super-spreaders and isolate them.
These scenes seem ripe for improvement by technology and AI. There are opt-in self-reporting systems, like those that were used to contain COVID-19 in South Korea, or the proposed NextTrace system in the West. In sci-fi, this can go further.
Scenario: Imagine an epidemiologist talking to the WHO AI and asking it to review public footage, social media platforms, and cell phone records to identify all the people that a given case has been in contact with. It could even reach out and do field work, calling humans (think Google Duplex) who might be able to fill in its information gaps. Field epidemiologists are focused on situations when the suspected cases don’t have phones or computers.
Or, for that matter, we should ask why the machine should wait to be asked. It should be set up as an agent, reviewing these data feeds continually, and reaching out in real time to manage an outbreak.
SCENE: Karen is walking down the sidewalk when her phone rings.
Computer voice:
Good afternoon, Karen. This is Florence, the AI working on behalf of the World Health Organization.
Karen:
Oh no. Am I sick?
Computer voice:
Public records indicate you were on a bus near a person who was just confirmed to be infected. Your phone tells me your heart rate has been elevated today. Can you hold the phone up to your face so I can check for a fever?
Karen does. As the phone does its scan, people on the sidewalk behind her can be seen to read texts on their phone and move to the other side of the street. Karen sees that Florence is done, and puts the phone back to her ear.
Computer voice:
It looks as if you do have a fever. You should begin social distancing immediately, and improvise a mask. But we still need a formal test to be sure. Can you make it to the testing center on your own, or may I summon an ambulance? It is a ten minute walk away.
Karen:
I think I can make it, but I’ll need directions.
Computer voice:
Of course. I have also contacted your employer and spun up an AI which will be at work in your stead while you self-isolate. Thank you for taking care of yourself, Karen. We can beat this together.
Design challenge: In the case of an agentive contact tracer, the display would be a social graph displayed over time, showing confirmed cases as they connect to suspected cases (using evidence-of-proximity or evidence-of-transmission) as well as the ongoing agent’s work in contacting them and arranging testing. It would show isolation monitoring and predicted risks to break isolation. It would prioritize cases that are greatest risk for spreading the pathogen, and reach out for human intervention when its contact attempts failed or met resistance. It could be simultaneously tracing contacts “forward” to minimize new infections and tracing contacts backward to find a pathogen’s origins.
Another consideration for such a display is extension beyond the human network. Most pathogens mutate and much more freely in livestock and wild animal populations, making their way into humans occasionally. it happened this way for SARS (bats → civets → people), MERS (bats → camels → people), and COVID-19 (bats → pangolin → people). (Read more about bats as a reservoir.) It’s not always bats, by the way, livestock are also notorious breeding grounds for novel pathogens. Remember Bird flu? Swine flu? This “zoonotic network” should be a part of any pathogen forensic or surveillance interface.
Design idea: Even the notion of what it means to do contact tracing can be rethought in sci-fi. Have you seen the Mythbusters episode “Contamination”? In it Adam Savage has a tube latexed to his face, right near his nose that drips a florescent dye at the same rate a person’s runny nose might drip. Then he attends a staged dinner party where, despite keeping a napkin on hand to dab at the fluid, the dye gets everywhere except the one germophobe. It brilliantly illustrates the notion of fomites and how quickly an individual can spread a pathogen socially.
Now imagine this same sort of tracing, but instead of dye, it is done with computation. A camera watches, say, grocery shelves, and notes who touched what where and records the digital “touch,” or touchprint, along with an ID for the individual and the area of contact. This touchprint could be exposed directly with augmented reality, appearing much like the dye under black light. The digital touch mark would only be removed from the digital record of the object if it is disinfected, or after the standard duration of surface stability expires. (Surface stability is how long a pathogen remains a threat on a given surface). The computer could further watch the object for who touches it next, and build an extended graph of the potential contact-through-fomites.
Ew, I got touchprint on me.
You could show the AR touchprint to the individual doing the touching, this would help remind them to wear protective gloves if the science calls for it, or to ask them to disinfect the object themselves. A digital touchprint could also be used for workers tasked with disinfecting the surfaces, or by disinfecting drones. Lastly, if an individual is confirmed to have the pathogen, the touchprint graph could immediately identify those who had touched an object at the same spot as the infected person. The system could provide field epidemiologists with an instant list of people to contact (and things to clean), or, if the Florence AI described above was active, the system could reach out to individuals directly. The amount of data in such a system would be massive, and the aforementioned privacy issues would be similarly massive, but in sci-fi you can bypass the technical constraints, and the privacy issues might just be a part of the diegesis.
In case you’re wondering how long that touch mark would last for SARS-CoV-2 (the virus that causes COVID-19), this study from the New England Journal of Medicine says it’s 4 hours for copper, 24 hours for paper and cardboard, and 72 hours on plastic and steel.
Anyway, all of this is to say that the ongoing efforts by the agent to do the easy contact tracing would be an excellent, complicated, cinegenic side-display to a spreading pathogen map.
Destroying non-human reservoirs
Another way to reduce the risk of infection is to seal or destroy reservoirs. Communities encourage residents to search their properties and remove any standing water to remove the breeding grounds for mosquitos, for example. There is the dark possibility that a pathogen is so lethal that a government might want to “nuke it from orbit” and kill even human reservoirs. Outbreak features an extended scene where soldiers seek to secure a neighborhood known to be infected with the fictional Motoba virus, and soldiers threaten to murder a man trying to escape with his family. For this dark reason, in addition to distance-from-reservoir, the location of actual reservoirs may be important to your spreading pathogen map. Maybe also counts of the Hail Mary tools that are available, their readiness, effects, etc.
To close out the topic of What Do We Do? Let me now point you to the excellent and widely-citied Medium article by Tomas Peuyo, “Act Today or People Will Die,” for thoughts on that real-world question.
The…winner(?)
At the time of publication, this is the longest post I’ve written on this blog. Partly that’s because I wanted to post it as a single thing, but also because it’s a deep subject that’s very important to the world, and there are lots and lots of variables to consider when designing one.
Which makes it not surprising that most of the examples in this mini survey are kind of weak, with only one true standout. That standout is the World War Z spreading disaster map, shown below.
World War Z (2013)
It goes by pretty quickly, but you can see more features discussed above in this clip than any of the other exmaples.
A combination of chorochromatic marking for the zombie infection, and cloropleth marking for countries. Note the signals showing countries where data is unavailable.Along the bottom, rates (not cases) are expressed as “Population remaining.” That bar of people along the bottom would start slow and then just explode to red, but it’s a nice “things getting worse” moment. Maybe it’s a log scale?A nice augmentation of the main graphic is down the right-hand side. A day count in the upper right (with its shout-out to zombie classic 28 Days Later), and what I’m guessing are resources, including nukes.
It doesn’t have that critical layer of forecasting data, but it got so much more right than its peers, I’m still happy to have it. Thanks to Mark Coleran for pointing me to it.
Let’s not forget that we are talking about fiction, and few people in the audience will be epidemiologists, standing up in the middle of the cinema (remember when we could go to cinemas?) to shout, “What’s with this R0 of 0.5? What is this, the LaCroix of viruses?” But c’mon, surely we can make something other than Andromeda Strain’s Pathogen Kaleidoscope, or Contagion’s Powerpoint wipe. Modern sci-fi interfaces are about spectacle, about overwhelming the users with information they can’t possibly process, and which they feel certain our heroes can—but they can still be grounded in reality.
Lastly, while I’ve enjoyed the escapism of talking about pandemics in fiction, COVID-19 is very much with us and very much a threat. Please take it seriously and adopt every containment behavior you can. Thank you for taking care of yourself. We can beat this together.
Distinguishing replicants from humans is a tricky business. Since they are indistinguishable biologically, it requires an empathy test, during which the subject hears empathy-eliciting scenarios and watched carefully for telltale signs such as, “capillary dilation—the so-called blush response…fluctuation of the pupil…involuntary dilation of the iris.” To aid the blade runner in this examination, they use a portable machine called the Voight-Kampff machine, named, presumably, for its inventors.
The device is the size of a thick laptop computer, and rests flat on the table between the blade runner and subject. When the blade runner prepares the machine for the test, they turn it on, and a small adjustable armature rises from the machine, the end of which is an intricate piece of hardware, housing a powerful camera, glowing red.
The blade runner trains this camera on one of the subject’s eyes. Then, while reading from the playbook book of scenarios, they keep watch on a large monitor, which shows an magnified image of the subject’s eye. (Ostensibly, anyway. More on this below.) A small bellows on the subject’s side of the machine raises and lowers. On the blade runner’s side of the machine, a row of lights reflect the volume of the subject’s speech. Three square, white buttons sit to the right of the main monitor. In Leon’s test we see Holden press the leftmost of the three, and the iris in the monitor becomes brighter, illuminated from some unseen light source. The purpose of the other two square buttons is unknown. Two smaller monochrome monitors sit to the left of the main monitor, showing moving but otherwise inscrutable forms of information.
In theory, the system allows the blade runner to more easily watch for the minute telltale changes in the eye and blush response, while keeping a comfortable social distance from the subject. Substandard responses reveal a lack of empathy and thereby a high probability that the subject is a replicant. Simple! But on review, it’s shit. I know this is going to upset fans, so let me enumerate the reasons, and then propose a better solution.
-2. Wouldn’t a genetic test make more sense?
If the replicants are genetically engineered for short lives, wouldn’t a genetic test make more sense? Take a drop of blood and look for markers of incredibly short telomeres or something.
-1. Wouldn’t an fMRI make more sense?
An fMRI would reveal empathic responses in the inferior frontal gyrus, or cognitive responses in the ventromedial prefrontal gyrus. (The brain structures responsible for these responses.) Certinaly more expensive, but more certain.
0. Wouldn’t a metal detector make more sense?
If you are testing employees to detect which ones are the murdery ones and which ones aren’t, you might want to test whether they are bringing a tool of murder with them. Because once they’re found out, they might want to murder you. This scene should be rewritten such that Leon leaps across the desk and strangles Holden, IMHO. It would make him, and other blade runners, seem much more feral and unpredictable.
(OK, those aren’t interface issues but seriously wtf. Onward.)
1. Labels, people
Controls needs labels. Especially when the buttons have no natural affordance and the costs of experimentation to discover the function are high. Remembering the functions of unlabeled controls adds to the cognitive load for a user who should be focusing on the person across the table. At least an illuminated button helps signal the state, so that, at least, is something.
2. It should be less intimidating
The physical design is quite intimidating: The way it puts a barrier in between the blade runner and subject. The fact that all the displays point away from the subject. The weird intricacy of the camera, its ominous HAL-like red glow. Regular readers may note that the eyepiece is red-on-black and pointy. That is to say, it is aposematic. That is to say, it looks evil. That is to say, intimidating.
I’m no emotion-scientist, but I’m pretty sure that if you’re testing for empathy, you don’t want to complicate things by introducing intimidation into the equation. Yes, yes, yes, the machine works by making the subject feel like they have to defend themselves from the accusations in the ethical dilemmas, but that stress should come from the content, not the machine.
2a. Holden should be less intimidating and not tip his hand
While we’re on this point, let me add that Holden should be less intimidating, too. When Holden tells Leon that a tortoise and a turtle are the same thing, (Narrator: They aren’t) he happens to glance down at the machine. At that moment, Leon says, “I’ve never seen a turtle,” a light shines on the pupil and the iris contracts. Holden sees this and then gets all “ok, replicant” and becomes hostile toward Leon.
In case it needs saying: If you are trying to tell whether the person across from you is a murderous replicant, and you suddenly think the answer is yes, you do not tip your hand and let them know what you know. Because they will no longer have a reason to hide their murderyness. Because they will murder you, and then escape, to murder again. That’s like, blade runner 101, HOLDEN.
3. It should display history
The glance moment points out another flaw in the interface. Holden happens to be looking down at the machine at that moment. If he wasn’t paying attention, he would have missed the signal. The machine needs to display the interview over time, and draw his attention to troublesome moments. That way, when his attention returns to the machine, he can see that something important happened, even if it’s not happening now, and tell at a glance what the thing was.
4. It should track the subject’s eyes
Holden asks Leon to stay very still. But people are bound to involuntarily move as their attention drifts to the content of the empathy dilemmas. Are we going to add noncompliance-guilt to the list of emotional complications? Use visual recognition algorithms and high-resolution cameras to just track the subject’s eyes no matter how they shift in their seat.
5. Really? A bellows?
The bellows doesn’t make much sense either. I don’t believe it could, at the distance it sits from the subject, help detect “capillary dilation” or “ophthalmological measurements”. But it’s certainly creepy and Terry Gilliam-esque. It adds to the pointless intimidation.
6. It should show the actual subject’s eye
The eye color that appears on the monitor (hazel) matches neither Leon’s (a striking blue) or Rachel’s (a rich brown). Hat tip to Typeset in the Future for this observation. His is a great review.
7. It should visualize things in ways that make it easy to detect differences in key measurements
Even if the inky, dancing black blob is meant to convey some sort of information, the shape is too organic for anyone to make meaningful readings from it. Like seriously, what is this meant to convey?
The spectrograph to the left looks a little more convincing, but it still requires the blade runner to do all the work of recognizing when things are out of expected ranges.
8. The machine should, you know, help them
The machine asks its blade runner to do a lot of work to use it. This is visual work and memory work and even work estimating when things are out of norms. But this is all something the machine could help them with. Fortunately, this is a tractable problem, using the mighty powers of logic and design.
Pupillary diameter
People are notoriously bad at estimating the sizes of things by sight. Computers, however, are good at it. Help the blade runner by providing a measurement of the thing they are watching for: pupillary diameter. (n.b. The script speaks of both iris constriction and pupillary diameter, but these are the same thing.) Keep it convincing and looking cool by having this be an overlay on the live video of the subject’s eye.
So now there’s some precision to work with. But as noted above, we don’t want to burden the user’s memory with having to remember stuff, and we don’t want them to just be glued to the screen, hoping they don’t miss something important. People are terrible at vigilance tasks. Computers are great at them. The machine should track and display the information from the whole session.
Note that the display illustrates radius, but displays diameter. That buys some efficiencies in the final interface.
Now, with the data-over-time, the user can glance to see what’s been happening and a precise comparison of that measurement over time. But, tracking in detail, we quickly run out of screen real estate. So let’s break the display into increments with differing scales.
There may be more useful increments, but microseconds and seconds feel pretty convincing, with the leftmost column compressing gradually over time to show everything from the beginning of the interview. Now the user has a whole picture to look at. But this still burdens them into noticing when these measurements are out of normal human ranges. So, let’s plot the threshold, and note when measurements fall outside of that. In this case, it feels right that replicants display less that normal pupillary dilation, so it’s a lower-boundary threshold. The interface should highlight when the measurement dips below this.
Blush
I think that covers everything for the pupillary diameter. The other measurement mentioned in the dialogue is capillary dilation of the face, or the “so-called blush response.” As we did for pupillary diameter, let’s also show a measurement of the subject’s skin temperature over time as a line chart. (You might think skin color is a more natural measurement, but for replicants with a darker skin tone than our two pasty examples Leon and Rachel, temperature via infrared is a more reliable metric.) For visual interest, let’s show thumbnails from the video. We can augment the image with degree-of-blush. Reduce the image to high contrast grayscale, use visual recognition to isolate the face, and then provide an overlay to the face that illustrates the degree of blush.
But again, we’re not just looking for blush changes. No, we’re looking for blush compared to human norms for the test. It would look different if we were looking for more blushing in our subject than humans, but since the replicants are less empathetic than humans, we would want to compare and highlight measurements below a threshold. In the thumbnails, the background can be colored to show the median for expected norms, to make comparisons to the face easy. (Shown in the drawing to the right, below.) If the face looks too pale compared to the norm, that’s an indication that we might be looking at a replicant. Or a psychopath.
So now we have solid displays that help the blade runner detect pupillary diameter and blush over time. But it’s not that any diameter changes or blushing is bad. The idea is to detect whether the subject has less of a reaction than norms to what the blade runner is saying. The display should be annotating what the blade runner has said at each moment in time. And since human psychology is a complex thing, it should also track video of the blade runner’s expressions as well, since, as we see above, not all blade runners are able to maintain a poker face. HOLDEN.
Anyway, we can use the same thumbnail display of the face, without augmentation. Below that we can display the waveform (because they look cool), and speech-to-text the words that are being spoken. To ensure that the blade runner’s administration of the text is not unduly influencing the results, let’s add an overlay to the ideal intonation targets. Despite evidence in the film, let’s presume Holden is a trained professional, and he does not stray from those targets, so let’s skip designing the highlight and recourse-for-infraction for now.
Finally, since they’re working from a structured script, we can provide a “chapter” marker at the bottom for easy reference later.
Now we can put it all together, and it looks like this. One last thing we can do to help the blade runner is to highlight when all the signals indicate replicant-ness at once. This signal can’t be too much, or replicants being tested would know from the light on the blade runner’s face when their jig is up, and try to flee. Or murder. HOLDEN.
For this comp, I added a gray overlay to the column where pupillary and blush responses both indicated trouble. A visual designer would find some more elegant treatment.
If we were redesigning this from scratch, we could specify a wide display to accomodate this width. But if we are trying to squeeze this display into the existing prop from the movie, here’s how we could do it.
Note the added labels for the white squares. I picked some labels that would make sense in the context. “Calibrate” and “record” should be obvious. The idea behind “mark” is an easy button for the blade runner to press when they see something that looks weird, like when doctors manually annotate cardiograph output.
Lying to Leon
There’s one more thing we can add to the machine that would help out, and that’s a display for the subject. Recall the machine is meant to test for replicant-ness, which happens to equate to murdery-ness. A positive result from the machine needs to be handled carefully so what happens to Holden in the movie doesn’t happen. I mentioned making the positive-overlay subtle above, but we can also make a placebo display on the subject’s side of the interface.
The visual hierarchy of this should make the subject feel like its purpose is to help them, but the real purpose is to make them think that everything’s fine. Given the script, I’d say a teleprompt of the empathy dilemma should take up the majority of this display. Oh, they think, this is to help me understand what’s being said, like a closed caption. Below the teleprompt, at a much smaller scale, a bar at the bottom is the real point.
On the left of this bar, a live waveform of the audio in the room helps the subject know that the machine is testing things live. In the middle, we can put one of those bouncy fuiget displays that clutters so many sci-fi interfaces. It’s there to be inscrutable, but convince the subject that the machine is really sophisticated. (Hey, a diegetic fuiget!) Lastly—and this is the important part—An area shows that everything is “within range.” This tells the subject that they can be at ease. This is good for the human subject, because they know they’re innocent. And if it’s a replicant subject, this false comfort protects the blade runner from sudden murder. This test might flicker or change occasionally to something ambiguous like “at range,” to convey that it is responding to real world input, but it would never change to something incriminating.
This way, once the blade runner has the data to confirm that the subject is a replicant, they can continue to the end of the module as if everything was normal, thank the replicant for their time, and let them leave the room believing they passed the test. Then the results can be sent to the precinct and authorizations returned so retirement can be planned with the added benefit of the element of surprise.
OK
Look, I’m sad about this, too. The Voight-Kampff machine is cool. It fits very well within the art direction of the Blade Runner universe. This coolness burned the machine into my memory when I saw this film the first dozen times, but despite that, it just doesn’t stand up to inspection. It’s not hopeless, but does need a lot of thinkwork and design to make it really fit to task, and convincing to us in the audience.
Whew. So we all waited on tenterhooks through November to see if somehow Tyrell Corporation would be founded, develop and commercialize general AI, and then advance robot evolution into the NEXUS phase, all while in the background space travel was perfected, Off-world colonies and asteroid mining established, global warming somehow drenched Los Angeles in permanent rain and flares, and flying cars appear on the market. None of that happened. At least not publicly. So, with Blade Runner squarely part of the paleofuture past, let’s grab our neon-tube umbrellas and head into the rain to check out this classic that features some interesting technologies and some interesting AI.
Release date: 25 Jun 1982
The punctuation-challenged crawl for the film:
“Early in the 21st Century, THE TYRELL CORPORATION advanced Robot evolution into the NEXUS phase—a being virtually identical to a human—known as a Replicant. [sic] The NEXUS 6 Replicants were superior in strength and agility, and at least equal in intelligence, to the genetic engineers who created them. Replicants were used Off-world as slave labor, in the hazardous exploration and colonisation of other planets. After a bloody mutiny by a NEXUS 6 combat team in an Off-world colony, Replicants were declared illegal on Earth—under penalty of death. Special police squads—BLADE RUNNER UNITS—had orders to shoot to kill, upon detection, any trespassing Replicants.
“This was not called execution. It was called retirement.”
Four murderous replicants make their way to Earth, to try and find a way to extend their genetically-shortened life spans. The Blade Runner named Deckard is coerced by his ex-superior Bryant and detective Gaff out of retirement and into finding and “retiring” these replicants.
Deckard meets Dr. Tyrell to interview him, and at Tyrell’s request tests Rachel on a Voight-Kampff machine, which is designed to help blade runners tell replicants from people. Deckard and Rachel learn that she is a replicant. Then with Gaff, he follows clues to the apartment of one exposed replicant, Leon, where he finds a synthetic snake scale in the bathtub and a set of photographs in a drawer. Using a sophisticated image inspection tool in his home, he scans one of the photos taken in Leon’s apartment, until he finds the reflection of a face. He prints the image to take with him.
He takes the snake scale to someone with an electron microscope who is able to read the micrometer-scale “maker’s serial number” there. He visits the maker, a person named “the Egyptian,” who tells Deckard he sold the snake to Taffey Lewis. Deckard visits Taffey’s bar, where he sees Zhora, another of the wanted replicants, perform a stage act with a snake. She matches the picture he holds. He heads backstage to talk to her in her dressing room, posing as a representative of the “American Federation of Variety Artists, Confidential Committee on Moral Abuses.” When she finishes pretending to prepare for her next act, she attacks him and flees. He chases and retires her. Leon happens to witness the killing, and attacks Deckard. Leon has the upper hand but Deckard is saved when Rachel appears from the crowd and shoots Leon in the head. They return to his apartment. They totally make out.
Meanwhile, Roy has learned of a Tyrell employee named Sebastian who does genetic design. On orders, Pris befriends Sebastian and dupes him into letting her into his apartment. She then lets Roy in. Sebastian figures out that they are replicants, but confesses he cannot help them directly. Roy intimidates Sebastian into arranging a meeting between him and Dr. Tyrell. At the meeting, Tyrell says there is nothing that can be done. In fury, Roy kills Tyrell and Sebastian.
The police investigating the scene contact Deckard with Sebastian’s address. Deckard heads there, where he finds, fights, and retires Pris. Roy is there, too, but proves too tough for Deckard to retire. Roy could kill Deckard but instead opts to die peacefully, even poetically. Witnessing this act of grace, Deckard comes to appreciate the “humanity” of the replicants, and returns home to elope with Rachel.
P.S. This series uses “The Final Cut” edit of the movie, so I don’t have to hear that wretchedly-scripted voiceover from the theatrical release. If you can, I recommend seeing that version.
As Colossus: The Forbin Project opens, we are treated to an establishing montage of 1970’s circuit boards (with resistors), whirring doodads, punched tape, ticking Nixie tube numerals, beeping lights, and jerking control data tapes. Then a human hand breaks into frame, and twiddles a few buttons as an oscilloscope draws lines creepily like an ECG cardiac cycle. This hand belongs to Charles Forbin, who walks alone in this massive underground compound, making sure final preparations are in order. The matte paintings make this space seem vast, inviting comparisons to the Krell technopolis from Forbidden Planet.
Forbin pulls out a remote control and presses something on its surface to illuminate rows and rows of lights. He walks across a drawbridge over a moat. Once on the far side, he uses the remote control to close the massive door, withdraw the bridge and seal the compound.
The remote control is about the size of a smartphone, with a long antenna extending out the top. Etched type across the top reads “COLOSSUS COMPUTER SYSTEMS.” A row of buttons is labeled A–E. Large red capital letters warn DANGER RADIATION above a safety cover. The cover has an arrow pointing right. Another row of five buttons is labeled SLIDING WALLS and numbered 1–5. A final row of three buttons is labeled RAMPS and numbered 1–3.
Forbin flips open the safety cover. He presses the red button underneath, and a blood-red light floods the bottom of the moat and turns blue-white hot, while a theremin-y whistle tells you this is no place a person should go. Forbin flips the cover back into place and walks out the sealed compound to the reporters and colleagues who await him.
I can’t help but ask one non-tech narrative question: Why is Forbin turning lights on when he is about to abandon the compound? It might be that the illumination is a side-effect of the power systems, but it looks like he’s turning on the lights just before leaving and locking the house. Does he want to fool people into thinking there’s someone home? Maybe it should be going from fully-lit to an eerie, red low-light kinda vibe.
The Remote Control
The layout is really messy. Some rows are crowded and others have way too much space. (Honestly, it looks like the director demanded there be moar buttins make tecc! and forced the prop designer to add the A–E.) The crowding makes it tough to immediately know what labels go with what controls. Are A–E the radiation bits, and the safety cover control sliding walls? Bounding boxes or white space or some alternate layout would make the connections clear.
You might be tempted to put all of the controls in strict chronological order, but the gamma shielding is the most dangerous thing, and having it in the center helps prevent accidental activation, so it belongs there. And otherwise, it is in chronological order.
The labeling is inconsistent. Sure, maybe A–E the five computer systems that comprise Colossus. Sliding walls and ramps are well labeled, but there’s no indication about what it is that causes the dangerous radiation. It should say something like “Gamma shielding: DANGER RADIATION.” It’s tiny, but I also think the little arrow is a bad graphic for showing which way the safety cover flips open. Existing designs show that the industrial design can signal this same information with easier-to-understand affordances. And since this gamma radiation is an immediate threat to life and health, how about foregoing the red lettering in favor of symbols that are more immediately recognizable by non-English speakers and illiterate people. The IAEA hadn’t invented its new sign yet, but the visual concepts were certainly around at the time, so let’s build on that. Also, why doesn’t the door to the compound come with the same radiation warning? Or any warning?
The buttons are a crap choice of control as well. They don’t show what the status of the remotely controlled thing is. So if Charles accidentally presses a button, and, say, raises a sliding wall that’s out of sight, how would he know? Labeled rocker switches help signal the state and would be a better choice.
But really, why would these things be controlled remotely? It be more secure to have two-handed momentary buttons on the walls, which would mean that a person would be there to visually verify that the wall was slid or the ramp retracted or whatever it is national security needed them to be.
There’s also the narrative question about why this remote control doesn’t come up later in the film when Unity is getting out of control. Couldn’t they have used this to open the fortification and go unplug the thing?
So all told, not a great bit of design, for either interaction or narrative, with lots of improvement for both.
Locking yourselves out and throwing away the key
At first glance, it seems weird that there should be interfaces in a compound that is meant to be uninhabited for most of its use. But this is the first launch of a new system, and these interfaces may be there in anticipation of the possibility that they would have to return inside after a failure. We can apologize these into believability.
But that doesn’t excuse the larger strategic question. Yes, we need defense systems to be secure. But that doesn’t mean sealing the processing and power systems for an untested AI away from all human access. The Control Problem is hard enough without humans actively limiting their own options. Which raises a narrative question: Why wasn’t there a segment of the film where the military is besieging this compound? Did Unity point a nuke at its own crunchy center? If not, siege! If so, well, maybe you can trick it into bombing itself. But I digress.
“And here is where we really screw our ability to recover from a mistake.”
Whether Unity should have had its plug pulled is the big philosophical question this movie does not want to ask, but I’ll save that for the big wrap up at the end.
The control token in Las Luchadras is a bracelet that slaps on and instantly renders its wearer an automaton, subject to the remote control.
Here’s something to note about this speculative technology. Orlak could have sold this, just this, to law enforcement around the world and made himself a very rich and powerful person. But the movie makes clear he is a mad engineer, not a mad businessperson, so we have to move on.
From Orlak’s point of view, getting the bracelet on its victim should be very easy. Fortunately, it does just that. Orlak can slap it on in a flick. But it’s also trivially easy for a bystander to remove, which seems like…a design oversight. It should work more like a handcuff, that requires a key to remove. It can’t look like a handcuff, of course, since Orlak wants it to go unnoticed. But in addition to the security, the handcuff function would enable the device to fit wrists of many sizes. As it is, it appears to be tailor-made to an individual.
As the diagram illustrates, not all wrists are made the same, and it would not help Orlak to have to carry around a sizing set when he hasn’t had time to secretly get the victim’s measurements.
Lastly, the audience might have benefited from seeing some visual connection between the bracelet and the remote, like a shared material that had an unusual color or glow, but Orlak would not want this connection since it could help someone identify him as the controller.
When Frito is driving Joe and Rita away from the cops, Joe happens to gesture with his hand above the car window, where a vending machine he happens to be passing spots the tattoo. Within seconds two harsh beeps sound in the car and a voice says, “You are harboring a fugitive named NOT SURE. Please, pull over and wait for the police to incarcerate your passenger.”
Frito’s car begins slowing down, and the dashboard screen shows a picture of Not Sure’s ID card and big red text zooming in a loop reading “PULL OVER”
The car interface has a column of buttons down the left reading:
NAV
WTF?
BEER
FART FAN
HOME
GIRLS
At the bottom is a square of icons: car, radiation, person, and the fourth is obscured by something in the foreground. Across the bottom is Frito’s car ID “FRITO’S F’N CAR” which appears to be a label for a system status of “EVERYTHING’S A-OK, BRO”, a button labeled CHECK INGN [sic], another labeled LOUDER, and a big green circle reading GO.
But the car doesn’t wait for him to pull over. With some tiny beeps it slows to a stop by itself. Frito says, “It turned off my battery!” Moments after they flee the car, it is converged upon by a ring of police officers with weapons loaded (including a rocket launcher pointed backward.)
Visual Design
Praise where it’s due: Zooming is the strongest visual attention-getting signals there is (symmetrical expansion is detected on the retina within 80 milliseconds!) and while I can’t find the source from which I learned it, I recall that blinking is somewhere in the top 5. Combining these with an audio signal means it’s hard to miss this critical signal. So that’s good.
In English: It’s comin’ right at us!
But then. Ugh. The fonts. The buttons on the chrome seem to be some free Blade Runner font knock off, the text reading “PULL OVER” is in some headachey clipped-corner freeware font that neither contrasts nor compliments the Blade Jogger font, or whatever it is. I can’t quite hold the system responsible for the font of the IPPA licence, but I just threw up a little into my Flaturin because of that rounded-top R.
Then there’s the bad-90s skeuomorphic, Bevel & Emboss buttons that might be defended for making the interactive parts apparent, except that this same button treatment is given to the label Frito’s F’n Car, which has no obvious reason why it would ever need to be pressed. It’s also used on the CHECK INGN and LOUDER buttons, taking their ADA-insulting contrast ratios and absolutely wrecking any readability.
I try not to second-guess designer’s intentions, but I’m pretty sure this is all deliberate. Part of the illustration of a world without much sense. Certainly no design sense.
In-Car Features
What about those features? NAV is pretty standard function, and having a HOME button is a useful shortcut. On current versions of Google Maps there’s an Explore Places Near You Function, which lists basic interests like Restaurants, Bars, and Events, and has a more menu with a big list of interests and services. It’s not a stretch to imagine that Frito has pressed GIRLS and BEER enough that it’s floated to the top nav.
That leaves only three “novel” buttons to think about: WTF, LOUDER, and FART FAN.
WTF?
If I have to guess, the WTF button is an all-purpose help button. Like a GM OnStar, but less well branded. Frito can press it and get connected to…well, I guess some idiot to see if they can help him with something. Not bad to have, though this probably should be higher in the visual hierarchy.
LOUDER
This bit of interface comedy is hilarious because, well, there’s no volume down affordance on the interface. Think of the “If it’s too loud, you’re too old” kind of idiocy. Of course, it could be that the media is on zero volume, and so it couldn’t be turned down any more, so the LOUDER button filled up the whole space, but…
The smarter convention is to leave the button in place and signal a disabled state, and
Given everything else about the interface, that’s giving the diegetic designer a WHOLE lot of credit. (And our real-world designer a pat on the back for subtle hilarity.)
FART FAN
This button is a little potty humor, and probably got a few snickers from anyone who caught it because amygdala, but I’m going to boldly say this is the most novel, least dumb thing about Frito’s F’n Car interface.
People fart. It stinks. Unless you have active charcoal filters under the fabric, you can be in for an unpleasant scramble to reclaim breathable air. The good news is that getting the airflow right to clear the car of the smell has, yes, been studied, well, if not by science, at least scientifically. The bad news is that it’s not a simple answer.
Your car’s built in extractor won’t be enough, so just cranking the A/C won’t cut it.
Rolling down windows in a moving aerodynamic car may not do the trick due to something called the boundary layer of air that “clings” to the surface of the car.
Rolling down windows in a less-aerodynamic car can be problematic because of the Helmholtz effect (the wub-wub-wub air pressure) and that makes this a risky tactic.
Opening a sunroof (if you have one) might be good, but pulls the stench up right past noses, so not ideal either.
The best strategy—according to that article and conversation amongst my less squeamish friends—is to crank the AC, then open the driver’s window a couple of inches, and then the rear passenger window half way.
But this generic strategy changes with each car, the weather (seriously, temperature matters, and you wouldn’t want to do this in heavy precipitation), and the skankness of the fart. This is all a LOT to manage when one’s eyes are meant to be on the road and you’re in an nauseated panic. Having the cabin air just refresh at the touch of one button is good for road safety.
If it’s so smart, then, why don’t we have Fart Fan panic buttons in our cars today?
I suspect car manufacturers don’t want the brand associations of having a button labeled FART FAN on their dashboards. But, IMHO, this sounds like a naming problem, not some intractable engineering problem. How about something obviously overpolite, like “Fast freshen”? I’m no longer in the travel and transportation business, but if you know someone at one of these companies, do the polite thing and share this with them.
Another way to deal with the problem, in the meantime.
So aside from the interface considerations, there are also some strategic ones to discuss with the remote kill switch, but that deserves it’s own post, next.
Another incidental interface is the pregnancy test that Joe finds in the garbage. We don’t see how the test is taken, which would be critical when considering its design. But we do see the results display in the orange light of Joe and Beth’s kitchen. It’s a cartoon baby with a rattle, swaying back and forth.
Sure it’s cute, but let’s note that the news of a pregnancy is not always good news. If the pregnancy is not welcome, the “Lucky you!” graphic is just going to rip her heart out. Much better is an unambiguous but neutral signal.
That said, Black Mirror is all about ripping our hearts out, so the cuteness of this interface is quite fitting to the world in which this appears. Narratively, it’s instantly recognizable as a pregnancy test, even to audience members who are unfamiliar with such products. It also sets up the following scene where Joe is super happy for the news, but Beth is upset that he’s seen it. So, while it’s awful for the real world; for the show, this is perfect.
Caveat #1: As a public service I have included some information about COVID-19 in the body of the post with a link to sources. These are called out the way this paragraph is, with a SARS-CoV-2 illustration floated on the left. I have done as much due diligence as one blogger can do to not spread disinformation, but keep in mind that our understanding of this disease and the context are changing rapidly. By the time you read this, facts may have changed. Follow links to sources to get the latest information. Do not rely solely on this post as a source. If you are reading this from the relative comfort of the future after COVID-19, feel free to skip these.
