The thanatorium is a speculative service for assisted suicide in Soylent Green. Suicide and death are not easy topics and I will do my best to address them seriously. Let me first take a moment to direct anyone who is considering or dealing with suicide to please stop reading this and talk to someone about it. I am unqualified to address—and this blog is not the place to work through—such issues.
There are four experiences to look at in the interface and service design of the Thanatorium: The patient, their beneficiaries, the usher to the beneficiaries, and the attendants to the patient. This post is about the attendants to the patient. Forewarning: This is the role we have the least information about. These Thanatorium personnel are there to assist the patient in their suicide, and deal with the body after the ceremony is complete.
The attendants have many goals and tasks to accomplish with each patient:
Help set the patient at ease so they complete the ceremony
Welcome the patient warmly
Assist them with tasks
Help them disrobe
Get them onto the gurney
Provide the hemlock
Set the patient in place for the cinerama experience
Press the gray buttons (which I interpret as ensuring medical monitoring, see below)
Set a liminal mood
Remove the clothes for donating and cup for cleaning
Leave the patient during the cinerama
Return to the body when the patient has passed
Usher the gurney through the portal
Nearly all of this is manual, with no speculative interfaces to speak of. A service design approach would look at this entire touchpoint, though. So, some quick notes.
Note their uniforms. Rather than the Guayabera shirt that the usher wears, the attendants wear vestments—white robes with goldenrod cuffs and cinctures around their waists. They even wear sandals to convey a sort of biblical, old-world holiness. It’s goofy and cheap, and kind of perfect.
Their manner is solemn, never speaking and performing their tasks with a sort of dance-like deliberateness. The behavior helps set off the space as liminal, somewhere not-quite like the world outside. No notes on the frontstage choreography.
The lighting begins a little flat, like overhead fluorescents in a school cafeteria. Maybe this is to give the patient a sense of certainty, of complete information about the room; but for my money the whole thing would seem more liminal with more dramatic lighting: A warm pool of light around the bed, maybe tiny amber incandescent bulbs flickering in a ring around the walls, like candles or stars.
Yes, closer to this.
There are some things we don’t get to see about the ceremony, like where the hemlock is stored and how it is presented to Sol, or how he gets up on a bed that’s above his waist, or what they do with his clothes. Or even—and this bit really bugs me—how the light changes from white to Sol’s requested orange at that moment. It’s not the usher, who is in the foyer about to intercept Thorn, and not the attendants, whose attention is on Sol. Maybe it’s on a timer, but that makes little sense. I really have to chalk it up to another movie-making error. Anyway, we’ll get to all this in the patient’s experience post, next.
For now let’s note that after the patient drinks the hemlock and they ease him back, we finally get to the one interface.
The ominous, inscrutable gray buttons…
Before departing the chamber, one of the attendants reaches down to a small metallic panel at the head of the bed. It consists of two square pushbuttons on the right, and a dial (or a plunger?) on the left.
The attendant presses and holds both of the buttons simultaneously for about three seconds. In the movie this attendant then gives the other a knowing glance, and they depart.
What the hell is this interface meant to be?
It’s quite unclear what state change this interface is meant to make, or why it needs to be a two-handed switch, when these sorts of things are mostly used for safety. My best guess is that since the drinking of the hemlock is the point of no return, and since the observation window is closed during that sequence so grief-stricken beneficiaries can’t interrupt; the two-handed switch is the silent signal from the attendants to the usher that everything is cool and they can open the observation window for final farewells. That’s low-confidence backworlding, though, since in the movie we know the usher is not present in the observation chamber at this time, but in the foyer of the thanatorium about to intercept Thorn. So, take this with a grain of salt.
But, if that’s the usual purpose, why have one panel with the two buttons? It’s a bit silly because they are close enough to be mashed by a single palm or even hip. It would make more sense if each attendant had their own button on each side of the bed, which they had to hold down. Have each button illuminate small green bulbs, and then jump-cut to the usher’s interface where two identical green bulbs labeled READY both illuminate. Then the usher can open the window and the beneficiary interface can switch to SPEAKING PERMITTED. This would make that weird interface moment make at least some sense.
Oh, and the dial? I have no idea. It’s unlabeled. Could be to control the bed height, or audio volume, or the brightness? Why one and not the other? There’s no way to tell and nothing makes a lot of sense given the rest of this scene. Provide your best guess in the comments, if you like. Otherwise my recommendation is to remove it.
Medical monitoring
One thing that seems to be missing the scene is some acknowledgment that the attendants are the ones to ensure that medical monitoring is operational, and do some troubleshooting if not. The monitoring is important, because the usher will await the clinical death signals before ending the cinerama and opening the observation window again for final viewing by the beneficiaries.
To help signal this, I recommend adding to the scene a quick shot of the surface of the bed before Sol lays down, showing inset silver disks, hinting at something like ECG electrodes, and then adding a panel at the head of the bed that an attendant can pull out to reveal the clinical death gauges described in the usher’s interface post.
These three, but with the dials in normal ranges for living patients.
The attendant can then close the panel, give the everything is in order look to the other, and the two of them depart for their break room, or jump seats, or watercooler; wherever they go for the interim.
This makes me realize the attendants just have to kind of hang out during the cinerama, and begs some sort of Rosencrantz and Guildenstern Are Dead philosophical dialog treatment. Please enter your drafts in the comments.
A final viewing
Once the patient passes, the attendants come in and push the gurney along its track into the portal. But this is for show, as the gurney is on a track, and after it leaves the theater to the “backstage,” it is pulled along by a mechanized track in the floor. So it could just be automated. But seeing the attendants moving it along gives the beneficiaries some last bit of theater that the body will be respectfully dealt with.
The thanatorium is a speculative device for assisted suicide in Soylent Green. Suicide and death are not easy topics and I will do my best to address them seriously. Let me first take a moment to direct anyone who is considering or dealing with suicide to please stop reading this and talk to someone about it. I am unqualified to address—and this blog is not the place to work through—such issues.
There are four experiences to look at in the interface and service design of the Thanatorium: The patient, their beneficiaries, the usher to the beneficiaries, and the attendants to the patient. This post is about the interface for the usher. This Thanatorium personnel is there as a stage manager of sorts, both to help the patient and the beneficiaries go where they need to go, ensure the beneficiaries do not do what they must not, and run the tech aspects of the ceremony.
The usher, left, ushing.
Note that—as I backworlded in the last post—these notes presume that the reason the beneficiaries are separated from the patients are to prevent them from trying to stop the event, and to minimize distractions during the cinerama display for gross biochemical reasons. Also recall that we’re having to go with a best-guess as to what the usual experience is, since we only see Thorn’s tardy thuggery in the film.
The usher’s tasks
Based on what we see in the film, the usher has a lot to do for each event…
Receive the patient’s preferences (music category, color, whatever other questions intake asked before we join that scene) from the intake personnel
Escort the patient to the “theater” and the beneficiaries to the observation room
Set the color of the light and the music to the patient’s preferences
Close the portal for the hemlock drinking
Open the portal for last farewells
Close the portal for the cinerama display
Start the cinerama display
Get help if the patient gets up or otherwise interrupts the ceremony
Wait for when the patient dies
Open the portal to view the body’s being shuttled away
Ensure the beneficiaries behave, answer any questions
Escort the beneficiaries back to the lobby
The interface barely touches on any of this
With all that in mind, we can see that this interface is woefully ill-equipped for any of his tasks. In the prior post I argue that the features for speaking to the patient—the speaker, the audio jack, and the SPEAKING PERMITTED indicator—should be separated from the usher’s stage manager functions. So we’re only going to pay attention in this post to the row of backlit rocker toggles labeled PORTAL, EFFECTS, CHAMBER 2, AUDIO, VISUAL, CHAMBER 1 and a little bit of the authorization key that looks like a square metal button in the screen cap above. And note I’m going to make suggestions that are appropriate to the early 1970s rather than use either modern real-world or speculative interface technologies.
First, that authorization key is pretty cool
The fact that it’s a featureless, long metal cuboid is so simple it feels sci-fi. Even the fact that its slot is unlabeled is good—it would help prevent a malicious or grief-panicked user from figuring out how to take control. You could even go one step further and have a hidden magnetic switch, so there’s not even a slot to clue in users. Production designer note, though, this means that the panel needs to be wood (or something non-magnetic) rather than a ferromagnetic metal. Aluminum, maybe, since it’s paramagnetic, but you also don’t want anything that can scratch or wear easily and give away the position of the secret spot.
This is a cabinet lock, but the same principle would apply.
But, the buttons don’t match the scene
The PORTAL button never changes state, though we see the portal open and close in the scene. AUDIO is dim though we hear the audio. Maybe dim equals on? No, because VISUAL is lit. There’s some gymnastics we could do to apologize for this, but Imma give up and just say it’s just a moviemaking error.
And they are poorly clustered
Why is CHAMBER 2 before CHAMBER 1? Why are the three AV buttons split up by CHAMBER 2? A more reasonable chunking of these would be PORTAL on its own, CHAMBER 1 & CHAMBER 2 together, and the remaining A/V buttons together. These groups should be separated to make them easier to identify and help avoid accidental activation (though the stakes here are pretty low.)
If we were just dealing with these 6 buttons, this might be a reasonable clustering. But, read on…
The PORTAL button is the wrong type and orientation
Look close at the screen shot and you’ll see that each button consists of three parts. A white, back-lit square which bears the label, and two black pushbuttons that act like rocker switches. That is, press the upper one in, and the lower one pops out. Press that popped-out lower one in, and the upper one pops out again. When the lower button is pressed in, the button is “on,” which you can tell because those are the only ones with the upper button popped out and the back light illuminated.
Rocker switches are good for things with two mutually exclusive states, like ON and OFF. The PORTAL button is the only one for which this makes unambiguous sense, with its two states being OPEN and CLOSED. But, we have to note that it is poorly mapped. The button has a vertical orientation, but the portal closes from right to left. It means the usher has to memorize which toggle state is open and which one is closed. It would more usable to have an inferrable affordance. Cheapest would be to turn the button sideways so it maps more clearly, but an even tighter mapping would be a slider mounted sideways with OPEN and CLOSED labels. I don’t think the backlit status indicator is necessary here because there’s already a giant signal of the state of the portal, and that’s the adjacent portal.
What do EFFECTS and CHAMBER even do?
What does the EFFECTS button do? I mean, if AUDIO and VISUAL have their own controls, what’s left? Lasers? A smoke machine? Happiness pheromones? (I’m getting The Cabin in the Woods vibes here.) Whatever it is, if there are multiple, they should have individual controls, in case the patient wants one but not the other, or if there’s any variability that needs controlling.
Also what do CHAMBER 1 and CHAMBER 2 do? It’s very poor labeling. What chambers do they refer to? Maybe the observation room is chamber 1 and the theater is chamber 2? If so, different names could save the usher’s having to memorize them. Also, what do these switches control? Lights? Door locks? We would need to know to really make design recommendations for these.
The AV controls are incomplete
Which takes us to AUDIO, and VISUAL. Each of these is missing something.
Sure, they might need ON/OFF controls as we see here. But how about a volume control to accommodate the hard-of-hearing and the sound-sensitive? How about a brightness control for the video? These could have an OFF state and replace the toggle switches.
We know from the movie itself that the service has offered Sol his choice of music genre. Where is the genre selector? This is a non-trivial problem since the number of genres is on the order of 1000. They probably don’t offer all of them, but at intake they do ask Sol his preference as an open-ended question, so it implies a huge selection. Radio band selectors would have made sense to audiences in the 1970s, and signal a huge number of options, but risk being “out of tune” and imply that it’s broadcast. So either have a small number of options with a 15° rotary switch (and rewrite the intake scene so Sol selects from a menu) or three 10-digit rotary switches with a “commit” momentary button, and have a small reference booklet hanging there.
I also want to believe that the theme of the video can be selected. Sol has chosen “nature” but you could imagine patients requesting for their end-of-life ceremony something else like “family,” “celestial,” “food” (given the diegesis, this should be first) or even “religious” (with a different one for each of the world’s twelve major religions). So it would make sense to have a video theme selector as well, say, on the order of 20 options. That could be a 15° rotary switch. Labeling gets tough, but it could just be numbers with an index label to the side.
I’m going to presume that they never need scrubbing controls (REWIND or FAST FORWARD) for the AV. The cinerama plays through once and stops. Sudden rewinding or fast forwarding would be jarring for the patient and ruin the immersion. Have a play button that remains depressed while the cinerama is ongoing. But if the patient passes more quickly than expected, a RESET button would make sense. So would a clock or a countdown timer, since Sol had confirmed at intake that it would be at least 20 minutes, and to let the usher know how much time they have left to get those neurotransmitter numbers up up up.
Some controls are straight up missing
How does an usher set the lights according to the patient’s preferences? They ask at intake, and we see Sol’s face washed with a soothing amber color once the attendants leave, so there should be a color selector. Three RGB slide potentiometers would provide perfect control, but I doubt anyone would quibble that the green they’d asked for was #009440 and not #96b300, so you could go with a selector. The XKCD color survey results show that there are on the order of about 30 colors, so something similar to the video-theme selector above would work, with a brightness potentiometer to the side.
The patient experience is a bit of a show, so to signal its beginning and end, there should be lighting controls for the usher to dim and raise the lights, like in a theater. So let’s add those.
Also, the usher has a minor medical task to accomplish: Monitoring the health of the patient to know when they’ve passed. The three metrics for clinical death are a cessation of all three of…
breath
blood flow
brain activity
…so there should be indicators for each of these. As discussed in the medical chapter of the book, this is ideally a display of values over time, but in the resource-poor and elecromechanical world of Soylent Green, it might have to be a collection of gauges, with an indicator bulb near the zero for when activity has stopped. A final, larger indicator bulb should light when all three gauges are still. To really underscore the morbidness of this interface, all those indicators should be green.
If you buy my backworlding, i.e. that part of the point of preventing interruptions is to maximize the dopamine and serotonin being released into the patient’s body, there should also be status indicators showing the level of these neurotransmitters in the patient’s bloodstream. They can be the same style of gauges, but I’d add a hand drawn arrow to some point along the chapter ring that reads “quota.” Those indicators should be larger than the clinical death indicators to match their importance to Soylent’s purposes.
Lastly, thinking of Thorn’s attack, the usher should have a panic button to summon help if the patient or the beneficiaries are getting violent (especially once they discover they’re locked in.) This should be hidden under the panel so it can be depressed secretly.
Where should this panel go?
As described in the beneficiaries post we’re going to leave the communication interface just below the portal, where they are now for those fleeting moments when they can wish the patient goodbye.
And there’s no need to put the usher’s controls under the nose of the beneficiaries. (In fact with the medical monitoring it would be kind of cruel.) So let the usher have a lectern beside the door, in a dim pool of light, and mount the controls to the reading top. (Also give them a stool to rest on, have we learned nothing?) Turn the lectern such that the interface is not visible to beneficiaries in the room. This lets the usher remain respectfully out of the center of attention, but in a place where they can keep an eye on both the patient when the portal is open, and the beneficiaries throughout.
Looks cheap? Perfect.
In total, the lectern panel would look something like this…
The Usher escorts Thorn into the room. Thorn rushes to the portal. The usher steps behind a lectern near the door.
Usher
It’s truly a shame you missed the overture.
The Usher slides a switch on the lectern panel, and the portal closes.
Thorn
I want to see him.
Usher, looking down at his interface
That is prohibited during the ceremony.
Worm’s eye view. Thorn takes a few steps toward him and knocks the lectern to the ground. It falls with its interface in the foreground. In the background, we see Thorn slam the usher against the wall.
Thorn
Well I can assure you, open that damned thing right now, or I swear to God you’ll die before he does!
Usher
OK, OK!
The usher falls to his hands and knees and we see him slide the switch to open the portal. Thorn steps back to it, and the usher gets on his feet to right the lectern
“Tunnel in the Sky” is the name of a 1955 Robert Heinlein novel that has nothing to do with this post. It is also the title of the following illustration by Muscovite digital artist Vladimir Manyukhin, which also has nothing to do with this post, but is gorgeous and evocative, and included here solely for visual interest.
Instead, this post is about the piloting display of the same name, and written specifically to sci-fi interface designers.
Last week in reviewing the spinners in Blade Runner, I included mention and a passing critique of the tunnel-in-the-sky display that sits in front of the pilot. While publishing, I realized that I’d seen this a handful of other times in sci-fi, and so I decided to do more focused (read: Internet) research about it. Turns out it’s a real thing, and it’s been studied and refined a lot over the past 60 years, and there are some important details to getting one right.
Though I looked at a lot of sources for this article, I must give a shout-out to Max Mulder of TU Delft. (Hallo, TU Delft!) Mulder’s PhD thesis paper from 1999 on the subject is truly a marvel of research and analysis, and it pulls in one of my favorite nerd topics: Cybernetics. Throughout this post I rely heavily on his paper, and you could go down many worse rabbit holes than cybernetics. n.b., it is not about cyborgs. Per se. Thank you, Max.
I’m going to breeze through the history, issues, and elements from the perspective of sci-fi interfaces, and then return to the three examples in the survey. If you want to go really in depth on the topic (and encounter awesome words like “psychophysics” and “egomotion” in their natural habitat), Mulder’s paper is available online for free from researchgate.net: “Cybernetics of Tunnel-in-the-Sky Displays.”
What the heck is it?
A tunnel-in-the-sky display assists pilots, helping them know where their aircraft is in relation to an ideal flight path. It consists of a set of similar shapes projected out into 3D space, circumscribing the ideal path. The pilot monitors their aircraft’s trajectory through this tunnel, and makes course corrections as they fly to keep themselves near its center.
This example comes from Michael P. Snow, as part of his “Flight Display Integration” paper, also on researchgate.net.
Please note that throughout this post, I will spell out the lengthy phrase “tunnel-in-the-sky” because the acronym is pointlessly distracting.
Quick History
In 1973, Volkmar Wilckens was a research engineer and experimental test pilot for the German Research and Testing Institute for Aerospace (now called the German Aerospace Center). He was doing a lot of thinking about flight safety in all-weather conditions, and came up with an idea. In his paper “Improvements In Pilot/Aircraft-Integration by Advanced Contact Analog Displays,” he sort of says, “Hey, it’s hard to put all the information from all the instruments together in your head and use that to fly, especially when you’re stressed out and flying conditions are crap. What if we took that data and rolled it up into a single easy-to-use display?” Figure 6 is his comp of just such a system. It was tested thoroughly in simulators and shown to improve pilot performance by making the key information (attitude, flight-path and position) perceivable rather than readable. It also enabled the pilot greater agency, by not having them just follow rules after instrument readings, but empowering them to navigate multiple variables within parameters to stay on target.
In Wilckens’ Fig. 6, above, you can see the basics of what would wind up on sci-fi screens decades later: shapes repeated into 3D space ahead of the aircraft to give the pilot a sense of an ideal path through the air. Stay in the tunnel and keep the plane safe.
Mulder notes that the next landmark developments come from the work of Arthur Grunwald & S. J. Merhav between 1976–1978. Their research illustrates the importance of augmenting the display and of including a preview of the aircraft in the display. They called this preview the Flight Path Predictor, or FPS. I’ve also seen it called the birdie in more modern papers, which is a lot more charming. It’s that plus symbol in the Grunwald illustration, below. Later in 1984, Grunwald also showed that a heads-up-display increased precision adhering to a curved path. So, HUDs good.
n.b. This is Mulder’s representation of Grunwald’s display format.
I have also seen lots of examples of—but cannot find the research provenance for—tools for helping the pilot stay centered, such as a “ghost” reticle at the center of each frame, or alternately brackets around the FPP, called the Flight Director Box, that the pilot can align to the corners of the frames. (I’ll just reference the brackets. Gestalt be damned!) The value of the birdie combined with the brackets seems very great, so though I can’t cite their inventor, and it wasn’t in Mulder’s thesis, I’ll include them as canon.
The takeaway from the history is really that these displays have a rich and studied history. The pattern has a high confidence.
Elements of an archetypical tunnel-in-the-sky display
There are lots of nuances that have been studied for these displays. Take for example the effect that angling the frames have on pilot banking, and the perfect time offset to nudge pilot behavior closer to ideal banking. For the purposes of sci-fi interfaces, however, we can reduce the critical components of the real world pattern down to four.
Square shapes (called frames) extending into the distance that describe an ideal path through space
The frame should be about five times the width of the craft. (The birdie you see below is not proportional and I don’t think it’s standard that they are.)
The distances between frames will change with speed, but be set such that the pilot encounters a new one every three seconds.
The frames should adopt perspective as if they were in the world, being perpendicular to the flight path. They should not face the display.
The frames should tilt, or bank, on curves.
The tunnel only needs to extend so far, about 20 seconds ahead in the flight path. This makes for about 6 frames visible at a time.
An aircraft reference symbol or Flight Path Predictor Symbol (FPS, or “birdie”) that predicts where the plane will be when it meets the position of the nearest frame. It can appear off-facing in relation to the cockpit.
These are often rendered as two L shapes turned base-to-base with some space between them. (See one such symbol in the Snow example above.)
Sometimes (and more intuitively, imho) as a circle with short lines extending out the sides and the top. Like a cartoon butt of a plane. (See below.)
Contour lines connect matching corners across frames
A horizon line
This comp illustrates those critical features.
There are of course lots of other bits of information that a pilot needs. Altitude and speed, for example. If you’re feeling ambitious, and want more than those four, there are other details directly related to steering that may help a pilot.
Degree-of-vertical-deviation indicator at a side edge
Degree-of-horizontal-deviation indicator at the top edge
Center-of-frame indicator, such as a reticle, appearing in the upcoming frame
A path predictor
Some sense of objects in the environment: If the display is a heads-up display, this can be a live view. If it is a separate screen, some stylized representation what the pilot would see if the display was superimposed onto their view.
What the risk is when off path: Just fuel? Passenger comfort? This is most important if that risk is imminent (collision with another craft, mountain) but then we’re starting to get agentive and I said we wouldn’t go there, so *crumbles up paper, tosses it*.
I haven’t seen a study showing efficacy of color and shading and line scale to provide additional cues, but look closely at that comp and you’ll see…
The background has been level-adjusted to increase contrast with the heads-up display
A dark outline around the white birdie and brackets to help visually distinguish them from the green lines and the clouds
A shadow under the birdie and brackets onto the frames and contours as an additional signal of 3D position
Contour lines diminishing in size as they extend into the distance, adding an additional perspective cue and limiting the amount of contour to the 20 second extents.
Some other interface elements added.
What can you play with when designing one in sci-fi?
Everything, of course. Signaling future-ness means extending known patterns, and sci-fi doesn’t answer to usability. Extend for story, extend for spectacle, extend for overwhelmedness. You know your job better than me. But if you want to keep a foot in believability, you should understand the point of each thing as you modify it and try not to lose that.
Each frame serves as a mini-game, challenging the pilot to meet its center. Once that frame passes, that game is done and the next one is the new goal. Frames describe the near term. Having corners to the frame shape helps convey banking better. Circles would hide banking.
Contour lines, if well designed, help describe the overall path and disambiguate the stack of frames. (As does lighting and shading and careful visual design, see above.) Contour lines convey the shape of the overall path and help guide steering between frames. Kind of like how you’d need to see the whole curve before drifitng your car through one, the contour lines help the pilot plan for the near future.
The birdie and brackets are what a pilot uses to know how close to the center they are. The birdie needs a center point. The brackets need to match the corners of the frame. Without these, it’s easier to drift off center.
A horizon line provides feedback for when the plane is banked.
THIS BAD: You can kill the sense of the display by altering (or in this case, omitting) too much.
Since I mentioned that each frame acts as a mini-game, a word of caution: Just as you should be skeptical when looking to sci-fi, you should be skeptical when looking to games for their interfaces. The simulator which is most known for accuracy (Microsoft Flight Simulator) doesn’t appear to have a tunnel-in-the-sky display, and other categories of games may not be optimizing for usability as much as just plain fun, with the risk of crashing your virtual craft just being part of the risk. That’s not an acceptable outcome in real-world piloting. So, be cautious considering game interfaces as models for this, either.
This clip of stall-testing in the forthcoming MSFS2020 still doesn’t appear to show one.
So now let’s look at the three examples of sci-fi tunnel-in-the-sky displays in chronological order of release, and see how they fare.
Three examples from sci-fi
So with those ideal components in mind, let’s look back at those three examples in the survey.
Quick aside on the Blade Runner interface: The spike at the top and the bottom of the frame help in straight tunnels to serve as a horizontal degree-of-deviation indicator. It would not help as much in curved tunnels, and is missing a matching vertical degree-of-deviation indicator. Unless that’s handled automatically, like a car on a road, its absence is notable.
Starship Troopers (1997) We only get 15 frames of this interface in Starship Troopers, as Ibanez pilots the escape shuttle to the surface of Planet P. It is very jarring to see as a repeating gif, so accept this still image instead.
Some obvious things we see missing from all of them are the birdie, the box, and the contour lines. Why is this? My guess is that the computational power in the 1976 was not enough to manage those extra lines, and Ridley Scott just went with the frames. Then, once the trope had been established in a blockbuster, designers just kept repeating the trope rather than looking to see how it worked in the real world, or having the time to work through the interaction logic. So let me say:
Without the birdie and box, the pilot has far too much leeway to make mistakes. And in sci-fi contexts, where the tunnel-in-the-sky display is shown mostly during critical ship maneuvers, their absence is glaring.
Also the lack of contour lines might not seem as important, since the screens typically aren’t shown for very long, but when they twist in crazy ways they should help signal the difficulty of the task ahead of the pilot very quickly.
Note that sci-fi will almost certainly encounter problems that real-world researchers will not have needed to consider, and so there’s plenty of room for imagination and additional design. Imagine helping a pilot…
Navigating the weird spacetime around a singularity
Bouncing close to a supernova while in hyperspace
Dodging chunks of spaceship, the bodies of your fallen comrades, and rising plasma bombs as you pilot shuttlecraft to safety on the planet below
AI on the ships that can predict complex flight paths and even modify them in real time, and even assist with it all
Needing to have the tunnel be occluded by objects visible in a heads up display, such as when a pilot is maneuvering amongst an impossibly-dense asteroid field.
…to name a few off my head. These things don’t happen in the real world, so would be novel design challenges for the sci-fi interface designer.
So, now we have a deeper basis for discussing, critiquing, and designing sci-fi tunnel-in-the-sky displays. If you are an aeronautic engineer, and have some more detail, let me hear it! I’d love for this to be a good general reference for sci-fi interface designers.
If you are a fan, and can provide other examples in the comments, it would be great to see other ones to compare.
Happy flying, and see you back in Blade Runner in the next post.
Distinguishing replicants from humans is a tricky business. Since they are indistinguishable biologically, it requires an empathy test, during which the subject hears empathy-eliciting scenarios and watched carefully for telltale signs such as, “capillary dilation—the so-called blush response…fluctuation of the pupil…involuntary dilation of the iris.” To aid the blade runner in this examination, they use a portable machine called the Voight-Kampff machine, named, presumably, for its inventors.
The device is the size of a thick laptop computer, and rests flat on the table between the blade runner and subject. When the blade runner prepares the machine for the test, they turn it on, and a small adjustable armature rises from the machine, the end of which is an intricate piece of hardware, housing a powerful camera, glowing red.
The blade runner trains this camera on one of the subject’s eyes. Then, while reading from the playbook book of scenarios, they keep watch on a large monitor, which shows an magnified image of the subject’s eye. (Ostensibly, anyway. More on this below.) A small bellows on the subject’s side of the machine raises and lowers. On the blade runner’s side of the machine, a row of lights reflect the volume of the subject’s speech. Three square, white buttons sit to the right of the main monitor. In Leon’s test we see Holden press the leftmost of the three, and the iris in the monitor becomes brighter, illuminated from some unseen light source. The purpose of the other two square buttons is unknown. Two smaller monochrome monitors sit to the left of the main monitor, showing moving but otherwise inscrutable forms of information.
In theory, the system allows the blade runner to more easily watch for the minute telltale changes in the eye and blush response, while keeping a comfortable social distance from the subject. Substandard responses reveal a lack of empathy and thereby a high probability that the subject is a replicant. Simple! But on review, it’s shit. I know this is going to upset fans, so let me enumerate the reasons, and then propose a better solution.
-2. Wouldn’t a genetic test make more sense?
If the replicants are genetically engineered for short lives, wouldn’t a genetic test make more sense? Take a drop of blood and look for markers of incredibly short telomeres or something.
-1. Wouldn’t an fMRI make more sense?
An fMRI would reveal empathic responses in the inferior frontal gyrus, or cognitive responses in the ventromedial prefrontal gyrus. (The brain structures responsible for these responses.) Certinaly more expensive, but more certain.
0. Wouldn’t a metal detector make more sense?
If you are testing employees to detect which ones are the murdery ones and which ones aren’t, you might want to test whether they are bringing a tool of murder with them. Because once they’re found out, they might want to murder you. This scene should be rewritten such that Leon leaps across the desk and strangles Holden, IMHO. It would make him, and other blade runners, seem much more feral and unpredictable.
(OK, those aren’t interface issues but seriously wtf. Onward.)
1. Labels, people
Controls needs labels. Especially when the buttons have no natural affordance and the costs of experimentation to discover the function are high. Remembering the functions of unlabeled controls adds to the cognitive load for a user who should be focusing on the person across the table. At least an illuminated button helps signal the state, so that, at least, is something.
2. It should be less intimidating
The physical design is quite intimidating: The way it puts a barrier in between the blade runner and subject. The fact that all the displays point away from the subject. The weird intricacy of the camera, its ominous HAL-like red glow. Regular readers may note that the eyepiece is red-on-black and pointy. That is to say, it is aposematic. That is to say, it looks evil. That is to say, intimidating.
I’m no emotion-scientist, but I’m pretty sure that if you’re testing for empathy, you don’t want to complicate things by introducing intimidation into the equation. Yes, yes, yes, the machine works by making the subject feel like they have to defend themselves from the accusations in the ethical dilemmas, but that stress should come from the content, not the machine.
2a. Holden should be less intimidating and not tip his hand
While we’re on this point, let me add that Holden should be less intimidating, too. When Holden tells Leon that a tortoise and a turtle are the same thing, (Narrator: They aren’t) he happens to glance down at the machine. At that moment, Leon says, “I’ve never seen a turtle,” a light shines on the pupil and the iris contracts. Holden sees this and then gets all “ok, replicant” and becomes hostile toward Leon.
In case it needs saying: If you are trying to tell whether the person across from you is a murderous replicant, and you suddenly think the answer is yes, you do not tip your hand and let them know what you know. Because they will no longer have a reason to hide their murderyness. Because they will murder you, and then escape, to murder again. That’s like, blade runner 101, HOLDEN.
3. It should display history
The glance moment points out another flaw in the interface. Holden happens to be looking down at the machine at that moment. If he wasn’t paying attention, he would have missed the signal. The machine needs to display the interview over time, and draw his attention to troublesome moments. That way, when his attention returns to the machine, he can see that something important happened, even if it’s not happening now, and tell at a glance what the thing was.
4. It should track the subject’s eyes
Holden asks Leon to stay very still. But people are bound to involuntarily move as their attention drifts to the content of the empathy dilemmas. Are we going to add noncompliance-guilt to the list of emotional complications? Use visual recognition algorithms and high-resolution cameras to just track the subject’s eyes no matter how they shift in their seat.
5. Really? A bellows?
The bellows doesn’t make much sense either. I don’t believe it could, at the distance it sits from the subject, help detect “capillary dilation” or “ophthalmological measurements”. But it’s certainly creepy and Terry Gilliam-esque. It adds to the pointless intimidation.
6. It should show the actual subject’s eye
The eye color that appears on the monitor (hazel) matches neither Leon’s (a striking blue) or Rachel’s (a rich brown). Hat tip to Typeset in the Future for this observation. His is a great review.
7. It should visualize things in ways that make it easy to detect differences in key measurements
Even if the inky, dancing black blob is meant to convey some sort of information, the shape is too organic for anyone to make meaningful readings from it. Like seriously, what is this meant to convey?
The spectrograph to the left looks a little more convincing, but it still requires the blade runner to do all the work of recognizing when things are out of expected ranges.
8. The machine should, you know, help them
The machine asks its blade runner to do a lot of work to use it. This is visual work and memory work and even work estimating when things are out of norms. But this is all something the machine could help them with. Fortunately, this is a tractable problem, using the mighty powers of logic and design.
Pupillary diameter
People are notoriously bad at estimating the sizes of things by sight. Computers, however, are good at it. Help the blade runner by providing a measurement of the thing they are watching for: pupillary diameter. (n.b. The script speaks of both iris constriction and pupillary diameter, but these are the same thing.) Keep it convincing and looking cool by having this be an overlay on the live video of the subject’s eye.
So now there’s some precision to work with. But as noted above, we don’t want to burden the user’s memory with having to remember stuff, and we don’t want them to just be glued to the screen, hoping they don’t miss something important. People are terrible at vigilance tasks. Computers are great at them. The machine should track and display the information from the whole session.
Note that the display illustrates radius, but displays diameter. That buys some efficiencies in the final interface.
Now, with the data-over-time, the user can glance to see what’s been happening and a precise comparison of that measurement over time. But, tracking in detail, we quickly run out of screen real estate. So let’s break the display into increments with differing scales.
There may be more useful increments, but microseconds and seconds feel pretty convincing, with the leftmost column compressing gradually over time to show everything from the beginning of the interview. Now the user has a whole picture to look at. But this still burdens them into noticing when these measurements are out of normal human ranges. So, let’s plot the threshold, and note when measurements fall outside of that. In this case, it feels right that replicants display less that normal pupillary dilation, so it’s a lower-boundary threshold. The interface should highlight when the measurement dips below this.
Blush
I think that covers everything for the pupillary diameter. The other measurement mentioned in the dialogue is capillary dilation of the face, or the “so-called blush response.” As we did for pupillary diameter, let’s also show a measurement of the subject’s skin temperature over time as a line chart. (You might think skin color is a more natural measurement, but for replicants with a darker skin tone than our two pasty examples Leon and Rachel, temperature via infrared is a more reliable metric.) For visual interest, let’s show thumbnails from the video. We can augment the image with degree-of-blush. Reduce the image to high contrast grayscale, use visual recognition to isolate the face, and then provide an overlay to the face that illustrates the degree of blush.
But again, we’re not just looking for blush changes. No, we’re looking for blush compared to human norms for the test. It would look different if we were looking for more blushing in our subject than humans, but since the replicants are less empathetic than humans, we would want to compare and highlight measurements below a threshold. In the thumbnails, the background can be colored to show the median for expected norms, to make comparisons to the face easy. (Shown in the drawing to the right, below.) If the face looks too pale compared to the norm, that’s an indication that we might be looking at a replicant. Or a psychopath.
So now we have solid displays that help the blade runner detect pupillary diameter and blush over time. But it’s not that any diameter changes or blushing is bad. The idea is to detect whether the subject has less of a reaction than norms to what the blade runner is saying. The display should be annotating what the blade runner has said at each moment in time. And since human psychology is a complex thing, it should also track video of the blade runner’s expressions as well, since, as we see above, not all blade runners are able to maintain a poker face. HOLDEN.
Anyway, we can use the same thumbnail display of the face, without augmentation. Below that we can display the waveform (because they look cool), and speech-to-text the words that are being spoken. To ensure that the blade runner’s administration of the text is not unduly influencing the results, let’s add an overlay to the ideal intonation targets. Despite evidence in the film, let’s presume Holden is a trained professional, and he does not stray from those targets, so let’s skip designing the highlight and recourse-for-infraction for now.
Finally, since they’re working from a structured script, we can provide a “chapter” marker at the bottom for easy reference later.
Now we can put it all together, and it looks like this. One last thing we can do to help the blade runner is to highlight when all the signals indicate replicant-ness at once. This signal can’t be too much, or replicants being tested would know from the light on the blade runner’s face when their jig is up, and try to flee. Or murder. HOLDEN.
For this comp, I added a gray overlay to the column where pupillary and blush responses both indicated trouble. A visual designer would find some more elegant treatment.
If we were redesigning this from scratch, we could specify a wide display to accomodate this width. But if we are trying to squeeze this display into the existing prop from the movie, here’s how we could do it.
Note the added labels for the white squares. I picked some labels that would make sense in the context. “Calibrate” and “record” should be obvious. The idea behind “mark” is an easy button for the blade runner to press when they see something that looks weird, like when doctors manually annotate cardiograph output.
Lying to Leon
There’s one more thing we can add to the machine that would help out, and that’s a display for the subject. Recall the machine is meant to test for replicant-ness, which happens to equate to murdery-ness. A positive result from the machine needs to be handled carefully so what happens to Holden in the movie doesn’t happen. I mentioned making the positive-overlay subtle above, but we can also make a placebo display on the subject’s side of the interface.
The visual hierarchy of this should make the subject feel like its purpose is to help them, but the real purpose is to make them think that everything’s fine. Given the script, I’d say a teleprompt of the empathy dilemma should take up the majority of this display. Oh, they think, this is to help me understand what’s being said, like a closed caption. Below the teleprompt, at a much smaller scale, a bar at the bottom is the real point.
On the left of this bar, a live waveform of the audio in the room helps the subject know that the machine is testing things live. In the middle, we can put one of those bouncy fuiget displays that clutters so many sci-fi interfaces. It’s there to be inscrutable, but convince the subject that the machine is really sophisticated. (Hey, a diegetic fuiget!) Lastly—and this is the important part—An area shows that everything is “within range.” This tells the subject that they can be at ease. This is good for the human subject, because they know they’re innocent. And if it’s a replicant subject, this false comfort protects the blade runner from sudden murder. This test might flicker or change occasionally to something ambiguous like “at range,” to convey that it is responding to real world input, but it would never change to something incriminating.
This way, once the blade runner has the data to confirm that the subject is a replicant, they can continue to the end of the module as if everything was normal, thank the replicant for their time, and let them leave the room believing they passed the test. Then the results can be sent to the precinct and authorizations returned so retirement can be planned with the added benefit of the element of surprise.
OK
Look, I’m sad about this, too. The Voight-Kampff machine is cool. It fits very well within the art direction of the Blade Runner universe. This coolness burned the machine into my memory when I saw this film the first dozen times, but despite that, it just doesn’t stand up to inspection. It’s not hopeless, but does need a lot of thinkwork and design to make it really fit to task, and convincing to us in the audience.
To provide the Victim Cards to the Robot Asesino, Orlak inserts it into an open slot in the robot’s chest, which then illuminates, confirming that the instructions have been received.
There is, I must admit, a sort of lovely, morbid poetry to a cardiogram being inserted into a slot where the robot heart would be to give the robot instructions to end the beating of the human heart described in the cardiogram. And we don’t see a lot of poetry in sci-fi interface designs. So, props for that.
The illumination is a nice bit of feedback, but I think it could convey the information in more useful and cinegenic ways.
In this new scenario…
Orlak has the robot pull back its coat
The chamfered slot is illuminated, signaling “card goes here.”
As Orlak inserts the target card, the slot light dims as the chest-cavity light brightens, signaling “I have the card.”
After a moment, the chest-cavity light turns blood red, signaling confirmation of the victim and the new dastardly mission.
When the robot returns to Orlak after completing a mission, the red light would dim as the slot light illuminates again, signaling that it is ready for its next mission.
These changes improve the interface by first drawing the user’s locus of attention exactly where it needs to go, and then distinguishing the internal system states as they happen. It would also work for the audience, who understands by association that red means danger.
The shape of the slot is pretty good for its base usability. It has clear affordances with its placement, orientation, and metallic lining. There’s plenty of room to insert the target card. It might benefit from a fillet or chamfer for the slot, to help avoid accidentally crumpling the paper cards when they are aimed poorly.
In addition to the tactical questions of illumination and shape of the slot, I have a few strategic questions.
There is no authorization in evidence. Can just anyone specify a target? Why doesn’t Gaby use her luchadora powers to Spin-A-Roonie a target card with Orlak’s face on it and let the robot save the day? Maybe the robot has a whitelist of heartbeats, and would fight to resist anyone else, but that’s just me making stuff up.
Also I’m not sure why the card stays in the robot. That leaves a discoverable paper trail of its crimes, perfect for a Scooby to hand over to the federales. Maybe the robot has some incinerator or shredder inside? If not, it would be better from Orlak’s perspective to design it as an insert-and-hold slot, which would in turn require a redesign of the card to have some obvious spot to hold it, and a bump-in on the slot to make way for fingers. Then he could remove the incriminating evidence and destroy it himself and not worry whether the robot’s paper shredder was working or not.
Another problem is that, since the robot doesn’t talk, it would be difficult to find out who its current target is at any given time. Since anyone can supply a target, Orlak can’t just rely on his memory to be certain. If the card was going to stay inside, it would be better to have it displayed so it’s easy to check.
How would Orlak cancel a target?
It is unclear how Orlak specifies whether the target is to be kidnapped or killed even though some are kidnapped and some are killed.
It’s also unclear about how Orlak might rescind or change an order once given.
It is also unclear how the assassin finds its target. Does it have internal maps with addresses? Or does it have unbelievably good hearing that can listen to every sound nearby, isolate the particular heartbeat in question, and just head in that direction, destroying any walls it encounters? Or can it reasonably navigate human cities and interiors to maintain its disguise? Because that would be some amazing technology for 1969. This last is admittedly not an interface question, but a backworlding question for believability.
So there’s a lot missing from the interface.
It’s the robot assassin designer’s job to not just tick a box to tell themselves that they have provided feedback, but to push through the scenarios of use to understand in detail how to convey to the evil scientist what’s happening with his murderous intent.
It’s Halloween, as if the news of the past week were not scary enough. Pipe bombs to Democratic leaders. The largest massacre of Jewish people in on American soil in history. The murder of two black senior citizens by a white supremacist in Kentucky. Now let’s add to it with this nightmare scene from Idiocracy. Full disclosure: We’re covering technology as old as civilization here, so there won’t be any screen interfaces.
Joe is wheeled into the courtroom in a cage. There is a large gallery there, all of whom are booing him. One throws his milkshake at the accused. Others throw trash. The narrator says, “Joe was arrested for not paying his hospital bill and not having his IPP tattoo. He would soon discover that in the future, Justice was not only blind, but had become rather retarded as well.”
Joe is let out of his cage. The judge, identified by his name plate as The Honorable Hector “The Hangman,” stands at his bench in a spotlight in front of a wall of logos, grinning in anticipation at a new victim. He slams a massive gavel and shouts at the booing crowd, “Listen up! Now. I’m fixin’ to commensurate this trial here. [All of this is sic.] We gon’ see if we can’t come up with a verdict up in here. Now. Since y’all say y’ain’t got no money, we have proprietarily obtained you one of them court-appointed lawyers. So, put your hands together to give it up for Frito Pendejo!”
Frito, wearing a long-sleeve t-shirt with “ATTORNEY AT LAW” running down the sleeve, sits and looks at a paper on the counsel table, saying “Says here you, uh, robbed a hospital?! Why’d you do that?” Joe says, “But I’m not guilty!” Frito replies, “That’s not what the other lawyer said.”
When the trial officially starts, the judge says, “Shut up. Shut up! Now, prosecutor. Why you think he done it?” The prosecutor stands up and says, “K. Number one, your honor. Just look at him.” Most everyone in the courtroom, including, Frito, laughs at this. Frito stands up and says, “He talks like a fag, too!” When more laughter dies down, the prospector continues, saying, “We’ve got all this, like, evidence, of how, like, this guy didn’t even pay at the hospital and I heard that he doesn’t even have his tattoo.” There are collective gasps. He continues, “I know! And I’m all, ‘You gotta be shitting me.’ But check this out, man, judge should be like, ‘Guilty!’ Peace.” The gallery erupts with applause and cheers. There is one guy wearing a helmet with a camera mounted to the top and he takes it all in dumbly.
When Frito stands to raise an objection, he says, “Your honor, I object…that this guy, also broke my apartment and shit.” There are gasps from the gallery, and Frito feels emboldened. “Yeah! And you know what else, I object that he’s not going to have any money to pay me after he pays back all the money he stole from the hospital!” This last bit is addressed to the gallery. They shout in anger. Frito finishes, saying, “And I object. I OBJECT that he interrupted me while I was watching, OW, MY BALLS! THAT IS NOT OK! I REST MY CASE.”
Joe stands and says, “Your Honor, I’m pretty sure we have a mistrial, here, sir.” Hector looks confused at this statement. Frito gets hostile and says, “I’m going to mistrial my foot up your ass if you don’t shut up!”
Joe ignores him and says, “Please listen!” The prosecuting attorney simply mocks him, “Please listen!” Hector laughs. The lawyers high five each other.
The narrator says, “Joe stated his case, logically and passionately. But his perceived effeminate voice only drew big gales of stupid laughter. Without adequate legal representation, Joe was given a stiff sentence.”
So…this nightmare
For most Americans, the drafting and adjudication of laws feels like something that happens “out there.” But upholding the law is something that, through jury duty or being part of a court case, is something most citizens directly experience some time in their lives. So it may seem like familiar, everyday stuff.
Wait. I know this.
But take that less trodden path to ask why it is the way it is, and you’ll eventually find yourself at the foundations of civilization and in the throes of philosophy. A key premise and promise of civilized society is having a fair and rational place for citizens to air grievances, and institutions that make and enforce just laws. In practice it’s far from perfect, but I’d bet most folks agree it’s better than the alternatives of lawlessness, personal violence, revenge, tribal feuds, and vigilantism.
Laws and courts are institutions that are so old and foundational, it’s hard to remember that:
They were, in fact, invented. There was a time before them.
There were reasons for the way they were designed.
Their design, like all designs, aren’t perfect, and involve some trade offs.
They evolve over time.
There are alternate designs to consider.
Idiocracy illustrates how important its norms are, and how tragic it can be when those norms are lost, and everyone involved is a fucking idiot. It’s not exactly monster rampage, body horror, or torture porn, but this scene is as scary a thing as you could ask for Halloween.
Violence as a means
Frito threatens violence a lot. So do a lot of the people in Idiocracy. I’m not even sure if they mean it, but they think that threatening or beating or shooting someone into silence is a fine way to disagree. One of the key promises of civilization is to undo that might-makes-right bullshit that kept generations of peasants suffering while an aristocracy lived the high life peeing in hallways, flubbing courtly love, and stuffng birds into each other like meaty nesting-dolls. We’ve moved on.
Have we, though?
Skepticism
It’s funny-terrifying that the case made against Joe is so stupid. There are crass appeals to emotion. Prosecution says that there’s “all this, like, evidence” but does not actually share any, like, evidence. Frito riles the Gallery up with insults “he talks like a fag” and personal grievances that aren’t germane to the case. (Ow My Balls! should not be evidence in, well, anything.) It’s emotional theater and the audience eats it up, because what they’re there for is hits to the amygdala bong: The emotional highs and lows that you might get from a sports show. People screaming in cheers when their confirmation bias is rewarded, or fury when there’s a bit of cognitive dissonance. There’s probably nothing like a Fair Witness left on the planet, and if it weren’t for Joe’s intelligence that gets him out of jail, he would wind up suffering in prison for no reason at all. The lack of skepticism, of applying doubt to all things—especially the things that thrill you emotionally—causes unnecessary suffering. Sure, it’s an unjust universe, but civilization was invented to try and hold a little light against that darkness. The citizenry of Idiocracy are gullible and blithely self-serving to a cruel fault.
Impartiality
The system that pits prosecution against defense and presumes is called “adversarial,” since each counsel will present opposing cases to an impartial judge or jury, who is presumed competent to sort through the competing claims to come to the truth. It’s contrasted with an inquisitorial (or nonadversarial) system where the judges run the line of inquiry. Congressional hearings, for example, are much more of an inquisitorial system. Congresspeople ask the questions, rather than the lawyers.
The adversarial system depends on the impartiality of the judge and jury. If they presume guilt or innocence from the start, why have the trial at all? (This is sometimes called a kangaroo court, where procedure and ethics are ignored, and the outcome is a foregone conclusion.) Only dispassionate, neutral participants can look past their feelers to find a fair and impartial truth. You need people capable of impartial, critical thinking to look past the tricks and their own cognitive biases. That’s out the window in Idiocracy.
Joe is presumed guilty by literally everyone involved. It’s not wholly their fault. Joe is brought in wearing prison fatigues. He is wheeled in a cage. On the surface, he looks guilty. Humans have an availability bias, wherein easily recalled information is believed to be true and representative, and for very stupid people, that can be strictly what they see. And Joe looks guilty. Everyone around me is shouting that he’s guilty. So, the idiot thinks, he must be guilty.
Posers, surface evidence, and meaning
One of the worst things about the scene—and this plays out in the movie generally—is how people’s status is based on stupid, surface markers. For instance: Why do they accept that the judge is smart? Well, he uses fancy, “faggy” words like “commensurate” (It’s a malaprop. He means commence. But the Idiocrats are too stupid to know this.) and “proprietarily.” (I think he means “properly,” but it’s marvelously hard to tell.) Investigating this moves us very quickly into the semiosis treadmill, whereby a sign comes slowly to become the signified, even when it stupidly contradicts the original thing that was signified.
How do courts protect against idiocy?
It doesn’t always. But generally, there are lots of checks against incompetence. Anyone can request a mistrial if it’s shown that counsel, a judge, or a jury is incompetent, prejudiced, or doesn’t follow the rules. Lawyers can be reviewed and disbarred. Remedy for judges varies by state, but judges can be impeached, or undergo judicial review and be removed from their office. Their cases can be overturned by higher courts. (Yes, even that one.)
But notably all these presume that the incompetence is abnormal, and that others are competent to review and judge their competence. When everyone either gets so stupid or so corrupt that they have no interest in checking each other for fairness, the system just collapses.
Can design do anything?
This blog is primarily targeted at people who work in technology and love sci-fi. (Like I do.) A natural question is: Can design do anything to help inoculate courts against Idiocracy? And normally, I try to offer some hopeful answer to these questions, but this one has me stumped. What design interventions could we impose to increase skepticism? Impartiality? Critical thinking? Understanding? Seriously, if you have something, I’d love to hear it, because as of right now, I’ve got nothing.
How’s that for a Halloween scare?
Trick or Treat
Any Trump acolyte who has gleefully shouted “Lock her up! Lock her up!” at one of their farrowing-crate rallies, despite Clinton’s exoneration by the FBI, despite Trump’s own ongoing litany of scandals (including that ongoing issue of security), is guilty of this same kind of mob mentality as these morons in Idiocracy. It is all raw hate without reason. Hits from the tribalism bong. It’s not just that the mob hasn’t heard reason, it’s not interested in reason, only in justification. Only in thinking of political conversations as hobo fights. Did our guy land a hit? Insult? Hur dur, yarp.
Yes, you, redhat.
It’s pretty soul crushing. So let’s have some spooky, escapist fun tonight for self-care. But as of tomorrow we’ll have 6 days and we should hit it hard.
While recording a podcast with the guys at DecipherSciFi about the twee(n) love story The Space Between Us, we spent some time kvetching about how silly it was that many of the scenes involved Gardner, on Mars, in a real-time text chat with a girl named Tulsa, on Earth. It’s partly bothersome because throughout the rest of the the movie, the story tries for a Mohs sci-fi hardness of, like, 1.5, somewhere between Real Life and Speculative Science, so it can’t really excuse itself through the Applied Phlebotinum that, say, Star Wars might use. The rest of the film feels like it’s trying to have believable science, but during these scenes it just whistles, looks the other way, and hopes you don’t notice that the two lovebirds are breaking the laws of physics as they swap flirt emoji.
Hopefully unnecessary science brief: Mars and Earth are far away from each other. Even if the communications transmissions are sent at light speed between them, it takes much longer than the 1 second of response time required to feel “instant.” How much longer? It depends. The planets orbit the sun at different speeds, so aren’t a constant distance apart. At their closest, it takes light 3 minutes to travel between Mars and Earth, and at their farthest—while not being blocked by the sun—it takes about 21 minutes. A round-trip is double that. So nothing akin to real-time chat is going to happen.
But I’m a designer, a sci-fi apologist, and a fairly talented backworlder. I want to make it work. And perhaps because of my recent dive into narrow AI, I began to realize that, well, in a way, maybe it could. It just requires rethinking what’s happening in the chat.
Let’s first acknowledge that we’ve solved long distance communications a long time ago. Gardner and Tulsa could just, you know, swap letters or, like the characters in 2001: A Space Odyssey, recorded video messages. There. Problem solved. It’s not real-time interaction, but it gets the job done. But kids aren’t so much into pen pals anymore, and we have to acknowledge that Gardner doesn’t want to tip his hand that he’s on Mars (it’s a grave NASA secret, for plot reasons). So the question is how could we make it work so it feels like a real time chat to her. Let’s first solve it for the case where he’s trying to disguise his location, and then how it might work when both participants are in the know.
Fooling Tulsa
Since 1984 (ping me, as always, if you can think of an earlier reference) sci-fi has had the notion of a digitally-replicated personality. Here I’m thinking of Gibson’s Neuromancer and the RAM boards on which Dixie Flatline “lives.” These RAM boards house an interactive digital personality of a person, built out of a lifetime of digital traces left behind: social media, emails, photos, video clips, connections, expressed interests, etc. Anyone in that story could hook the RAM board up to a computer, and have conversations with the personality housed there that would closely approximate how that person would (or would have) respond in real life.
Listen to the podcast for a mini-rant on translucent screens, followed by apologetics.
Is this likely to actually happen? Well it kind of already is. Here in the real world, we’re seeing early, crude “me bots” populate the net which are taking baby steps toward the same thing. (See MessinaBot, https://bottr.me/, https://sensay.it/, the forthcoming http://bot.me/) By the time we actually get a colony to Mars (plus the 16 years for Gardner to mature), mebot technology should should be able to stand in for him convincingly enough in basic online conversations.
Training the bot
So in the story, he would look through cached social media feeds to find a young lady he wanted to strike up a conversation with, and then ask his bot-maker engine to look at her public social media to build a herBot with whom he could chat, to train it for conversations. During this training, the TulsaBot would chat about topics of interest gathered from her social media. He could pause the conversation to look up references or prepare convincing answers to the trickier questions TulsaBot asks. He could also add some topics to the conversation they might have in common, and questions he might want to ask her. By doing this, his GardnerBot isn’t just some generic thing he sends out to troll any young woman with. It’s a more genuine, interactive first “letter” sent directly to her. He sends this GardnerBot to servers on Earth.
A demonstration of a chat with a short Martian delay. (Yes, it’s an animated gif.)
Launching the bot
GardnerBot would wait until it saw Tulsa online and strike up the conversation with her. It would send a signal back to Gardner that the chat has begun so he can sit on his end and read a space-delayed transcript of the chat. GardnerBot would try its best to manage the chat based on what it knows about awkward teen conversation, Turing test best practices, what it knows about Gardner, and how it has been trained specifically for Tulsa. Gardner would assuage some of his guilt by having it dodge and carefully frame the truth, but not outright lie.
Buying time
If during the conversation she raised a topic or asked a question for which GardnerBot was not trained, it could promise an answer later, and then deflect, knowing that it should pad the conversation in the meantime:
Ask her to answer the same question first, probing into details to understand rationale and buy more time
Dive down into a related subtopic in which the bot has confidence, and which promises to answer the initial question
Deflect conversation to another topic in which it has a high degree of confidence and lots of detail to share
Text a story that Gardner likes to tell that is known to take about as long as the current round-trip signal
Example
TULSA
OK, here’s one: If you had to live anywhere on Earth where they don’t speak English, where would you live?
GardnerBot has a low confidence that it knows Gardner’s answer. It could respond…
(you first) “Oh wow. That is a tough one. Can I have a couple of minutes to think about it? I promise I’ll answer, but you tell me yours first.”
(related subtopic) “I’m thinking about this foreign movie that I saw one time. There were a lot of animals in it and a waterfall. Does that sound familiar?”
(new topic) “What? How am I supposed to answer that one? 🙂 Umm…While I think about it, tell me…what kind of animal would you want to be reincarnated as. And you have to say why.”
(story delay) “Ha. Sure, but can I tell a story first? When I was a little kid, I used to be obsessed with this music that I would hear drifting into my room from somewhere around my house…”
Lagged-realtime training
Each of those responses is a delay tactic that allows the chat transcript to travel to Mars for Gardner to do some bot training on the topic. He would be watching the time-delayed transcript of the chat, keeping an eye on an adjacent track of data containing the meta information about what the bot is doing, conversationally speaking. When he saw it hit low-confidence or high-stakes topic and deflect, it would provide a chat window for him to tell the GardnerBot what it should do or say.
To the stalling GARDNERBOT…
GARDNER
For now, I’m going to pick India, because it’s warm and I bet I would really like the spicy food and the rain. Whatever that colored powder festival is called. I’m also interested in their culture, Bollywood, and Hinduism.
As he types, the message travels back to Earth where GardnerBot begins to incorporate his answers to the chat…
At a natural break in the conversation…
GARDNERBOT
OK. I think I finally have an answer to your earlier question. How about…India?
TULSA
India?
GARDNERBOT
Think about it! Running around in warm rain. Or trying some of the street food under an umbrella. Have you seen youTube videos from that festival with the colored powder everywhere? It looks so cool. Do you know what it’s called?
Note that the bot could easily look it up and replace “that festival with the colored powder everywhere” with “Holi Festival of Color” but it shouldn’t. Gardner doesn’t know that fact, so the bot shouldn’t pretend it knows it. A Cyrano-de-Bergerac software—where it makes him sound more eloquent, intelligent, or charming than he really is to woo her—would be a worse kind of deception. Gardner wants to hide where he is, not who he is.
That said, Gardner should be able to direct the bot, to change its tactics. “OMG. GardnerBot! You’re getting too personal! Back off!” It might not be enough to cover a flub made 42 minutes ago, but of course the bot should know how to apologize on Gardner’s behalf and ask conversational forgiveness.
Gotta go
If the signal to Mars got interrupted or the bot got into too much trouble with pressure to talk about low confidence or high stakes topics, it could use a believable, pre-rolled excuse to end the conversation.
GARDNERBOT
Oh crap. Will you be online later? I’ve got chores I have to do.
Then, Gardner could chat with TulsaBot on his end without time pressure to refine GardnerBot per their most recent topics, which would be sent back to Earth servers to be ready for the next chat.
In this way he could have “chats” with Tulsa that are run by a bot but quite custom to the two of them. It’s really Gardner’s questions, topics, jokes, and interest, but a bot-managed delivery of these things.
So it could work, does it fit the movie? I think so. It would be believable because he’s a nerd raised by scientists. He made his own robot, why not his own bot?
From the audience’s perspective, it might look like they’re chatting in real time, but subtle cues on Gardner’s interface reward the diligent with hints that he’s watching a time delay. Maybe the chat we see in the film is even just cleverly edited to remove the bots.
How he manages to hide this data stream from NASA to avoid detection is another question better handled by someone else.
An honest version: bot envoy
So that solves the logic from the movie’s perspective but of course it’s still squickish. He is ultimately deceiving her. Once he returns to Mars and she is back on Earth, could they still use the same system, but with full knowledge of its botness? Would real world astronauts use it?
Would it be too fake?
I don’t think it would be too fake. Sure, the bot is not the real person, but neither are the pictures, videos, and letters we fondly keep with us as we travel far from home. We know they’re just simulacra, souvenir likenesses of someone we love. We don’t throw these away in disgust for being fakes. They are precious because they are reminders of the real thing. So would the themBot.
GARDNER
Hey, TulsaBot. Remember when we were knee deep in the Pacific Ocean? I was thinking about that today.
TULSABOT
I do. It’s weird how it messes with your sense of balance, right? Did you end up dreaming about it later? I sometimes do after being in waves a long time.
GARDNER
I can’t remember, but someday I hope to come back to Earth and feel it again. OK. I have to go, but let me know how training is going. Have you been on the G machine yet?
Nicely, you wouldn’t need stall tactics in the honest version. Or maybe it uses them, but can be called out.
TULSA
GardnerBot, you don’t have to stall. Just tell Gardner to watch Mission to Mars and update you. Because it’s hilarious and we have to go check out the face when I’m there.
Sending your loved one the transcript will turn it into a kind of love letter. The transcript could even be appended with a letter that jokes about the bot. The example above was too short for any semi-realtime insertions in the text, but maybe that would encourage longer chats. Then the bot serves as charming filler, covering the delays between real contact.
Ultimately, yes, I think we can backworld what looks physics-breaking into something that makes sense, and might even be a new kind of interactive memento between interplanetary sweethearts, family, and friends.
As Jack searches early in the film for Drone 172, he parks his bike next to a sinkhole in the desert and cautiously peers into it. As he does so, he is being observed from afar by a sinister looking Scav through a set of asymmetrical…well, it’s not exactly right to call them binoculars.
They look kind of binocular, but that term technically refers to a machine that displays two slighty-offset images shown independently to each eye such that the user perceives stereopsis, or a single field in 3D. But a quick shot from the Scav’s perspective shows that this is not what is shown at all. Continue reading →
The TETVision display is the only display Vika is shown interacting with directly—using gestures and controls—whereas the other screens on the desktop seem to be informational only. This screen is broken up into three main sections:
The left side panel
The main map area
The right side panel
The left side panel
The communications status is at the top of the left side panel and shows Vika the status of whether the desktop is online or offline with the TET as it orbits the Earth. Directly underneath this is the video communications feed for Sally.
Beneath Sally’s video feed is the map legend section, which serves the dual purposes of providing data transfer to the TET and to the Bubbleship as well as a simple legend for the icons used on the map.
The communications controls, which are at the bottom of the left side panel, allow Vika to toggle the audio communications with Jack and with Sally. Continue reading →
Forgive me, as I am but a humble interaction designer (i.e., neither a professional visual designer nor video editor) but here’s my shot at a redesigned DuoMento, taking into account everything I’d noted in the review.
There’s only one click for Carl to initiate this test.
To decrease the risk of a false positive, this interface draws from a large category of concrete, visual and visceral concepts to be sent telepathically, and displays them visually.
It contrasts Carl’s brainwave frequencies (smooth and controlled) with Johnny’s (spiky and chaotic).
It reads both the brain of the sender and the receiver for some crude images from their visual cortex. (It would be better at this stage to have the actors wear some glowing attachment near a crown to show how this information was being read.)
These changes are the sort that even in passing would help tell a more convincing narrative by being more believable, and even illustrating how not-psychic Johnny really is.
