High Tech Binoculars

11 Jan 2017 by scifihughf

In Johnny Mnemonic we see two different types of binoculars with augmented reality overlays and other enhancements: Yakuz-oculars, and LoTek-oculars.

Yakuz-oculars

The Yakuza are the last to be seen but also the simpler of the two. They look just like a pair of current day binoculars, but this is the view when the leader surveys the LoTek bridge.

I assume that the characters here are Japanese? Anyone?

In the centre is a fixed-size green reticule. At the bottom right is what looks like the magnification factor. At the top left and bottom left are numbers, using Western digits, that change as the binoculars move. Without knowing what the labels are I can only guess that they could be azimuth and elevation angles, or distance and height to the centre of the reticule. (The latter implies some sort of rangefinder.)

So far, this is a simple uncluttered display. But why is there a brightly glowing Pharmakom logo at the top right? It blocks part of the view, and probably doesn’t help anyone trying to keep their eyes adapted for night vision.

LoTek-oculars

The LoTeks, despite their name, have more impressive binoculars. They’re first used when Johnny gets out of his airport taxi.

There’s a third tube above the optics, a rectangular inlet, and an antenna.

In these binoculars, the augmented reality overlay is much more dynamic. Instead of a fixed circle, green lines converge in a bounding box around the image of Johnny. Text slides onto the display from left to right, the last line turning yellow.

Zoomrect

The animated transition of the bounding box resembles what Classic MacOS programmers of the 1990s called “zoomrects” used for showing windows opening or closing. It’s a very effective technique to draw attention to a particular area of an image.

Animated text

Text appearing character by character is ubiquitous in film interfaces. In the 1960s and 1970s mainframe and minicomputer terminals really did display incrementally, as the characters arrived one by one over slow serial port links. On any more recent computer it actually takes extra programming to achieve this effect, as the normal display of text is so fast that we would perceive it as instantaneous. But people like to see incremental text, or have been conditioned by film to expect it, so why not?

Bioscanning

The binoculars detect Johnny’s implant. It might just be possible to detect this passively from infrared or electronic signals, but more likely the binoculars include a high resolution microwave radar as well. If there had been more than one person in view, the bounding box would indicate which one the text refers to. And note that the last line of text is a different color. What that means is unclear here, but it becomes clear (and I’ll discuss it) later.

The second time we see the LoTek binoculars is when a lookout spots Street Preacher, a very bad guy and another who wants to remove Johnny’s head. Once again the binoculars have performed more than just a visual scan.

The binocular view and overlay are being relayed to another character, the LoTek leader J-Bone who can watch on a monitor. Here the film anticipates the WiFi webcam.

The overlay text now changes.

Narrow AI?

This is interesting, because the binoculars can not only detect implants and other cyborg modifications, but are apparently able to evaluate and offer advice. It appears that the green text is used for the factual (more or less) information about what has been detected, while yellow text is uncertain or or speculative.

Does this imply a general artificial intelligence? Not necessarily. This warning could be based solely on the detected signature, in the same way that current day military passive sonars and radar warning receivers can identify threats based on identifying characteristics of a received signal. In the world of Johnny Mnemonic it would make sense to assume that anyone with full custom biomechanics is extremely dangerous. Or, since Street Preacher is a resident rather than a stranger and already feared by others, his appearance and the warning could have been entered into a LoTek facial recognition database that the binocular system uses as a reference.

These textual overlays are an excellent interface, not interfering with normal vision and providing a fast and easy-to-understand analysis. But, the user must have faith that the computer analysis is accurate. There’s no reason given as to why any of the text is displayed. If Johnny was carrying an implant in his pocket instead of his brain, would the computer know the difference?

An alternative approach would be some kind of sensor fusion or false spectrum display, with the raw infrared or radar image overlaid over the visuals and the viewer responsible for interpreting the data. The problem with such systems is that our visual system didn’t evolve to interpret such imagery, so a lot of training and practice is required to be both fast and accurate. And the overlay itself interferes with our normal visual recognition and processing. If the computer can do a better job of deciphering the meaning of non-visual data, it should do so and summarise for the human viewer.

Further advantages of this interface are that even a novice sentry will benefit from the built-in scanning and threat analysis, and the wireless transmission ensures that the information is shared rather than being limited to the person on watch.

Brain Upload

7 Dec 2016 by scifihughf

Once Johnny has installed his motion detector on the door, the brain upload can begin.

3. Building it

Johnny starts by opening his briefcase and removing various components, which he connects together into the complete upload system. Some of the parts are disguised, and the whole sequence is similar to an assassin in a thriller film assembling a gun out of harmless looking pieces.

It looks strange today to see a computer system with so many external devices connected by cables. We’ve become accustomed to one piece computing devices with integrated functionality, and keyboards, mice, cameras, printers, and headphones that connect wirelessly.

Cables and other connections are not always considered as interfaces, but “all parts of a thing which enable its use” is the definition according to Chris. In the early to mid 1990s most computer user were well aware of the potential for confusion and frustration in such interfaces. A personal computer could have connections to monitor, keyboard, mouse, modem, CD drive, and joystick – and every single device would use a different type of cable. USB, while not perfect, is one of the greatest ever improvements in user interfaces.

Why not go wireless? Wireless devices remove the need for a physical connection, but this means that anyone, not just you, could potentially connect. So instead of worrying about whether we have the right kind of cable, we now worry about the right kind of Bluetooth pairing and WiFi encryption password scheme. Mobile wireless devices also need their own batteries, which have to be charged. So wireless may seem visually cleaner, but comes with its own set of problems.

As of early 2016 we have two new standards, Lightning and USB-C, that are orientation-independent (only fifty years after audio cables), high bandwidth, and able to transmit power to peripherals as well. Perhaps by 2021 cables will have made a comeback as the usual way to connect devices.

2. Explaining it

Johnny explains the process to the scientists. He needs them to begin the upload by pushing a button, helpfully labelled “start”, on the gadget that resembles an optical disk drive. There’s a big red button as well, which is not explained but would make an excellent “cancel” button.

It would be simpler if Johnny just did this himself. But we will shortly discover that the upload process is apparently very painful. If Johnny had his hands near the system, he might involuntarily push another button or disturb a cable. So for them, having a single, easily differentiated button to press minimizes their chance of messing it up.

1. Making codes

He also sticks a small black disk on the hotel room’s silver remote control. The small disk is evidently is a wireless controller or camera of some kind. The scientists must watch the upload progress counter, and as it approaches the end, use this modified remote to grab three frames from the TV display, which will become the “access code” for the data. (More on this below.)

None of the buttons on this remote have markings or labels, but neither Johnny nor the scientist who will be using it are bothered. Perhaps this hotel chain tries to please every possible guest by not favouring any particular language? But even in that case, I’d expect there to be some kind of symbols on the buttons and a multilingual manual to explain the meaning of each. Maybe Johnny spends so much time in hotel suites that he has memorised the button layout?

Short of a mind reading remote that can translate any button press into “what the user intended”, I have to admit this is a terrible interface.

(There is a label on the black disk, but I have no idea what it means or even which script that is. Anyone?)

0. Go go go

Johnny plugs in his implant, puts on a headset with more cables, and bites down on a mouthguard. He’s ready.

The scientist pushes the start button and the upload begins. Johnny sees the data stream in his headset as a flood of graphics and text.

Why does he need the headset when there is a direct cable connection to the implant? The movie doesn’t make it explicit. It could be related to the images used as the access code. (More on this below.) Perhaps the images need to be processed by the recipient’s own optic nerve system for more reliable storage?

Still, in the spirit of apologetics we should try to find a better explanation than “an opportunity for 1995 cutting edge computer generated graphics.” Perhaps it is a very flashy progress indicator? Older computer systems had blinking lights on disk drives to indicate activity, copied on some of today’s USB sticks. Current-day file upload or download GUIs have progress bars. As processing and graphics capabilities increase, it will be possible for software to display thumbnails or previews of the actual data being transferred without slowing down.

Unfortunately there is an argument against this, which is that the obvious upload progress indicator is a numeric display counting gigabytes down to zero, and it makes a fast chirping sound as a sonic indicator as well. The counter shows the data flowing at gigabytes per second, the entire upload lasting about a minute. There’s also the problem that it’s not Johnny who is interested in knowing whether the upload is scientific data rather than, say, a video collection; but the scientists, and they can’t see it.

As the counter drops below one hundred, the scientist points the remote with black disk at the TV display, currently showing a cartoon, and presses the middle button. The image from the TV appears overlaid on the data stream to Johnny. This is a little odd, because Johnny assured the scientists that he wouldn’t know what the access codes were himself. Maybe these brief flashes are not enough time for him to remember these particular images among the gigabytes of visual content. But the way they’re shown to us, I’ll bet you can remember them when they come up again later in the plot.

Two more images are grabbed before the counter stops. When the upload finishes, the three images are printed out. (In the original film this is shown upside down, so I have rotated the image.)

Tagged:

So what are the images for? The script isn’t clear. I suggest that the images are being used as the equivalents of very large random numbers for whatever cryptography scheme protects the data against unauthorised access. Some current day systems use the timing of key presses and mouse movements as a source of randomness because humans simply can’t move their fingers with microsecond precision. Here, the human element makes it impossible to predict exactly which frame is chosen.

Humans also find images much easier to recognise than hundred digit numbers. Anyone who has seen the printout will be able to say whether a particular image is part of the access code or not with a high degree of confidence. In computer systems today, Secure Shell, or ssh, is a widely used encrypted terminal program for secure access to servers. Recent versions of ssh have a ‘randomart’ capability which shows a small ASCII icon generated from the current cryptographic key to everyone who logs on. If this ASCII icon appears different, this alerts everyone that the server key has been changed.

There’s one potential usability problem with the whole “pick three random images” mechanism. The last frame was grabbed when the counter was very close to zero. What would have happened if he had been too slow and missed altogether? Wouldn’t it be more reliable to have the upload system automatically grab the images rather than rely on a human? Chris suggests that maybe it secretly did grab three images that could have used without human input, but privileged the human input since it was more reliably random.

Quick aside: You may be asking, if images would be so wonderful, why aren’t we using them in this way already? It’s because our current security systems need not just very large random numbers, but very large random numbers with particular mathematical properties such as being prime. But let’s cut Johnny Mnemonic some slack, saying that by 2021 we may have new algorithms.

OK, back to the plot.

-1. Sharing the codes

The access codes are to be faxed from Beijing to Newark, although this gets interrupted by the Yakuza intruders. This is yet another device with unmarked buttons.

This device makes the same beeps and screeches as a 1990s analog fax machine. Since we’ll later learn that all the fax messages and phone calls are stored digitally in cyberspace, this must be a skeuomorphism, the old familiar audio tones now serving just as progress indicators.

As with other audio output, the tones allow the user to know that the transmission is proceeding and when it ends without having to pay full attention to the device. On the other hand, there is potential for confusion here as the digital upload is (presumably) much faster. Most current day computer systems could upload three photos, even in high resolution, well before the sequence of tones would complete. Users would most likely wait longer than actually necessary before moving on to their next task.

-2. Washing up

During the upload Johnny clenches his fists and bites his mouthguard. When the upload finishes, he retreats to the bathroom in considerable pain. At one point blood flows from his nose, and he swipes his hand over the tap to wash it down the drain. The bathroom announces that the water temperature is 17 degrees. We’ll come back to this later.

As Make It So emphasises in the chapter on brain interfaces, there is nothing in our current knowledge to suggest that writing or reading memories to or from a human brain would be painful. On the other hand, we know that information in the brain isthe shape of the neurons in the brain. Who knows what side effects will happen as those neurons are disconnected and reconnected as they need to be? We don’t know, so can’t really say whether it would hurt or not.

-3. Escaping the Yakuza

As mentioned in a prior post, while he is in the bathroom, the motion detector Johnny installed on the hotel door isn’t very effective and the Yakuza break in, kill everyone else, and acquire the second of the three access code images. Johnny escapes with the first image and flies to Newark, North America.

Compartment 21

14 Aug 2014 by Christopher Noessel

After the Communications Tower is knocked off, Barcalow, looking out the viewport, somehow knows exactly where the damage to the ship has occured. This is a little like Captain Edward Smith looking out over the bow of the RMS Titanic and smelling which compartment was ripped open by the iceburg, but we must accept the givens of the scene. Barcalow turns to Ibanez and tells her to “Close compartment 21!” She turns to her left, reaches out, and presses a green maintained-contact button labeled ENABLE. This button is right next to a similar-but-black button also labeled ENABLE. As she presses the button, a nearby green LED flashes for a total of 4 frames, or 0.16 second.

She looks up at some unseen interface, and, pleased with what she sees there, begins to relax, the crisis passed.

A weary analysis

Let’s presume he is looking at some useful but out-of-character-for-this-bridge display, and that it does help him identify that yes, out of all the compartments that might have been the one they heard damaged, it is the 21st that needs closing.

Why does he have the information but she have the control? Time is wasted (and air—not to mention lives, people—is lost) in the time it takes him to instruct and her to react.
How did she find the right button when it’s labeled exactly the same way as adjacent button? Did she have to memorize the positions of all of them? Or the color? (How many compartments and therefore colors would that mean she would need to memorize?) Wouldn’t a label reading, say, “21” have been more useful in this regard?
What good does an LED do to flash so quickly? Certainly, she would want to know that the instruction was received, but it’s a very fast signal. It’s easy to miss. Shouldn’t it have stayed on to indicate not the moment of contact, but the state?
Why was this a maintained-contact button? Those look very similar when pressed or depressed. A toggle switch would display its state immediately, and would permit flipping a lot of them quickly, in case a lot of compartments need sealing.
Why is there some second place she must look to verify the results of her action, that is a completely separate place from Barcalow (remember he looks forward, she looks up). Sure, maybe redundancy. Sure, maybe he’s looking at data and she’s looking at video feeds, but wouldn’t it be better if they were looking at the same thing?

I know it’s a very quick interaction. And props to the scriptwriters for thinking about leaking air in space. But this entire interaction needs rethinking.

Otto’s Manual Control

7 Jan 2014 by claytonbeese

When it refused to give up authority, the Captain wrested control of the Axiom from the artificial intelligence autopilot, Otto. Otto’s body is the helm wheel of the ship and fights back against the Captain. Otto wants to fulfil BNL’s orders to keep the ship in space. As they fight, the Captain dislodges a cover panel for Otto’s off-switch. When the captain sees the switch, he immediately realizes that he can regain control of the ship by deactivating Otto. After fighting his way to the switch and flipping it, Otto deactivates and reverts to a manual control interface for the ship.

The panel of buttons showing Otto’s current status next to the on/off switch deactivates half its lights when the Captain switches over to manual. The dimmed icons are indicating which systems are now offline. Effortlessly, the captain then returns the ship to its proper flight path with a quick turn of the controls.

One interesting note is the similarity between Otto’s stalk control keypad, and the keypad on the Eve Pod. Both have the circular button in the middle, with blue buttons in a semi-radial pattern around it. Given the Eve Pod’s interface, this should also be a series of start-up buttons or option commands. The main difference here is that they are all lit, where the Eve Pod’s buttons were dim until hit. Since every other interface on the Axiom glows when in use, it looks like all of Otto’s commands and autopilot options are active when the Captain deactivates him.

A hint of practicality…

The panel is in a place that is accessible and would be easily located by service crew or trained operators. Given that the Axiom is a spaceship, the systems on board are probably heavily regulated and redundant. However, the panel isn’t easily visible thanks to specific decisions by BNL. This system makes sense for a company that doesn’t think people need or want to deal with this kind of thing on their own.

Once the panel is open, the operator has a clear view of which systems are on, and which are off. The major downside to this keypad (like the Eve Pod) is that the coding of the information is obscure. These cryptic buttons would only be understandable for a highly trained operator/programmer/setup technician for the system. Given the current state of the Axiom, unless the crew were to check the autopilot manual, it is likely that no one on board the ship knows what those buttons mean anymore.

Thankfully, the most important button is in clear English. We know English is important to BNL because it is the language of the ship and the language seen being taught to the new children on board. Anyone who had an issue with the autopilot system and could locate the button, would know which button press would turn Otto off (as we then see the Captain immediately do).

Considering that Buy-N-Large’s mission is to create robots to fill humans’ every need, saving them from every tedious or unenjoyable job (garbage collecting, long-distance transportation, complex integrated systems, sports), it was both interesting and reassuring to see that there are manual over-rides on their mission-critical equipment.

…But hidden

The opposite situation could get a little tricky though. If the ship was in manual mode, with the door closed, and no qualified or trained personnel on the bridge, it would be incredibly difficult for them to figure out how to physically turn the ship back to auto-pilot. A hidden emergency control is useless in an emergency.

Hopefully, considering the heavy use of voice recognition on the ship, there is a way for the ship to recognize an emergency situation and quickly take control. We know this is possible because we see the ship completely take over and run through a Code Green procedure to analyze whether Eve had actually returned a plant from Earth. In that instance, the ship only required a short, confused grunt from the Captain to initiate a very complex procedure.

Security isn’t an issue here because we already know that the Axiom screens visitors to the bridge (the Gatekeeper). By tracking who is entering the bridge using the Axiom’s current systems, the ship would know who is and isn’t allowed to activate certain commands. The Gatekeeper would either already have this information coded in, or be able to activate it when he allowed people into the bridge.

For very critical emergencies, a system that could recognize a spoken ‘off’ command from senior staff or trained technicians on the Axiom would be ideal.

Anti-interaction as Standard Operating Procedure

The hidden door, and the obscure hard-wired off button continue the mission of Buy-N-Large: to encourage citizens to give up control for comfort, and make it difficult to undo that decision. Seeing as how the citizens are more than happy to give up that control at first, it looks like profitable assumption for Buy-N-Large, at least in the short term. In the long term we can take comfort that the human spirit–aided by an adorable little robot–will prevail.

So for BNL’s goals, this interface is fairly well designed. But for the real world, you would want some sort of graceful degradation that would enable qualified people to easily take control in an emergency. Even the most highly trained technicians appreciate clearly labeled controls and overrides so that they can deal directly with the problem at hand rather than fighting with the interface.

The Lifeboat Controls

2 Jan 2014 by claytonbeese

After Wall-E and Eve return to the Axiom, Otto steals the Earth plant and has his security bot place it on a lifeboat for removal from the ship. Wall-E follows the plant onboard the pod, and is launched from the Axiom when the security bot remotely activates the pod. The Pod has an autopilot function (labeled an auto-lock, and not obviously sentient), and a Self-Destruct function, both of which the security bot activates at launch. Wall-E first tries to turn the auto-pilot off by pushing the large red button on the control panel. This doesn’t work.

Wall-E then desperately tries to turn off the auto-destruct by randomly pushing buttons on the pod’s control panel. He quickly gives up as the destruct continues counting down and he makes no progress on turning it off. In desperation, Wall-E grabs a fire extinguisher and pulls the emergency exit handle on the main door of the pod to escape.

The Auto-Destruct

There are two phases of display on the controls for the Auto-Destruct system: off and countdown. In its off mode, the area of the display dedicated to the destruct countdown is plain and blue, with no label or number. The large physical button in the center is unlit and hidden, flush with the console. There is no indication of which sequence of keypresses activates the auto-destruct.

When it’s on, the area turns bright red, with a pulsing countdown in large numbers, a large ‘Auto-Destruct’ label on the left. The giant red pushbutton in the center is elevated above the console, surrounded by hazard striping, and lit from within.

The odd part is that when the button in the center gets pushed down, nothing happens. This is the first thing Wall-E does to turn the system off, and it’s has every affordance for being a button to stop the auto-destruct panel in which it sits. It’s possible that this center button is really just a pop-up alert light to add immediacy to the audible and other visual cues of impending destruction.

If so, the pod’s controls are seriously inadequate.

Wall-E wants to shut the system off, and the button is the most obvious choice for that action. Self-destruction is an irreversible process. If accidentally activated, it is something that needs to be immediately shut off. It is also something that would cause panicked decision making in the escape pod’s users.

The blinking button in the center of the control area is the best and most obvious target to “SHUT IT OFF NOW!”

Of course this is just part of the fish-out-of-water humor of the scene, but is there a real reason it’s not responding like it obviously should? One possibility is that the pod is running an authority scan of all the occupants (much like the Gatekeeper for the bridge or what I suggested for Eve’s gun), and is deciding that Wall-E isn’t cleared to use that control. If so, that kind of biometric scanning should be disabled for a control like the Anti-Auto-Destruct. None of the other controls (up to and including the airlock door exit) are disabled in the same way, which causes serious cognitive dissonance for Wall-E.

The Axiom is able to defend itself from anyone interested in taking advantage of this system through the use of weapons like Eve’s gun and the Security robots’ force fields.

Anything that causes such a serious effect should have an undo or an off switch. The duration of the countdown gives Wall-E plenty of time to react, but the pod should accept that panicked response as a request to turn the destruct off, especially as a fail-safe in case its biometric scan isn’t functioning properly, and there might be lives in the balance.

The Other Controls

No Labels.

Seriously?

This escape pod is meant to be used in an emergency, and so the automatic systems should degrade as gracefully as possible.

While beautiful, extremely well grouped by apparent function, and incredibly responsive to touch inputs, labels would have made the control panel usable for even a moderately skilled crewmember in the pilot seat. Labels would also provide reinforcement of a crew member’s training in a panic-driven situation.

Buy-N-Large: Beautifully Designed Dystopia

A design should empower the people using it, and provide reinforcement to expert training in a situation where memory can be strained because of panic. The escape-pod has many benefits: clear seating positions, several emergency launch controls, and an effective auto-pilot. Adding extra backups to provide context for a panicked human pilot would add to the pod’s safety and help crew and passengers understand their options in an emergency.

Mr. Yuk

28 Jan 2013 by Christopher Noessel

On the side of the valley in which the first complex is found, there is a giant skull carved into the overlooking crag. It’s easy—given the other transgressions in the film—to dismiss this as spookhouse attempt at being scary. But what if (stay with me here) it’s a warning sign, an alien Mr. Yuk, put there for other sentient humanoids to understand that this place is deadly with a capital D? This explains why the outpost hasn’t been disturbed by rescuers of their own race. They were smart enough to see the warning and turn right back around. (Why they didn’t nuke it from orbit is another question.)

The Material

Seeing this as a warning label raises other questions. Why wouldn’t a warning be technological or linguistic, like most of the interfaces inside the complex? The black infection material is still deadly after 2000 years. Who knows how much longer it will be viable? So where the interfaces inside are for immediate use, the warning outside needs to be effective for millennia, outlasting both the power reserves that would drive technology and the persistent semantics that would cement linguistic understanding. Rock, in contrast, lasts a very, very long time. Even during the erosion the shape and its clear meaning will simply lose clarity, not wink out altogether.

The symbol

Similarly, this shape is a clear symbol of death that is tied to biology, which changes on evolutionary timeframes, guaranteeing its readability for—hopefully—longer than the xenomorph liquid would be a danger.

For these reasons, this is labeling that is more than a Castle Grayskull set dressing attempt at scaaaarrrry, but a reasonable choice at providing an effective warning that will last as long as the danger. You know, providing visiting scientists actually pay attention to such things.

Sci-fi interfaces

Stop watching sci-fi. Start using it.

Tag Archives: label