Computers & interfaces of the future - what REALLY would happen?

[UsableThought]

Not the best thread title; improvements welcome.

Let me give an example of what I mean. In the original "Star Trek," all the consoles had raised buttons, so operating the ship was kind of like using one of the old IBM Selectric typewriters, or an early PC keyboard like the one on the IBM PC/AT - hefty click-y finger feel w/ some muscle needed to make it happen.

However by the time of "Next Generation", all consoles were what we think of today as touchscreens - and extremely sensitive at that, e.g. Data's hands can dance with extreme rapidity over the console to execute zillions of commands; and frequently we see sliding or gliding motions of a fingertip. And typically the entire surface of a console, whether at helm or in engineering or sick bay, is completely covered with buttons or other command glyphs.

So . . . I've always had this picture in my mind of what would REALLY happen, sooner or later, if the helm consoles were as sensitive as they are depicted to be.

(INT. SHOT - ENTERPRISE BRIDGE)

CAPT PICARD: Jordi, how long will it take to get to the deep space station at Alpha Numeric 7?
JORDI (seated at the helm in front): Well, Captain, about a week at Warp 3, I think.
PICARD: You think? Where is Data when I need him? Can't you do the math in your head and give me a more precise answer than that?
JORDI: Well, let's see. Hmm.

(CLOSE UP on JORDI frowning, deep in thought. He has half-pivoted in his seat so as to hear the captain better. Warp calculations are difficult! As he works through the problem in his mind, he absent-mindedly places an elbow on the helm console so he can rest his chin in is hand. His elbow slips a bit.)

(EXTERIOR SHOT OF ENTERPRISE: The Enterprise whips horizontally through two 720 rotations, first to port then to starboard, in 1/10th of a second; then instantly excutes a perfect https://en.wikipedia.org/wiki/List_of_snowboard_tricks#Flips_and_Inverted_Rotations. Hull stresses have exceeded design capabilities and the ship explodes.)

 

[NTL2009]

Tactile feedback from a touch screen might be an improvement. Maybe a 'shape-shifting' screen, so we could feel the buttons?

Or maybe just detecting the position of our hands in 3-D space (a bit like a Theremin)?

 

[Ryan_m_b]

Along these lines if you look at a show like the Expanse everything is controlled by gesture and natural language. The latter would be useful in some circumstances, gesture I can't help but feel would be too prone to error and don't see any advantage beyond a touch-based system.

Coincidentally a friend sent me this recently, it's an interesting and entertaining analysis of interfaces in science fiction

 

[UsableThought]

Coincidentally a friend sent me this recently, it's an interesting and entertaining analysis of interfaces in science fiction

Right about 9:50 on there is one example from a film that is perfect!

 

[mfb]

We have those questions for machines today as well, and there are some typical ways to prevent accidents:

- impose hard limits on actions if operation beyond those limits should never happen
- let the user confirm actions that are outside normal operation ranges

Add a camera or similar device that evaluates what the user is doing: Do they touch the screen with their hand and watch it, or do they just lean their arm on it?
Add an AI that evaluates the probability that the user really wants to perform this action now, and asks for confirmation if the probability is too low.

 

[UsableThought]

impose hard limits on actions if operation beyond those limits should never happen

- let the user confirm actions that are outside normal operation ranges

Add an AI that evaluates the probability that the user really wants to perform this action now, and asks for confirmation if the probability is too low.

Interesting that in Next Generation, this is never shown for the hardware (e.g. consoles) but is a common theme for command procedures (e.g. to set the auto-destruct requires at the captain and first officer to tell the computer they want to do this, give their secret decoder ring codes, etc.). Of course story demands drive this, so that's OK. Dramatic to show auto-destruct being set; not so dramatic to have the helm console on the Enterprise speak out loud in a little squeaky voice to ask Jordi ", Do you really want to do that, sir?"

It would be interesting to know if there's a sci fi movie where a serious attempt was made to make the interfaces as genuine as possible, whether re: safety or some other constraints that we might recognize as meaningful; but unlikely, since there's probably zero bang for the buck. Since everything's imaginary anyway, a vivid interface has more value than a meaningful one; and "realistic" only means "seems realistic."

 

[UsableThought]

Interfaces as memes. Some just get repeated, e.g. the auto-destruct in Star Trek . . . and Alien . . . and probably a bunch of other movies. Others carry on, but evolve as they do so.

Case in point: Blade Runner had the cool scene where Harrison Ford as the cop fed a 2D snapshot photo into his computer, and was then able to ask the computer to transform a still 2D photo into what was effectively 3D, peeking around corners to scan the entire room: "Enhance 224-176 . . . enhance . . . stop. Move in." I was reminded of this now ancient scene while watching scenes with a cop in the sci-fi (now Amazon Streaming) TV show The Expanse; major differences are, the images he's working with are from a vast police/public/private database of some sort & are being holographically projected by what looks to be a smart phone, and plus he's able to manipulate them with his hands as well as voice, a la Tony Stark.

 

[mfb]

Concerning self-destruct buttons: They don't exist. There might be cases in the future where it could be interesting to have them, but at least for current spaceflight there is no such thing. Spacecraft do not have a mechanism to destroy themselves. Typically they do not even have a procedure to shut down everything: Such a command could be sent accidentally (or get induced accidentally by cosmic rays), and then the spacecraft would be lost.

Some satellites in Earth orbit keep some fuel to move out of their current orbits at the end of their lifetime - to make them re-enter the atmosphere faster (in low Earth orbits) or at least move out of the busy ring of geostationary orbits. But initiating that routine is a lot of effort, and it is not something you cannot stop if it starts unplanned.

 

[UsableThought]

Concerning self-destruct buttons: They don't exist. There might be cases in the future where it could be interesting to have them, but at least for current spaceflight there is no such thing. Spacecraft do not have a mechanism to destroy themselves.

A significant difference is that our spacecraft are not truly military, at least not yet.

The common sci-fi metaphor for space fleets is ships at sea - i.e. merchant ships and warships. So we can look at that to see what might be plausible in our imaginary future, when military spacecraft are developed & deployed. Historically, there has been much less motive to scuttle or otherwise destroy civilian or merchant ships; but it has been common in warfare. For whatever reason, I've been on a WWII "big warship" reading jag lately, so I have a couple of examples out of what must be many thousands.

The German pocket battleship Graf Spee was deliberately scuttled (and blown up to boot) by her captain, rather than let her be captured; and one of the operations by the British to try and take away potential safe harbors for the Tirpitz, sister vessel to the Bismarck, was to ram an obsolete destroyer full of high explosive (and commandos) into the gates of the enormous dry dock at the port of Normandie, France; the commandos off-loaded and began an attack, and sometime later the explosives detonated. The operation was actually pointless since the Germans never intended to move the Tirpitz there, but never mind.

Point is, in an imaginary future where it's practical to have fleets of military spaceships, being able to blow them up on occasion is a reasonable extension of what we know about warfare. Whether an auto-destruct would really be implemented as the means of doing so is another question. WWII warships could open the sea-cocks and set up the explosive they already carried for their shells to blow out the bulkheads and help the ship sink; they didn't need "auto-destruct."

As for the chance that space auto-destruct might be touched off accidentally, such a risk might be assessed differently for a warship than for a civilian craft - warships being higher-risk by nature, even when not engaged in battle. E.g. battleships from the European wars in the age of sail on through WWII and after have always had the big risk that the powder magazines could explode, even if not in a fight - filling cartridges in a sailing man o' war required placing the lamp (required since the magazine was below desk) behind special glass and metal panes, etc.; quite elaborate. And even just firing one of the big guns on a WWII ship during practice carried risk because of what is called "flash"; every single door involved had to be not just closed but hammered tight with metal clips. This was wholly outside the risk that an enemy shot could hit a magazine, as happened to Hood when she fought Bismarck.

Interestingly, but maybe not surprisingly, we don't see many malfunctions or other suggestions in sci fi movies that their hi tech weapons might be risky in and of themselves: "Uh, Emperor? You know the Death Star that was almost operationally ready? Well, they were running a little test, and uh, the new OS did a blue screen and . . . and we have to start over."

 

[Ryan_m_b]

A significant difference is that our spacecraft are not truly military, at least not yet.

The common sci-fi metaphor for space fleets is ships at sea - i.e. merchant ships and warships. So we can look at that to see what might be plausible in our imaginary future, when military spacecraft are developed & deployed.

Whilst your post was very interesting there's a disconnect here that the SF-pedant in me needs to bring up: just because most authors treat space as though it's an ocean doesn't mean that they are in any way correct, nor can we necessarily learn anything from looking at historical naval events/traditions/technologies.

You're still correct because it comes under the broader category of not wanting key technology to fall into the wrong hands. There has already been a history of designing space infrastructure to defend against that (hello Salyut 3) but given that very few countries have the ability to nick technology from space based facilities it's just not a thing. A more modern analogy might be drones which, I'm given to understand but googling turns up a hell of a lot of noise, are sometimes fitted with devices to prevent their reverse engineering in event of a crash.

Stepping back from logic for a bit and into the Star Trek universe: whilst the Enterprises, DS9, Voyager etc. might find themselves in extreme situations more than normal the fact is the STU is incredibly dangerous. It seems that barely a few days go by before some utterly bizarre alien or physical threat nearly takes over the ship in attempt to use it for malicious purposes. Rewatching TNG I once joked to my girlfriend about writing fanfic about all the other starfleet ships that weren't as lucky as the flagship. We quickly came to the depressing conclusion that Federation News must be even more horrific than real life, full of reports of colonies and ships getting possessed/infected/destroyed/eaten every other day.

 

[UsableThought]

just because most authors treat space as though it's an ocean doesn't mean that they are in any way correct,

"Correct"? We're talking fiction. Ain't no such thing as "correct"; there is only what some percentage of readers/viewers will enjoy; sci fi does not differ from other genres in this regard (see Booth's The Rhetoric of Fiction). Speaking for myself, and I suspect for at least some others, threads like this aren't about dissing sci fi for not not being "correct" in terms of science fact, but only for amusement, citing only that which is most implausible or most funny. If we really insisted on correctness we'd lose 99.99 percent of sci fi story-telling.

Nor am I saying a naval analogy is the only way to view a spacecraft military, or the only good way to view it; just a common way. I'd be curious what your own preference is and why the naval analogy bothers you so much?

nor can we necessarily learn anything from looking at historical naval events/traditions/technologies.

"Learn" anything? Again, what are we supposed to "learn"? We're still talking fiction with only mild excursions into what might be possible . . . in imagined futures that are all of them fantasy. Again, clearly you have some tastes or preferences in sci fi that you haven't yet articulated here; that's fine as you long as you understand others are free to have tastes or preferences that differ.

Beyond that, the only correspondence I suggested is one of motive, i.e. it is rather obvious that military craft will continue to have different priorities than civilian. I gave the naval examples of scuttling mostly because I think they're interesting in and of themselves. Certainly it's also interesting to consider "more modern analogies", e.g. drones; these may suggest new wrinkles for story-telling & engage new readers. Of course they too will date, but that's not the kiss of death; if we look back at old "hard" sci fi stories, they're full of dated references, but when the writing is good we still enjoy them.

Other than those two nit-picks I enjoyed your post. Yes, life aboard a Trek vessel is never dull, eh? Like any adventure serial, more happens than ever possibly could. Of course there were those French novelists back in the '50s or '60s, I forget which, who tried to write experimental novels in which nothing much happened, just like in real life; there's a good reason for why I can't remember more about them.

 

[Ryan_m_b]

"Correct"? We're talking fiction. Ain't no such thing as "correct"; there is only what some percentage of readers/viewers will enjoy; sci fi does not differ from other genres in this regard (see Booth's The Rhetoric of Fiction). Speaking for myself, and I suspect for at least some others, threads like this aren't about dissing sci fi for not not being "correct" in terms of science fact, but only for amusement, citing only that which is most implausible or most funny. If we really insisted on correctness we'd lose 99.99 percent of sci fi story-telling.

Nor am I saying a naval analogy is the only way to view a spacecraft military, or the only good way to view it; just a common way. I'd be curious what your own preference is and why the naval analogy bothers you so much?

We seem to be talking at cross purposes. If the aim is to explore how a theme works within the genre then there is little need to examine the basic assumption of the ocean analogy. But if the purpose is to look for a more realistic, and perhaps interesting, view of life in space then you have to take a step back. I'm not arguing that science fiction should be as realistic as possible, IMO all good science fiction and fantasy is consistent with reality, internal rules or theme to varying extent. And above all story matters. Bit this thread did start with the question "what really would happen" which implies an interest in realism.

"Learn" anything? Again, what are we supposed to "learn"? We're still talking fiction with only mild excursions into what might be possible . . . in imagined futures that are all of them fantasy. Again, clearly you have some tastes or preferences in sci fi that you haven't yet articulated here; that's fine as you long as you understand others are free to have tastes or preferences that differ.

"Learn" in the sense of draw novel conclusions. It's interesting you're making assumptions on my preferences (and assuming they are so limited) yet your stated intent with this thread was to enquire about realistic development of computer interfaces.

Beyond that, the only correspondence I suggested is one of motive, i.e. it is rather obvious that military craft will continue to have different priorities than civilian. I gave the naval examples of scuttling mostly because I think they're interesting in and of themselves. Certainly it's also interesting to consider "more modern analogies", e.g. drones; these may suggest new wrinkles for story-telling & engage new readers. Of course they too will date, but that's not the kiss of death; if we look back at old "hard" sci fi stories, they're full of dated references, but when the writing is good we still enjoy them.

I'd posit you're looking for a fight that was never offered. I'm not posting out of antagonism, merely pointing out that one can only take the naval analogy so far (i.e. the fact it's a military branch through which technology transfer to foreign powers would be undesirable).

Other than those two nit-picks I enjoyed your post. Yes, life aboard a Trek vessel is never dull, eh? Like any adventure serial, more happens than ever possibly could. Of course there were those French novelists back in the '50s or '60s, I forget which, who tried to write experimental novels in which nothing much happened, just like in real life; there's a good reason for why I can't remember more about them.

Again you seem to be taking a dig assuming that realism is fundamentally equivalent to a boring story. Aside from the myriad of genres for which the everday can be made exciting there are examples of science fiction in which the realistic, everyday story is the focus. Whether it be something as close to current day as The Martian or as personal as Star Ship Troopers/The Forever War that gave a convincing account of life-on-the-ground in an incomprehensible war.

 

[john101]

"In 2154, Romulans covertly conspired with V'Las, head of the Vulcan High Command, to invade Andoria. V'Las's Romulan contact had the stated agenda of reunification with the Vulcans.

A few months later, the Empire sent prototype holoships remote-controlled from Romulus to disrupt a peace conference between Andorians and Tellarites. The Romulans piloted the ships using an abducted Aenar; " Wiki.

The Aenar wore a suit and headgear with all sorts of sensors whereby he could control (in turn he was controlled by the Romulans) the holoships over vast distances.

I thought that was a pretty futuristic way to interface. (I believe that was a series of Enterprise episodes.)

 

[UsableThought]

I'd posit you're looking for a fight that was never offered.

I think where you stepped wrong - and I do think it was a wrong step - was here:

Whilst your post was very interesting there's a disconnect here that the SF-pedant in me needs to bring up: just because most authors treat space as though it's an ocean doesn't mean that they are in any way correct, nor can we necessarily learn anything from looking at historical naval events/traditions/technologies.

I'm very aware my response to you was prickly; but bringing up notions of "correctness" and implying someone else has been "incorrect" are a red flag in pretty much any discussion of fiction; and you didn't offer much in the way of qualification of that statement. Plus, as I argued pretty strongly, you misread my post just as much as you say I misread yours!

So maybe we should just agree we both misread each other - not uncommon.

On the other hand, I was also curious to hear more - your initial post didn't really lay out what you felt was important or an alternative to the old "naval" motif. I felt there might be something intriguing in your notions of "correct" & I did - and still do - want to find out more.

If you have time, maybe you could expand on this comment in particular: "But if the purpose is to look for a more realistic, and perhaps interesting, view of life in space then you have to take a step back." Your thought here seems similar to the old "hard" school, e.g. Larry Niven, etc. etc. - I think the Wikipedia article on this is decent, and in particular see the "scientific rigor" subtopic: https://en.wikipedia.org/wiki/Hard_science_fiction#Scientific_rigor

But I'm guessing what your'e saying isn't identical to previous "hard" views, either, since time moves on & the zeitgeist keeps getting updated. Have you written up your POV on this with reference to particular shows, novels, stories, or authors that serve as good examples - whether in a thread or comment on PhysicsForums.com, or elsewhere, e.g. some other forum or blog? If so do you have a link or two I can investigate? It needn't be restricted to any particular meme or trope - all is fair game. I'd also certainly be interested in anything that talks about interfaces, since as @mfb has pointed out, these are hit-or-miss when it comes to any notion of realism - mostly "miss".

 

[UsableThought]

On the "good" side of "computer interfaces of the future," there is the first Alien flick. I think most reviewers & audience members had a very positive response to the seeming authenticity of the interface for Nostromo's main computer, "Mother" - although the backstory to the computer was that it was a super-smart AI, the captain's interface in particular was quite clunky-looking - which to anyone with any historical knowledge of computers implied "built to be rugged, not to be fancy"; e.g. very much in line with the design philosophy for the computer installed on board the Space Shuttle, the IBM System/4 Pi AP-101. The navigation & piloting instrumentation and controls also seemed to have a rugged, hard-used quality; though I don't know how truly authentic they were, they certainly were a step above the total BS for controls depicted in the various Star Wars movies, especially the installments that came after the original trilogy - the filmakers in that case seemed to think that gloss or lustre was all that was wanted for interfaces (again, see the video in comment #3 above, right around the 9:50 mark).

Ridley Scott followed up this effort for plausibility or authenticity with his team's efforts on Prometheus. To be honest I really disliked the sum of that movie, for many reasons; but individual bits and pieces were very well done. I especially liked the exploration, via the android and his onboard studies while everyone else slept, of trying to guess how the unknown aliens might speak and how therefore the android might learn to interpret any glyphs of spoken speech the explorers came in contact with.

Here's what must have been a promo video for "Prometheus UI Graphics" - very pretty of course as the ship in that film was so brand-new it practically had a ribbon with a bow tied around it; not a semi-junker like Nostromo.

 

[UsableThought]

The Aenar wore a suit and headgear with all sorts of sensors whereby he could control (in turn he was controlled by the Romulans) the holoships over vast distances. I thought that was a pretty futuristic way to interface. (I believe that was a series of Enterprise episodes.)

Yes, good one. An extension of the gadgets we have already begun to use for gaming, right?

There was an episode in Next Generation literally titled "Interface," in which Jordi wore a special suit that allowed him to "see" through the eyes of a probe, and control the probe by moving his own body; the intention was to allow really facile investigation of environments that were too dangerous for an a person in a spacesuit. The story had him using the probe to investigate the drifting ruins of a spaceship. Of course there will silly complications to "heighten drama," e.g. if Jordi wore the probe suit too long, the level of "neural stimulation" would get too high and possibly cripple or kill him, yadda yadda.

http://memory-alpha.wikia.com/wiki/Interface_(episode)

 

[UsableThought]

if Jordi wore the probe suit too long, the level of "neural stimulation" would get too high and possibly cripple or kill him

The bit about "wearing the suit too long might cripple or kill him" reminds me of a meme that has emerged with many such computer interfaces that can temporarily be fit to a human body to enhance our senses or abilities: it is frequently the case that they are dangerous if used too much, or that they tempt us to pursue activities that are dangerous, etc.

For example the "squid" device that Ralph what's-his-name's character wears in Strange Days; the probe suit that Jordi wears in the NextGen episode mentioned above; and probably many others that aren't coming to mind just at this moment. Also, cyborgs in sci fi are typically depicted as having had to sacrifice the gentler or more "human" sides of their personalities to receive their implants, e.g. in a novel by Frank Herbert about genetic engineering of fetuses this was the case; don't remember the title.

What's especially interesting is that this meme goes back way, way before computer interfaces; it pretty much includes any transformation or shape-shifting that grants a person extra abilities; e.g. the episode in the X-Men comic book series where the Beast first acquires blue fur and extra strength via a serum - but finds he's abused the stuf & can't change back. And yes, this is Dr. Jekyll and Mr. Hyde; and there must be many, far older stories along these lines. Similarly in the Miyazaki anime Howl's[/PLAIN] Moving Castle, made from a young adult fantasy novel by Diana Wynne Jones, wizards who abuse their shape-shifting powers find they can't change back either.

A bit more like sci-fi is the 1969 novel The House on the Strand by Daphne du Maurier wherein a scientist (a biophysicist) has developed a special drug that allows a person to walk around in our time, while seeing every detail of the landscape, people, and events of a particular previous time in the 14th century. The drug damages the central nervous system if used too often; plus has other hazards, e.g. the scientist, while using the drug, walks onto a railroad track without seeing it and is killed by a train. And the ending implies the protagonist, having used the drug too often, is developing a paralysis.

But going back to @mfb's point about well-designed interfaces of today typically exerting constraints so the user doesn't totally mess up - you would think Jordi's suit would have a time-limiting factor built in, etc. But then where would the fun be? It's almost a riff on the old fear of exploring our Dionysian side too much.

 

[Rubidium_71]

Data's hands can dance with extreme rapidity over the console to execute zillions of commands

I always thought Data having to work a console was inefficient. Data should have had on board wi-fi. Soong was able to create an AI, but didn't equip it with wi-fi. There were many situations in the show where a wireless connection from Data would have been very helpful.

 

[UsableThought]

I always thought Data having to work a console was inefficient.

Yes, but they could show his fingers moving really quickly! Whereas with wi-fi we would have seen . . . nothing.

It did seem to me that Next Generation, along with countless other sci-fi shows/movies, seems to have very little in the way of what we used to call "cyborgs", even though in theory nanotech would make this easy to do. The Borg being a very awkward-seeming exception.

 

[GTOM]

Personally i think about neural interfaces reading brainwaves, already significant progress has been made. Since it is not like anyone could control a ship, they could have time to learn the individual user' s brainwaves.
Maybe injected contrast material that amplifies that the brainwaves.

 

[Rubidium_71]

Whereas with wi-fi we would have seen . . . nothing.

I disagree, we would've seen the display rapidly changing and being updated like when he set his very long password on the bridge in the episode "Brothers." The changing displays would have been more expensive to produce for the show, but it's the 24th century we're talking about. Even Lore upgraded himself for wi-fi (he controlled Data's emotion program wirelessly.)

 

[UsableThought]

I disagree, we would've seen the display rapidly changing and being updated like when he set his very long password on the bridge in the episode "Brothers." The changing displays would have been more expensive to produce for the show, but it's the 24th century we're talking about. Even Lore upgraded himself for wi-fi (he controlled Data's emotion program wirelessly.)

Yeah, you're right, something could have been done.

This also reminds me of the time when Lt. Barkley (in The Nth Degree), under the influence of a powerful alien intelligence-boosting program, has to solve an emergency that may blow up the ship; and gets very impatient with the existing keypad interface to the ship's computer being too slow to let him input the necessary commands. He gets the inspiration to find a way to hook his brain directly to the computer, in effect transferring parts of his consciousness into the machine.

He runs into the holodeck & starts telling the computer how to create the initial manual interface to initiate the transfer. This temporary interface includes . . . you guessed it . . . a keypad. Looks kind of like the one that I used to use for gaming on a PC, made by Belkin and oddly named after the ship in Alien.

And once he's done the transfer, the human-computer link is visually symbolized by blue laser beam lights, from somewhere high above where he's sitting, shooting back & forth from up there down to his cranium in a flickering diagonal pattern. Still not exactly wi-fi, but obviously meant to suggest something far more effective than a keypad.

 

[UsableThought]

And then of course there's the whole Ghost in the Shell manga/movie/etc. series from Japan. I'm a big fan of the first movie & to a lesser extent the second movie. In these & also in the various TV series, we're shown what must be wifi links, and also hardwire links; cyborg agents can use both types of links to access networks, talk to each other, etc. etc.

However, we're also shown quite a few shots of super-fast cyborg hands dancing over keyboards, people who've had their eyes replaced with microscopes, etc. These sorts of conceits are more shocking as they suggest dehumanization; whereas the wifi linkages seem more practical and somehow more likely.

 

[Rubidium_71]

I liked the Nth degree too, but I agree the need for a keypad was dubious.