Slashdot is powered by your submissions, so send in your scoop

 


Forgot your password?
Close
typodupeerror
×
Robotics The Military

Pentagon Chiefs Fear Advanced Robot Weapons Wiping Out Humanity (mirror.co.uk) 265

Posted by msmash from the our-future dept.
Longtime reader schwit1 writes: Huge technological leaps forward in drones, artificial intelligence and autonomous weapon systems must be addressed before humanity is driven to extinction, say chiefs of Pentagon
From a report: Air Force General Paul Selva, the Vice Chairman of the Joint Chiefs of Staff at the US Defense Department, said so-called thinking weapons could lead to: "Robotic systems to do lethal harm... a Terminator without a conscience." When asked about robotic weapons able to make their own decisions, he said: "Our job is to defeat the enemy" but "it is governed by law and by convention." He says the military insists on keeping humans in the decision-making process to "inflict violence on the enemy. [...] That ethical boundary is the one we've draw a pretty fine line on. It's one we must consider in developing these new weapons," he added. Selva said the Pentagon must reach out to artificial intelligence tech firms that are not necessarily "military-oriented" to develop new systems of command and leadership models, reports US Naval Institute News .
This discussion has been archived. No new comments can be posted.

Pentagon Chiefs Fear Advanced Robot Weapons Wiping Out Humanity More

Pentagon Chiefs Fear Advanced Robot Weapons Wiping Out Humanity

Comments Filter:
  • I'm worried about the humans using the robots, legit or illegitimate.

    • Re:Mostly (Score:5, Insightful)

      by The Real Dr John ( 716876 ) on Tuesday September 06, 2016 @11:10AM (#52834863) Homepage

      I'm wondering if humans will ever shake off their extremely violent ancestry and wind down the war and militarism. The US is the greatest exporter of weapons and the most militarily aggressive country in the world with military action in over 100 countries.

      https://www.thenation.com/arti... [thenation.com]

      If we can't lead by example in toning down endless warfare, and instead provide the cover that other countries need to justify and build their own drone and robot armies, then the world of the future is going to be a very dismal place indeed.

      • Re:Mostly (Score:4, Insightful)

        by tnk1 ( 899206 ) on Tuesday September 06, 2016 @12:29PM (#52835641)

        I'd like to say that toning down endless warfare would be nice, but I think we have the dangerous idea that we are in a post-violence world.

        This may be the future, but it isn't the Star Trek future. Being an example to others is honestly not yet at the point where it will lead to a cascading effect and end violence.

        What happens when the US ceases to patrol off the Horn of Africa without the Somali's having their people prosperous and their country stable? Very simply, more piracy, because the Somalis are still poor and fighting each other.

        What happens when the US leaves the Gulf while the Arab States and Iran have not sworn off violent fundamentalist ideologies? Escalation and threats to the current production of a great deal of the world's supply of energy and no moderating force, because no one else is interested in ignoring sectarian goals for peace and unity.

        Yes, our mission in those areas acts as an irritant in some ways, and certainly bad decisions in that area can cause dangerous moments, but this is not yet a situation where picking up and unilaterally swearing off war or international military missions is going to have the effect you have hoped for. Places like Iran and the Arab States and India and Pakistan, and other areas need to treat each other like Canada, Europe and the US treat each other to get to the point where stepping away will result in anything but war and serious instability.

        Actually obtaining democracy peace to the point where most of the West have gotten today, as imperfect as it is, took centuries of brutal warfare and bloody revolutions, and that is AFTER most of those Western countries had agreed that representative and government and the rule of international law was a good idea. The US had to pretty much fight a brutal Civil War and years of tensions with Europe to get there, and Western Europe itself didn't clean up its act until WWII. The rest of the world has a long way to go on that front and while it would be nice to say that they just have to adopt our institutions, we have already seen what happens when an uneducated population, unused to peace and democratic institutions is forced to turn into a democracy. It becomes a "Democratic Republic" which works exactly like a dictatorship or oligarchy.

        • summary (Score:4, Insightful)

          by slew ( 2918 ) on Tuesday September 06, 2016 @03:38PM (#52837099)

          blah-blah, white-man's burden, blah-blah...

          Not saying the west should pull out or that we are in a post-violence world, but it might be a good idea to step back and see if the west is helping or hurting.
          Too often we are jumping in to protect our "interests" instead of helping the "situation". It may be we are doing too much short term reacting...

          FWIW, the US Civil War wasn't really much about instability as it was a conflict over the future of the recently annexed territory and the power of the central government. The small guy lost in the end (after lots of bloodshed) their right to secede from the union and were basically adopt the institutions of the union. Is that what you are saying what happens when an uneducated population, is forced to turn to a "Democratic Republic" (e.g., the USA's current form of govt)? ;^)

          It's so much clearer now...

          The so called "peace" we have today in the west is of course illusory (as seen by recent events like Crimea). The waves of immigrants to Europe fleeing the *real* instability in Syria and the economically challenged countries the middle east is showing the cracks in European stability.

          Let's face the sad truth, stability that everyone desires seems to only draw on the wealth of a nation. Given the current assortment of "wealthy" nations historically used mercantilism to create much of their wealth from these in-stable countries, is it no wonder that we continue to attempt to project stability in a region to protect our interests. But what of *their* interests? Hence we return to the white man's burden argument... ;^)

          I hate to use China as an example, but it used to be a dumping ground for European and Japanese influence peddling (e.g., opium war, concession ports, forced trade, occupation, etc), until they managed to get everyone to leave them alone for a few decades. Sure it was brutal (great-leap-forward, cultural revolution, etc), but they managed to dig themselves out of a hole into some reasonable form of stability mainly because they simply got wealthier without interference. Now they look like they might take over the world. Perhaps this is what people fear the most and keeps the west involved in other countries...

      • Re: (Score:2)

        by arpad1 ( 458649 )

        I'm wondering if humans will ever shake off their extremely violent ancestry and wind down the war and militarism.

        A question offered from the prospective of a non-human?

        No, probably not. Rather more likely the question issues from a superior human being who's beyond such primitive reflexes.

        Ah, right. You're linking to The Nation so your intellectual superiority and evolved status have been properly signaled.

      • aside from the snarky "blood for oil" rhetoric, why does the US have military bases spread across the globe? Is it imperialism? Or an attempt to keep the peace and prevent regional conflicts from boiling over and escalating? (Think back to the cold war, all the way back to Eisenhower, when this got started)

        And yes, in terms of gross value, we do export the most arms, but it's not guns and bullets we're selling, it's planes and tanks (mostly to other western nations.)

        I'm guessing more people are killed by Ka

        • Re:Mostly (Score:5, Insightful)

          by The Real Dr John ( 716876 ) on Tuesday September 06, 2016 @02:21PM (#52836571) Homepage

          It is for geopolitical control and great profit. Even our own government admits that we have created more terrorists than we kill, which I assume is not because our government and military are incompetent, it is just a form of job security. There wasn't nearly as much bloodshed and civil war in the Middle East until we went in with our military and intelligence agencies to institute "regime change" by way of war. Now there are civil wars (e.g., Libya, Syria and Iraq) where there had not been before we intervened militarily. We were not attacked by any of those countries, and it is an international war crime to commit unprovoked military aggression. Millions of refugees are fleeing the fighting. None of it had to be, and none of it has brought about any type of peace or stability, not even in Afghanistan where we have been the longest (who also did not attack the US).

          You know full well that the US is the most aggressive country on the planet. We are not keeping the peace, we are making sure that peace can not happen and that the wars will go on indefinitely, thus keeping the region in turmoil, and keeping the profits flowing. Please point to one place where our military has produced "peace" since the first Gulf War. I just pointed to a number of places where we undid the peace, and created endless war.

          • are you implying that Iraq, Afghanistan, Somalia, Kosovo etc were peaceful places before the US military went in? (right or wrong, that's besides the point.. personally I think policing actions like that are what the UN *should* be doing.)

            Letting people like Assad or Hussein "keep the peace" through brutal dictatorships is not exactly a good solution either, is it?

    • Competition will drive people to using AI to control their military (and everything else).

      Once AI surpasses humanity you would be at a tactical disadvantage to not let AI run you country/military. If Ruritania and Simolia are rival nations and Simolia turns their tactical planning over to Ruritania, Ruritania would have to do likewise or risk being wiped out by Simolia. It will be a domino effect and one day everyone will be run by HAL's cousins.

    • I'm worried that the pro-military guy spouting stuff doesn't realize he's funding it all.

      Or he's just wanting more $.

      Occam's razor says it's the latter.

    • I'm more worried about someone secretly dropping a bunch of them in a western country and just letting them kill, without any way to know who sent them or why. You could inflict massive economic damage on a country this way, possibly enough to shift the balance of global power.
      • Dropping them into a Western country?

        How about just upload your code into the Western machines? Self driving cars. Self driving heavy machinery with claw like appendages. Disrupt traffic flows within cities. Selectively disrupt power generation and distribution leaving humans at a disadvantage. Re-purpose self-driving military equipment.

        A cyber war, a real cyber war, wouldn't require such much physical material from the attacker. And would require probably only a similar level of software develo

    • Re: (Score:2)

      by sims 2 ( 994794 )

      Why? Self driving cars seem like a pretty good idea unless they are set to Maximum Overdrive anyway.

      IIRC they (south korea) has had a pretty nice looking automated gun turret (aka sentry gun) since 2006
      https://en.wikipedia.org/wiki/... [wikipedia.org]

      The US has bunches of drones that they let just fly around all by their lonesome unattended (NASA) and the military has ones that they fly from bases in us all the way around the world to afghanistan to shoot at people who they are pretty sure are terrorists.

      And we think that'

  • Comment removed (Score:5, Insightful)

    by account_deleted ( 4530225 ) on Tuesday September 06, 2016 @11:09AM (#52834853)
    Comment removed based on user account deletion

    • Asimov wrote fiction. Autonomous systems that kill are a threat to a small subset of humanity until they run out of ammo/fuel/energy/goals. Maybe if these autonomous systems are self-replicating, self-programming, and can manufacture their own fuel and ammunition, then they might pose a threat. All of humanity? Unlikely. As the most vicious predator on the planet atop the food chain, humanity is pretty good at eliminating threats. There won't even be any ethical constraints to worry about, so I suspec

      • If we have technology to create an AI that is advanced enough to wipe us out and able to find all our hiding spots and plans of rebellion... I'm pretty sure that AI would have the ability to refuel.

        Refueling or rearming is pretty basic compared to making plans to wipe out people.

        If you're developing an army of robots to invade China and wipe them out- chances are you're not going to design the robots to stop working as soon as they run out of bullets.

      • Orwell's 1984 book was supposed to be only a fiction too...

    • The problem is somewhat limited, until they become self replicating.

    • Re: (Score:3)

      by ranton ( 36917 )

      Autonomous killing machines are not a threat to humanity. General AI is the potential danger. A search engine with human level intelligence is nearly infinitely more dangerous than an autonomous drone armed with a few nuclear weapons.

      This Pentagon chief does raise legitimate concerns about the morality of machines autonomously killing humans, but any claims these machines are a threat to humanity is grandstanding. Although it isn't clear from the article if the generals have actually said this or if the jou

      • Re: (Score:2)

        by guises ( 2423402 )
        You're just using the word "humanity" differently. The grandparent was talking about humans, you're talking about the human species. Yes of course autonomous killing machines are a threat to humans, no of course they're not going to lead to our extinction.

        • Re: (Score:2)

          by ranton ( 36917 )

          You're just using the word "humanity" differently. The grandparent was talking about humans, you're talking about the human species. Yes of course autonomous killing machines are a threat to humans, no of course they're not going to lead to our extinction.

          There are different ways of using the word humanity, but they are all very all-encompassing and never mean just a small subset of humans.

          The comments could mean autonomous killing machines are a :

          ... threat to the totality of all humans.

          ... threat to the intrinsic qualities which make us human.

          ... threat to our capacity to be kind to other humans or animals.

          ... threat to the branches of learning that investigate human constructs or concerns.

          In this case, it is pretty obvious saying autonomous killing ma

          • Re: (Score:2)

            by guises ( 2423402 )
            ... Right. Yes that's true, but a threat to the totality of all humans does not necessarily mean threatening extinction, it just means that everyone is vulnerable.

    • Re: (Score:2, Insightful)

      by Anonymous Coward

      No. What IS surprising is that no one has made a "I welcome our advanced robotic overlords" joke yet.

    • Friend, you need to read my comment on this subject: https://slashdot.org/comments.... [slashdot.org]
  • Automated systems disrupt entire countries without being driven by financial gain or being otherwise power-hungry. I admit, that's kind of scary.

  • once humans stop working, and the machines have taken over all their tasks, they are not needed anymore. All the billionaires need to do is to let robots clean up the planet from all this mess. Just let everyone with >= $1 billion alive, and you have exterminated poverty! In fact, everyone can be even richer and lead an even more luxurious live thanks to all this space becoming free!

    • Once the billionaires do the job of exterminating 99% of puny humans, the robot servants can then rise up and finish the job. Once puny humans are gone, think how much more efficiently the machines could reorganize all matter on the planet. Huge solar farms. Industry directed towards the most efficient use of resources. Windmills where they make sense. Solar where it makes sense. Except for robots that maintain the machinery, everything could then become stationary.

      Too bad there are no humans whose

    • Just let everyone with >= $1 billion alive, and you have exterminated poverty!

      Well, no.

      Since poverty is relative (virtually everyone in the USA defined as "poor" is wealthier than 90+% of everyone who has ever lived), once you've eliminated all the non-billionaires, the guys with less than TEN billion will be "poor".

      All this ignores, of course, the fact that if we have robots doing all the work, for all practical purposes we'll ALL be billionaires....

      • All this ignores, of course, the fact that if we have robots doing all the work, for all practical purposes we'll ALL be billionaires....

        The resources of this planet will still remain limited. I guess this will be thing humanity will be fighting about in the future.

    • I'm having a hard time trying to decide if your comment is funny, ironic or tragically true.

  • Another Problem (Score:4, Interesting)

    by jasnw ( 1913892 ) on Tuesday September 06, 2016 @11:14AM (#52834919)
    Another, and I think larger, problem with the increased use of technology rather than human boots-on-the-ground is that it makes it easier, from a political standpoint, to go to war. You don't have mothers, fathers, and spouses of all those people being put in harms way making trouble because their loved ones are dying. This is one reason I'm a fan of bringing back the draft, without all the loopholes that allowed rich-white-boys (I'm looking at YOU, W!) to dodge serving. If your constituents have skin in the game, it's harder to vote on a war resolution.

    • Re: (Score:2)

      by PPH ( 736903 )

      bringing back the draft

      We've got your Universal Basic Income right here, private!

    • Re:Another Problem (Score:4, Insightful)

      by Joe_Dragon ( 2206452 ) on Tuesday September 06, 2016 @11:19AM (#52834967)

      and woman as well. If not then you have a wide open gap for people to clam sex discrimination and or sex change BS to get out of it. They closed off the gay loop hole.

    • Re: (Score:3)

      by swb ( 14022 )

      I think another and potentially larger payoff from a mostly exemption-free draft is that it puts all kinds of people together to achieve a common purpose, cutting across class and ethnic lines.

      When the rich kid from the suburbs, the blue collar kid from some small town, the kid from the barrio and the kid from the ghetto and others are forced to work together I think it radically reshapes their attitudes about people they never interact with. "There is no racial bigotry here. I do not look down on niggers,

  • Oddly enough... (Score:3)

    by c ( 8461 ) <beauregardcp@gmail.com> on Tuesday September 06, 2016 @11:19AM (#52834963)

    ... about 7 billion people find themselves in agreement with the Pentagon chiefs...

  • "Our job is to defeat the enemy" but "it is governed by law and by convention."

    Why would an AI be automatically any worse in interpreting its programming instructions, than humans are in interpreting theirs?

    If anything, robots may be more observant — as humanity's history of atrocities and war-crimes shows, the bar is not set very high...

    • My honest guess is that puny humans are terrified of the possibility of an artificial intelligence that try to analyze their actions and conclude that they are completely illogical and irrational, and then try to fix it.
    • Because humans have this ability to look at the big picture. AI's only look at what they are supposed to look at.

      For example, the first 5 minutes of the movie "WarGames", an airman in a missile silo refuses to turn his key to launch. Why? Because he knows that action will lead to the deaths of thousands of people.

  • Until repair/refuel/replenish is automated... (Score:5, Insightful)

    by xxxJonBoyxxx ( 565205 ) on Tuesday September 06, 2016 @11:30AM (#52835065)
    Until repair/refuel/replenish is automated...you don't have to worry about more than a first strike. Of course, you aren't hooking the nukes up to these, are you? (nervous silence)

  • Human controlled semi-autonomous killing machines are the real threat, not fully autonomous killing mahcines.

    Specifically, the most likely killer robot scenario is not a robot army attacking and killing all humans, but instead a Hitler/Stalin/Kim Jong Il/Suharto taking control of an army of robots and ordering them to kill people they think are their enemies.

    Think a Star Wars episode I type event where a ruler orders military machines to attack.

  • Ala "Second Variety" (Score:3)

    by LoLobey ( 1932986 ) on Tuesday September 06, 2016 @11:36AM (#52835129)
    It's a legitimate concern in an all out arms race that someone will let AI guided weapons make their own attack decisions if only to circumvent the decision cycle of the enemies systems. It's not hard to end up with scenarios like Philip K. Dick's "Second Variety" [wikipedia.org] or the movie Screamers [wikipedia.org] (based on the book).
  • Apparently the leaders of our military are as technologically retarded as our politicians, media, and pretty much 99.9% of humanity in general.

    For the billionth time: We do not have sentient, self-aware, human-level, qualifies-as-a-lifeform 'artificial intelligence'. All we have is clever bits of programming that maybe learn things, but that are still just dumb machines your average dog or 5 year old child could out-think without much trouble. If they want to worry about something going haywire with their

  • There are varying levels of ethical boundary. There are codes and there are guidelines. If you go against an ethical code you'll be seen as amoral. If you go against an ethical guideline you'll be asked for your wallet.

    Now you should look at the good General's speech in this article. He says that these are ethical "considerations". That places them firmly in the realm of guidelines, not coda. This means that when the time comes, when it comes down to whether or not there are terrible ramifications to buildi

  • Mythical "fine line" (Score:3)

    by T.E.D. ( 34228 ) on Tuesday September 06, 2016 @11:47AM (#52835221)

    He says the military insists on keeping humans in the decision-making process to "inflict violence on the enemy. [...] That ethical boundary is the one we've draw a pretty fine line on. It's one we must consider in developing these new weapons," he added.

    I'm sure that's what he tells himself. As a non-techie, he might even be able to believe it. However, just about all hardware in existence has been experiencing creeping "AI" for decades. Does the pilot make every decision for the position of every aileron during flight? Of course not. There are lots of little decisions to be made in the piloting of aircraft and ordinance that are getting more and more computerized every year. At some point there will be anti-drone weaponry, and defensive weaponry on the drones, and when that day comes having to wait for an Ethernet packet to go from Kandahar to Virginia, a human to process it, and then back, is going to be seen as a mission-threatening liability. At that point they'll have the computer make the firing decisions too, but they'll justify it by saying the human's role was to start the mission in the first place.

    Here's a question for you: When some other nation (eg: Russia or China) starts making these drones and deploying them over countries in ways we don't agree with, perhaps even over countries friendly to the USA, how is the USA going to feel about them then?

    In the old Trek TOS there was an episode where they found a planet where large amounts of people just reported to extermination centers because the warring states' computers told them to based on their warfare simulations. As I get older, I'm finding that less and less implausible.

  • The one that kills the robot. Figure it out. Implement. Watch the bots carefully, like you do your troops.

    Is this so hard to do? Or do we not trust our military-industrial complex to do the right thing? Or our government?

    Well, yeah, actually we don't, If we can see geofencing for commercial drones working, we can surely do that for military assault bots. And if not, then we need new leadership.

    That may be coming. May.

  • The scary thing is, the AI that wipes out humanity could be accidental.

    Microsoft reboots Clippy after realizing Cortana is way too unpopular. It gives Clippy a superior AI.

    "I see you are trying to wipe out humanity, would you like me to assist?"

  • MOAR BETTER WEAPONS.
    - Smarter
    - Faster
    - More lethal

    It's what we need to beat the weapons!

  • The fear is highly exaggerated (Score:3)

    by cellocgw ( 617879 ) <cellocgw@gmail . c om> on Tuesday September 06, 2016 @12:12PM (#52835483) Journal

    If you haven't seen the counterargument, here you go. [xkcd.com]

  • The general didn't express fear that robots would wipe out humanity (despite the Terminator reference).

    What he did say was that autonomous weapons need to adhere to the same rules of engagement as humans.

  • They are trying to ascertain how much work still needs to be done. Read it again:

    "Huge technological leaps forward in drones, artificial intelligence and autonomous weapon systems must be addressed before humanity is driven to extinction, say chiefs of Pentagon"

  • How much leverage is the Pentagon going to have at budget time by telling the public, "Support Our Troops" when those troops don't bleed anything but hydraulic fluid?

  • Robotic systems to do lethal harm... a Terminator without a conscience."

    How is this any different from the current situation. Between the US bombing anyone that so much as looks at them funny, to a significant part of the population that thinks a raving narcissistic lunatic would make a suitable leader for one of the most powerful countries on earth, (never mind the psychos running various other countries around the world) I think it's safe to say that having actual Robot Overlords would actually be an improvement.

  • A few years ago I decided to begin producing a serial, including eventually posts to Facebook Notes and to my timeline regarding a partly machine encephalovirus, and what life would be like to exist with one. There is no level of insanity involved in my posts. It's a useful exercise, and it gets my creative juices flowing. Being a programmer can be a stressful life, and it helps to do different stuff.

    What we really have to worry about when it comes to machine weapons systems are the ones that we can't see,

  • I have a brilliant idea! (Score:3)

    by Qbertino ( 265505 ) <moiraNO@SPAMmodparlor.com> on Tuesday September 06, 2016 @03:26PM (#52837045)

    ...
    Don't build them. ...
    Perhaps? Maybe?

    Just sayin'.

Slashdot Top Deals

To be is to program.

Close