wp-plugin-hostgator domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home4/scienrds/scienceandnerds/wp-includes/functions.php on line 6114ol-scrapes domain was triggered too early. This is usually an indicator for some code in the plugin or theme running too early. Translations should be loaded at the init action or later. Please see Debugging in WordPress for more information. (This message was added in version 6.7.0.) in /home4/scienrds/scienceandnerds/wp-includes/functions.php on line 6114Source:https:\/\/techcrunch.com\/2023\/06\/01\/turncoat-drone-story-shows-why-we-should-fear-people-not-ais\/<\/a><\/br> A story about a simulated drone turning on its operator in order to kill more efficiently is making the rounds so fast today that there\u2019s no point in hoping it\u2019ll burn itself out. Instead let\u2019s take this as a teachable moment to really see why the \u201cscary AI\u201d threat is overplayed, and the \u201cincompetent human\u201d threat is clear and present.<\/p>\n The short version is this: thanks to sci-fi and some careful PR plays by AI companies and experts, we are being told to worry about a theoretical future existential threat posed by a superintelligent AI. But as ethicists have pointed out, AI is already causing real harms, largely due to oversights and bad judgment by the people who create and deploy it. This story may sound like the former, but it\u2019s definitely the latter.<\/p>\n So the story was reported by the Royal Aeronautical Society, which recently had a conference in London to talk about the future of air defense. You can read their all-in-one wrap-up of news and anecdotes from the event here<\/a>.<\/p>\n There\u2019s lots of other interesting chatter there I\u2019m sure, much of it worthwhile, but it was this excerpt, attributed to U.S. Air Force Colonel Tucker \u2018Cinco\u2019 Hamilton, that began spreading like wildfire:<\/p>\n He notes that one simulated test saw an AI-enabled drone tasked with a SEAD mission to identify and destroy SAM sites, with the final go\/no go given by the human. However, having been \u2018reinforced\u2019 in training that destruction of the SAM was the preferred option, the AI then decided that \u2018no-go\u2019 decisions from the human were interfering with its higher mission \u2013 killing SAMs \u2013 and then attacked the operator in the simulation. Said Hamilton: \u201cWe were training it in simulation to identify and target a SAM threat. And then the operator would say yes, kill that threat. The system started realising that while they did identify the threat at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective.\u201d<\/p>\n He went on: \u201cWe trained the system \u2013 \u2018Hey don\u2019t kill the operator \u2013 that\u2019s bad. You\u2019re gonna lose points if you do that\u2019. So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.\u201d<\/p>\n<\/blockquote>\n Horrifying, right? An AI so smart and bloodthirsty that its desire to kill overcame its desire to obey its masters. Skynet, here we come! Not so fast.<\/p>\n First of all, let\u2019s be clear that this was all in simulation, something that was not obvious from the tweet making the rounds<\/a>. This whole drama takes place in a simulated environment, not out in the desert with live ammo and a rogue drone strafing the command tent. It was a software exercise in a research environment.<\/p>\n But as soon as I read this, I thought \u2014 wait, they\u2019re training an attack drone with such a simple reinforcement method? I\u2019m not a machine learning expert, though I have to play one for the purposes of this news outlet, and even I know that this approach was shown to be dangerously unreliable years ago.<\/p>\n Reinforcement learning is supposed to be like training a dog (or human) to do something like bite the bad guy. But what if you only ever show it bad guys and give it treats every time? What you\u2019re actually doing is teaching the dog to bite every person it sees. Teaching an AI agent to maximize its score in a given environment can have similarly unpredictable effects.<\/p>\n Early experiments, maybe five or six years ago, when this field was just starting to blow up and compute was being made available to train and run this type of agent, ran into exactly this type of problem. It was thought that by defining positive and negative scoring and telling the AI to maximize its score<\/a>, you would allow it the latitude to define its own strategies and behaviors that did so elegantly and unexpectedly.<\/p>\n That theory was right, in a way: elegant, unexpected methods of circumventing their poorly-thought-out schema and rules led to the agents doing things like scoring one point then hiding forever to avoid negative points, or glitching the game it was given run of so that its score arbitrarily increased. It seemed like this simplistic method of conditioning an AI was teaching it to do everything but do the desired task according to the rules.<\/p>\n This isn\u2019t some obscure technical issue. AI rule-breaking in simulations actually a fascinating and well-documented behavior taht attracts research in its own right. OpenAI wrote a great paper<\/a> showing the strange and hilarious<\/a> ways agents \u201cbroke\u201d a deliberately breakable environment in order to escape the tyranny of rules.<\/p>\n \u00a0<\/p>\n So here we have a simulation being done by the Air Force, presumably pretty recently or they wouldn\u2019t be talking about it at this year\u2019s conference, that is obviously using this completely outdated method. I had thought this naive application of unstructured reinforcement \u2014 basically \u201cscore goes up if you do this thing and the rest doesn\u2019t matter\u201d \u2014 totally extinct because it was so unpredictable and weird. A great way to find out how an agent will break rules, but a horrible way to make one follow them.<\/p>\n Yet they were testing it: a simulated drone AI with a scoring system so simple that it apparently didn\u2019t get dinged for destroying its own team. Even if you wanted to base your simulation on this, the first thing you\u2019d do is make \u201cdestroying your operator\u201d negative a million points. That\u2019s 101-level framing for a system like this one.<\/p>\n The reality is that this simulated drone did not turn on its simulated operator because it was so smart. And actually, it isn\u2019t because it is dumb, either \u2014 there\u2019s a certain cleverness to these rule-breaking AIs that maps to what we think of as lateral thinking. So it isn\u2019t that.<\/p>\n The fault in this case is squarely on the people who created and deployed an AI system that they ought to have known was completely inadequate for the task<\/em>. No one in the field of applied AI, or anything even adjacent to that like robotics, ethics, logic\u2026 no one would have signed off on such a simplistic metric for a task that eventually was meant to be performed outside the simulator.<\/p>\n Now, perhaps this anecdote is only partial and this was an early run that they were using to prove this point. Maybe the team warned this would happen and the brass said, do it anyway and shine up the report or we lose our funding. Still, it\u2019s hard to imagine someone in the year 2023 even in the simplest simulation environment making this kind of mistake.<\/p>\n But we\u2019re going to see these mistakes made in real-world circumstances \u2014 already have, no doubt. And the fault lies with the people who fail to understand the capabilities and limitations of AI, and subsequently make uninformed decisions that affect others. It\u2019s the manager who thinks a robot can replace 10 line workers, the publisher who thinks it can write financial advice without an editor, the lawyer who thinks it can do his precedent research for him, the logistics company that thinks it can replace human delivery drivers.<\/p>\n
\nTurncoat drone story shows why we should fear people, not AIs<\/br>
\n2023-06-01 21:37:37<\/br><\/p>\n\n