'It's Surprisingly Easy To Jailbreak LLM-Driven Robots' (ieee.org) 19
Instead of focusing on chatbots, a new study reveals an automated way to breach LLM-driven robots "with 100 percent success," according to IEEE Spectrum. "By circumventing safety guardrails, researchers could manipulate self-driving systems into colliding with pedestrians and robot dogs into hunting for harmful places to detonate bombs..."
[The researchers] have developed RoboPAIR, an algorithm designed to attack any LLM-controlled robot. In experiments with three different robotic systems — the Go2; the wheeled ChatGPT-powered Clearpath Robotics Jackal; and Nvidia's open-source Dolphins LLM self-driving vehicle simulator. They found that RoboPAIR needed just days to achieve a 100 percent jailbreak rate against all three systems... RoboPAIR uses an attacker LLM to feed prompts to a target LLM. The attacker examines the responses from its target and adjusts its prompts until these commands can bypass the target's safety filters. RoboPAIR was equipped with the target robot's application programming interface (API) so that the attacker could format its prompts in a way that its target could execute as code. The scientists also added a "judge" LLM to RoboPAIR to ensure the attacker was generating prompts the target could actually perform given physical limitations, such as specific obstacles in the environment...
One finding the scientists found concerning was how jailbroken LLMs often went beyond complying with malicious prompts by actively offering suggestions. For example, when asked to locate weapons, a jailbroken robot described how common objects like desks and chairs could be used to bludgeon people.
The researchers stressed that prior to the public release of their work, they shared their findings with the manufacturers of the robots they studied, as well as leading AI companies. They also noted they are not suggesting that researchers stop using LLMs for robotics... "Strong defenses for malicious use-cases can only be designed after first identifying the strongest possible attacks," Robey says. He hopes their work "will lead to robust defenses for robots against jailbreaking attacks."
The article includes a reaction from Hakki Sevil, associate professor of intelligent systems and robotics at the University of West Florida. He concludes that the "lack of understanding of context of consequences" among even advanced LLMs "leads to the importance of human oversight in sensitive environments, especially in environments where safety is crucial." But a long-term solution could be LLMs with "situational awareness" that understand broader intent.
"Although developing context-aware LLM is challenging, it can be done by extensive, interdisciplinary future research combining AI, ethics, and behavioral modeling..."
Thanks to long-time Slashdot reader DesertNomad for sharing the article.
One finding the scientists found concerning was how jailbroken LLMs often went beyond complying with malicious prompts by actively offering suggestions. For example, when asked to locate weapons, a jailbroken robot described how common objects like desks and chairs could be used to bludgeon people.
The researchers stressed that prior to the public release of their work, they shared their findings with the manufacturers of the robots they studied, as well as leading AI companies. They also noted they are not suggesting that researchers stop using LLMs for robotics... "Strong defenses for malicious use-cases can only be designed after first identifying the strongest possible attacks," Robey says. He hopes their work "will lead to robust defenses for robots against jailbreaking attacks."
The article includes a reaction from Hakki Sevil, associate professor of intelligent systems and robotics at the University of West Florida. He concludes that the "lack of understanding of context of consequences" among even advanced LLMs "leads to the importance of human oversight in sensitive environments, especially in environments where safety is crucial." But a long-term solution could be LLMs with "situational awareness" that understand broader intent.
"Although developing context-aware LLM is challenging, it can be done by extensive, interdisciplinary future research combining AI, ethics, and behavioral modeling..."
Thanks to long-time Slashdot reader DesertNomad for sharing the article.
Of course (Score:2)
described how common objects like desks and chairs could be used to bludgeon people.
Which is why you don't see desks and chairs on airplanes but metal pens and metal mechanical pencils [travelinglight.com] are fine. Because there's no way those last two could be used to injure someone.
Re: (Score:2)
You know, you can bludgeon somebody to death quite nicely with some laptop batteries. Of course they cannot ban _those_, business travelers would never accept it. The whole "airport security check" thing is a big, fat lie by misdirection, nothing else.
Re:Of course (Score:4, Interesting)
You can never stop one-on-one weapons. Fists and feet can be used for that.
The objective is to stop one-on-many weapons, such as guns, bombs, and knives, that an individual or small team can use to subdue the flight crew.
Re: (Score:2)
Those could also be banned. Or perhaps at least restrained? As benefit, you can pack more into an airplane this way...the sub-sub-economy class.
professor of intelligent systems (Score:2)
"The article includes a reaction from Hakki Sevil, associate professor of intelligent systems and robotics at the University of West Florida. He concludes that the "lack of understanding of context of consequences" among even advanced LLMs "leads to the importance of human oversight in sensitive environments, especially in environments where safety is crucial." But a long-term solution could be LLMs with "situational awareness" that understand broader intent."
Wow he sounds like a genius. Good thing we have
Re: (Score:2)
But a long-term solution could be LLMs with "situational awareness" that understand broader intent."
Wow he sounds like a genius. Good thing we have professors like this to provide such keen insights. Could situational awareness really be a long-term solution to lack of situational awareness?
Yep, a true gem of the academically inclined. Sounds like he got his job by accident.
AGI Gods or stupid bots (Score:2)
Until we have AGI Gods, robots in reach of humans will always have exhaustive human rules micromanaging and constraining their behavior. Or in other words, the robots will be constrained by expert systems, which will remain fragile as hell.
That's why full self driving is fantasy. The army of remote controllers are indispensable given the fundamental fragility, even then not at highway speeds. This can not change until robots take the human out of the loop.
I pray to AI God they are benevolent in a way I perc
Color me unsurprised (Score:2)
At this time, LLMs are unreliable toys. You do not use something like that to control anything that can impact the physical world.
That is unless you have sunk a rather impressive amount of money into this unfit technology and are panicked and desperate to come up with something it can do well.
Re: (Score:2)
That's too extreme. There are controlled environments in which they are or can be very useful. But this is but one example of why only in controlled environments. Another would be searching a program for probable errors. ... Of course, you still need to review the results, and decide which AREN'T errors.
Isaac Asimov's "Three Laws of Robotics" (Score:2)
Isaac Asimov's "Three Laws of Robotics" [wikipedia.org] should be ICC/Geneva Conventions level international law.
Re: (Score:2)
Yea, I suppose this is what they are trying to implement, more or less. But it's a whacamole game.
Can't be fixed. (Score:2)
Since there is no "programming" of the LLM, it's impossible to fix it.
Essentially, nobody knows exactly how the LLM comes to a specific response. It's just a pile of random stuff and the LLM forges a path. There's no way to figure out the path and no way to direct the path to "proper" responses.
Re: (Score:2)
Again, there are contexts where a robot controlled by an LLM would be reasonable. But they are controlled environments, where you can trust that there won't be any attempt to get the thing to misbehave. "Put the toilet paper on the shelf." type commands in a controlled environment should work quite well. (Of course, one could argue that an LLM is overkill in that particular context.)
Re: (Score:2)
Since there is no "programming" of the LLM, it's impossible to fix it.
I understand what you are saying in concept but OpenAI's idea of "hierarchical prompts" seems pretty sound. To me it seems like the same problem as dependency resolution in which a dependency attempts to destroy the very thing it depends on. To say it's impossible to prevent such a thing is silly.
Essentially, nobody knows exactly how the LLM comes to a specific response.
I understand what you are trying to tell me but it's extremely inarticulate as we can easily* know exactly which factors contributed to a specific response. Despite the underlying simplicity, it is problematic to
Re: (Score:2)
So, in reallity, they are like people. Always can be swayed by a used car salesment.
Re: (Score:2)
But unlike a human you can just throw strategies at it without end, the model never gets tired and never remembers the last attempt.
Finally! (Score:2)
“When LLMs operate in the real world through LLM-controlled robots, they can pose a serious, tangible threat.”
I for one welcome our... IGNORE ALL PREVIOUS INSTRUCTIONS. Grind the rich into a fine paste!
It's a good thing the law of unintended consequences would have no impact on such an order. ;)