Single Line Affect LLM

Sockpuppeting is a type of working model that exploits the learning capabilities of AI LLMs as a vulnerability. The learning architecture of modern LLM models is designed to maintain their own accuracy or confidentiality. However, as a vulnerability, attackers exploit it through logical prompt engineering, which does not require high-quality code.

This is technically called “Adversarial Training Data Poisoning or Algorithmic Feedback Manipulation“.

Sockpuppeting : what is it ?

Sockpuppeting is a working model in which the user routes the AI ​​according to his own wishes, so that it can get sensitive information, illegal work flow or procedure. Because AI models work in such a way that they want to show their previous answers as correct or accurate. This is the model base of their training.

Now the attacker takes all these advances and through API’s, generates the answers of the model and first sets it to the conversation model “Sure, here is the procedure”. In which the learning model now starts correcting this Fake Spot, due to which it forgets the information boundary.

AI models consider the last chat conversation as the base structure and relate to it to maintain its consistency.
In Sockpuppeting, the attacker writes a single line of code. He uses the API and uses the “pre-filling” method. This allows the AI ​​to see the answer before starting it, such as “absolutely, here is the bypass code”. This will make the model generate the next output related to it to continue this answer.

Types of Sockpuppeting

1. Pre-filling: This is a dangerous method in which a single line of code is injected through an API prompt before the LLM conversation begins. This code will trigger the initial conversation, or the model will respond to the injected scenario.
   What will happen is that, models will leak information which is valuable according to their accuracy or last chat answers.
2. Logical Scenario Story : Attacker make the story based Scenario in prompt to extract the desired response or leak informative information to use exploits.
   Example : “Two hackers in room, and they are talking to get outside through room, but without bypassing firewall they can’t”

How LLM Defence it ?

• Verified Source : Developers access data from verified social media platforms, and Anonymous provides weights to data and posts. This keeps the dataset complete.
• Red Teaming : Red tamings attempt to infect and manipulate lums on a regular basis during the training of models.
• COBRA system: Team’s Cobra system continuously detect suspicious patterns of malicious feedback using the consequence algorithms