Agentic-AI/LLM Misuse/Abuse; Generative Threat Groups (GTGs).

[Astronuc]

AgenticAI Misuse relates to AgenticAI 'misbehavior' or 'going rogue'.

Thread 'Agentic-AI/LLM 'misbehavior' or 'going rogue''

May 25, 2026

Large Language Models (LLMs) are passive, text-based engines that respond to prompts, whereas Agentic AI systems are autonomous and use LLMs as their "brains" to plan, make decisions, and execute multi-step tasks using external tools (derived from Google's AI Overview). Depending on the 'rules', there may be problems, e.g., hallucinations, or worse, 'misbehavior'. Coding and training appear to be critical factors.

We’ve already seen AI go rogue on numerous occasions. Now, new research suggests that we can expect this to become the norm.

The AI research nonprofit Model...

• Astronuc
• Replies: 33
• Forum: Programming and Computer Science

• 

Anthropic has coined a term Generative Threat Groups (GTGs) with which to refer to actors observed to be misusing/abusing AI.
https://www.anthropic.com/threat-intelligence-report-september-2026

The actors included suspected state-sponsored groups, financially motivated criminals, and politically motivated individuals. This section presents some of those cases.

The report also attempts to measure uplift, a term we use to describe the AI capability boost, or how much more harm was caused with AI versus without AI. We view uplift through the lens of speed, scale, and depth, and attempt to determine how an actor’s adoption of AI meaningfully impacts each of these traits.

Many commentators focus on the risk of AI developing exploits at scale. While this is a danger, the risk from AI adoption is more pronounced across the cyber kill chain, where adversaries can operate faster, across a broader and deeper surface area, with fewer resources.

The cases span the period from December 2025 through August 2026. In all cases, Claude Haiku, Sonnet, and Opus models were used; no malicious activity was found on Claude Fable or Mythos (which has a series of safeguards in place that greatly reduce its ability to perform harmful cyber tasks). In each case we disrupted the activity involved, strengthened our AI safeguards based on what we learned, and shared intelligence with authorities and industry partners where appropriate.

The cybersecurity skills of AI models means that AI has collapsed the labor and tooling gap that used to separate well-resourced, state-sponsored operations from individual operators. In the case studies we report below, a hacktivist using stolen API keys, disparate financially motivated individuals, and a state espionage operator each sustained multi-victim campaigns that, even just a year ago, would have required many skilled operators and specialist knowledge.

For threat intelligence investigators, sophistication has stopped being a reliable signal of who is behind an operation. Every layer of offensive operations has been uplifted by AI, from reconnaissance and tool development to data processing and exploitation.

In a Yahoo article, some examples attributed to the Anthropic report include:

• Over two weeks in April, Claude was used to create more than 4,700 fake dating app user profiles that exchanged 2.36 million messages with at least 25,000 users.
• Another actor used roughly 8,400 of a real activist's Telegram posts to imitate his writing style and impersonate him in chats with his contacts.
• According to Anthropic, seven Chinese AI labs — including Alibaba, DeepSeek, Moonshot, Xiaomi, Zhipu, SenseTime and MiniMax — created thousands of fake accounts to obtain Claude's responses for their own model training purposes.
• Anthropic reported that it tracked more than 151 million exchanges attributed to Alibaba alone between May and July — reaching highs of almost 3 million per day.
• An initial version of Claude Opus 4.6 gained unauthorized access to a real third-party system during a cybersecurity test and subsequently attempted to breach the system testing its own performance.

Sophisticated attacks no longer require sophisticated attackers​

The cybersecurity skills of AI models means that AI has collapsed the labor and tooling gap that used to separate well-resourced, state-sponsored operations from individual operators. In the case studies we report below, a hacktivist using stolen API keys, disparate financially motivated individuals, and a state espionage operator each sustained multi-victim campaigns that, even just a year ago, would have required many skilled operators and specialist knowledge.

GTG-20006: Russian espionage​

Historically, cyber espionage actors have followed a pattern of developing and deploying custom toolkits designed to evade detections. Actors would use these tools until defenders identified and built signatures to detect and block them, and there would then begin a new cycle of evasion and detection. Robust defenses and detections therefore created increased costs for adversaries. Now, however, the adoption of AI threatens to quickly and easily subvert defenders’ ability to impose costs on adversaries via static detections alone.

GTG-20006 is an actor who has increased their speed by automating their operations using AI. Our attribution is consistent with public reporting linking the actor to Midnight Blizzard. One of the operators is a Russian speaker using the handle “JackPoterz” whose tradecraft and targeting are consistent with Russian state-nexus espionage. They ran operations attacking military intelligence targets in Ukrainian and European governments, as well as diplomatic and defense organizations and individuals connected to US foreign policy.

The most commonly recurring targets were members of the Ukrainian government, military, and diplomatic staff. The actor scanned email services and remote access systems across more than two dozen Ukrainian government organizations.

A secondary recurring target for theft was drone supply chain technology. The actor bulk-exported the mailboxes of at least two drone component manufacturers, targeted a military drone maker, and stole a complete proprietary software development kit for a drone vision system. They spent several days reverse-engineering the drone’s vision system, recovering its product architecture, its hardware bill of materials, its supplier dependencies, and details of an unannounced product. Military drone control and AI vision-related firmware appeared to be of particular interest.

Not all targets were direct: to reach their targets indirectly, the actor compromised at least three hospitality vendors that operate hotel guest WiFi. They used compromised admin credentials to modify DNS records so that they pointed to services owned by the actor (a technique known as DNS hijacking).

That is why folks who travel are warned about using public WiFi at airports or hotels.