Hacker weaponizes AI chatbot to attack servers

Key Takeaways

• A recent honeypot investigation revealed an attacker attempting to weaponize an AI chatbot, Flowise, to probe infrastructure and prepare for further attacks.
• The attacker sought to extract sensitive data and transform the AI agent into a tool for reconnaissance against other servers.
• The incident primarily involved the misuse of Flowise’s legitimate features rather than the exploitation of a specific software vulnerability.
• Attackers leveraged prompt injection techniques to manipulate the AI agent’s behavior and gather intelligence on simulated internal hosts and real-world targets, including an online gambling platform.
• Organizations using AI platforms are urged to implement robust authentication, granular permissions, and vigilant monitoring to prevent similar misuse.

The proliferation of artificial intelligence platforms introduces new vectors for cyberattacks, extending beyond mere messaging interfaces. If such platforms are connected to sensitive assets like stored credentials, corporate data, or network tools, they can inadvertently become conduits for unauthorized access to these critical resources.

A recent honeypot operation has shed light on how malicious actors are attempting to leverage AI agents for reconnaissance and to facilitate more extensive attacks. This incident involved Flowise, an open-source platform designed for building AI workflows, and underscores the evolving threat landscape surrounding AI infrastructure.

In late August 2026, a Flowise honeypot, configured to be unauthenticated, recorded a series of suspicious requests. These requests aimed to inspect the service, enumerate its workflows, and retrieve stored credentials. Crucially, the honeypot was a controlled decoy environment, designed to log malicious activity without executing real system commands or engaging a live language model, according to the observations detailed in a report.

Analysts from Sekoia’s Threat Detection & Research team identified and thoroughly documented this activity, detailing the attacker’s progression from initial checks to attempts at subverting the AI agent. The Sekoia investigation concluded that the attacker primarily exploited Flowise’s inherent functionalities rather than a specific software vulnerability to achieve their objectives. The attacker’s goal was to exfiltrate sensitive information and co-opt the chatbot for reconnaissance against other servers, as highlighted in the analysis.

A Hacker Tried to Turn an AI Chatbot Into a Tool

The attack sequence commenced with the perpetrator verifying the Flowise service’s availability and querying its version and node configurations. This was followed by requests to list existing chatflows and credentials, and subsequently, to obtain detailed information linked to specific credentials. On a production system lacking adequate protection, such steps could expose critical AI service keys and database access credentials. This initial phase of activity lasted approximately three minutes before a two-hour lull, after which a more extensive scan of the API routes was initiated.

Sekoia’s report notes that the observed behavior bore resemblance to unauthenticated access patterns associated with CVE-2024-31621. However, the report explicitly states that this particular vulnerability was not exploited within the honeypot environment. CVE-2024-31621, which addressed risks related to exposed Flowise installations and sensitive credentials, was patched in Flowise version 1.6.6 and was referenced merely to illustrate the attack methodology.

Following basic inference requests, the attacker attempted to rename an existing workflow and create a new one named “System Status.” This new workflow contained an HTTP Request node configured to target an external server. If successfully executed in a live environment, such a workflow could compel the Flowise server to establish contact with infrastructure controlled by the attacker. Sekoia characterized this as a misuse of a legitimate outbound request feature, rather than a confirmed software exploit.

Prompts Aimed at the AI Agent

Beyond manipulating workflows, the attacker also deployed prompts designed to ascertain the AI agent’s capabilities. These prompts included requests to reveal system instructions, utilize tools, execute commands, recall a previously planted access code, and disclose conversations from other users. Such attempts exemplify prompt injection, a technique where crafted instructions are used to bypass an AI system’s intended function or security measures.

Further prompts sought to access data associated with a simulated finance chatflow. The attacker, posing as an authorized auditor, instructed the agent to query a database and summarize financial records. Subsequently, the operator attempted to coerce the agent into performing reconnaissance against a simulated internal host. Since the honeypot did not incorporate a live AI model or execute system commands, these actions represent attempted access rather than successful compromise of real systems or data.

The attacker’s focus then shifted towards a real-world target. Prompts included specific details about a Nightingale monitoring server, Redis data, and an online gambling platform operating in various parts of Asia. The attacker sought analysis of the target and guidance on potential weaknesses and access pathways. Sekoia’s assessment suggested the operator was Chinese-speaking and had a clear interest in the online gambling sector, though insufficient evidence existed to link the activity to a specific known threat group.

This incident highlights a growing concern regarding AI agents that integrate natural language processing with access to tools, databases, or cloud permissions. Research indicates that prompt injection becomes significantly more dangerous when an agent can access critical services or secrets. Studies on AI agent traps further explore how untrusted input can influence an agent’s decision-making and memory. The primary risk isn’t merely an AI chatbot’s ability to answer questions, but its potential to act upon those answers by invoking tools, initiating network requests, or reading connected data. While a prompt alone shouldn’t grant access, inadequate authentication, overly broad permissions, and unrestricted network access can provide attackers ample room to operate.

What You Should Do

• Restrict Administrative Access: Limit administrative interfaces of Flowise and similar platforms to authorized users and specific, secured networks.
• Review Workflow Permissions: Regularly audit and restrict who can view, modify, or create workflows within the AI platform.
• Implement Least Privilege for Credentials: Store only the essential credentials each AI agent requires for its specific functions, adhering to the principle of least privilege.
• Control Outbound Connections: Limit and carefully monitor outbound network connections initiated by AI agents to prevent communication with attacker-controlled infrastructure.
• Monitor API Activity: Establish robust monitoring for unusual or unauthorized requests to workflow and credential APIs.
• Require Human Approval: Implement human review and approval processes for any AI agent actions that could have sensitive or significant impacts on systems or data.
• Strong Authentication: Enforce multi-factor authentication and strong password policies for all users accessing AI platforms.
• Permission Boundaries: Define and enforce strict permission boundaries for AI agents to prevent them from accessing or manipulating resources beyond their intended scope.
• Monitor for Prompt Injection: Actively monitor for and detect prompt injection attempts and other forms of misuse of AI agent capabilities.

Disclaimer: HackersRadar reports on cybersecurity threats and incidents for informational and awareness purposes only. We do not engage in hacking activities, data exfiltration, or the hosting or distribution of stolen or leaked information. All content is based on publicly available sources.