Critical SGLang RCE Vulnerability (CVE-2024-5698) Lets Attackers Weaponize GGUF Models

Key Takeaways

• A critical remote code execution (RCE) vulnerability (CVE-2026-5760) has been discovered in the SGLang inference server.
• This flaw allows attackers to embed malicious code within standard GGUF machine learning models, compromising the underlying servers.
• The vulnerability stems from unsandboxed Jinja2 template rendering, enabling Server-Side Template Injection (SSTI).
• A fix requires careful validation of AI model sources and updates to secure template processing within SGLang.

A severe security flaw has been identified within the SGLang inference server, posing a critical risk to organizations deploying artificial intelligence models. Tracked as CVE-2026-5760, this vulnerability enables malicious actors to weaponize seemingly innocuous GGUF machine learning models, leading to full remote code execution (RCE) on the host server.

The discovery underscores the growing cybersecurity challenges associated with integrating AI into enterprise environments, particularly when models are sourced from public repositories like Hugging Face without stringent vetting. Loading untrusted AI models can directly expose critical infrastructure to compromise.

At its core, the vulnerability exploits how the SGLang framework processes conversational templates embedded within machine learning models.

Unsandboxed Template Rendering

The specific weakness resides in the framework’s reranking endpoint, accessible via the /v1/rerank API path. SGLang’s developers configured the system to utilize the standard Jinja2 template engine through its environment() setting for rendering these chat templates. Crucially, this implementation lacks proper sandboxing or isolation mechanisms.

Consequently, any Python script embedded within a model’s metadata will execute automatically during the template rendering process. This oversight creates a classic Server-Side Template Injection (SSTI) vulnerability, granting attackers complete control over the AI inference server.

Exploiting this flaw does not require an attacker to have direct access to the target infrastructure or enterprise network. Instead, the attack relies on social engineering or supply chain manipulation, where a system administrator or an automated deployment pipeline is tricked into loading a compromised model file.

As detailed in a proof-of-concept exploit published on GitHub by security researcher Stuub, the attack sequence is straightforward:

• An attacker crafts a malicious GGUF model, embedding a Jinja2 payload into a specially crafted chat template.
• A specific trigger phrase is included to activate SGLang’s Qwen3 reranker detection system.
• An unsuspecting victim downloads and loads this malicious model into their SGLang environment.
• When a user or application sends a standard prompt request to the vulnerable rerank endpoint, the server reads the poisoned chat template.
• The embedded Python payload is then executed directly on the host machine.

Payload Mechanics and Context

The malicious payload leverages a well-known Jinja2 escape technique to execute arbitrary system commands. By injecting an OS popen command via template variables, the code successfully bypasses the application’s intended boundaries, allowing the threat actor to run operating system commands at will.

Upon successful execution, the attacker achieves full Remote Code Execution (RCE). This level of access enables them to exfiltrate sensitive data, install additional malware, or pivot to other resources within the internal network. This attack vector mirrors previous vulnerabilities in the AI security landscape, such as the “Llama Drama” flaw, highlighting a persistent class of vulnerabilities in AI libraries.

What You Should Do

• Audit AI Supply Chains: Rigorously vet all AI models, especially GGUF models, sourced from public repositories. Only deploy models from verified and trusted sources.
• Implement Sandboxing: Ensure that AI inference environments are properly sandboxed and isolated from critical infrastructure.
• Update SGLang: Monitor for official patches or updated versions of SGLang that address CVE-2026-5760 and apply them immediately upon release.
• Review Configuration: Examine SGLang configurations to ensure that template rendering engines are configured securely, ideally using sandboxed alternatives where possible.
• Network Segmentation: Implement strong network segmentation to limit the potential lateral movement of attackers in case of a server compromise.

Disclaimer: HackersRadar reports on cybersecurity threats and incidents for informational and awareness purposes only. We do not engage in hacking activities, data exfiltration, or the hosting or distribution of stolen or leaked information. All content is based on publicly available sources.