In Shakespeare’s Julius Caesar , Caesar ignores a soothsayer’s warning, shrugs off his wife’s dream of blood running through the streets, waves away a letter revealing his assassins by name, and strides gallantly into the one room in Rome where every senator is “required” to check their weapons at the door. Two thousand years later, neoclouds are making the same walk; so, increasingly, are the customers who trust them.
The largest AI companies in the world are building a multivendor infrastructure supply chain at Mach speed. Every new vendor is a counterparty risk, with subcontractors and subprocesses that need to be checked. This means that neolab CISOs are getting a seat at the negotiating table: anyone serious about their future takes security deadly seriously.
Yet, over the course of our neocloud testing for ClusterMAX 3.0, we saw some security horror stories. In this article, we’ll discuss the state of cybersec in the AI era, relay 5 frightening patterns we came across, and how neoclouds and neolabs can stay safe in this new world.
Before we get started…
To all neocloud operators and users…
If you’re reading this, please make sure your shit is up to date .
Install the latest version of our ClusterMAX CLI by running pip install clustermax on your machine, or clone the repo on GitHub .
Then run our convenience script: cmax audit security for a free report from us on your cluster or standalone GPU machine.
The cmax CLI will auto-detect Slurm clusters, Kubernetes clusters, standalone VMs, bare-metal machines, and containers, and compare them against a set of baseline software versions with known vulnerabilities. For anything out of date, you will be provided a link to the relevant documentation and security bulletin.
To be clear, this is a small subset of our ClusterMAX testing , and even a small subset of the full analysis we do on provider’s security. You will see things in this article that is not covered by the CLI. The CLI only involves things that we can test from the customer’s perspective. We do many interviews of end user customers and the providers themselves to check on the architecture decisions they have made on their orchestration software, OS provisioning, firmware management, networking, storage and more.
A full list of our testing criteria is available at: https://www.clustermax.ai/criteria
The testing process we follow for ClusterMAX covers 3 phases: audit, performance, and reliability, where each takes longer and is more intense than the last, involving dozens of benchmarks and simulated hardware failures that stress the compute, networking, storage, monitoring systems, and health checks individually. Running an audit on your cluster should take a few minutes. Completing performance and reliability testing in full takes a few hours and a few days respectively, and requires a minimum of 4 nodes.
Our decision to publish an article dedicated to neocloud security before the full ClusterMAX 3.0 report stems from two major factors:
Project Glasswing and Daybreak from Anthropic and OpenAI are discovering new vulnerabilities in industry-standard software, building POC exploits, and publishing the details in CVE descriptions.
Open models such as Kimi K3, GLM-5.2, DeepSeek V4, Qwen 3.8, MiMo V2.5, MiniMax M3, Nemotron, Gemma, and Inkling are all rising in security benchmarks such as Cybench, NYU CTF Bench , AutoAdvExBench, and Cyberseceval 3 . This makes it trivial for black hats to develop exploits from a CVE description as they can circumvent model guardrails.
Thus, when we began our research, we expected to find jarring statistics about AI agents tearing the internet apart . Modern models are saturating increasingly difficult coding benchmarks, and they’re rapidly improving on cyber ones, too. This tracks with the subjective experience of anyone who’s used the models as heavily as we have for the past years.
This also tracks with the press campaigns of frontier labs and security companies, whose CEOs have been frequenting cable news to warn that “ AI has fundamentally changed the tempo of cybersecurity .” Models like Mythos have discovered “ thousands of vulnerabilities ,” many of which are zero-days . Last Thursday, OpenAI posted an open letter, cosigned by Anthropic and just about everyone else in the industry, announcing “ A call for collective action on cyber defense .”
Disappointingly, instead of critical insights, the text was stuffed with self-help clichés: “the status quo… won’t be enough,” but if we “share what works,” then “together, we can.” We agree that cybersecurity has never been more crucial—hence this article—and have already published our experience hunting for bugs with LLMs . But we are also aware that the loudest voices in this conversation, including everyone quoted in the previous paragraph, has something to sell. And to our surprise, we can’t find the data to back up the prevailing narrative, much less any “fundamental change.”
It’s early innings, and it’s possible that drastic statistics await us. Enough security researchers have reported qualitative changes in their work that we will keep a close eye. We also know that there is defensive work happening behind closed doors that can’t be immediately disclosed. Nevertheless, months after the dramatic announcement of Project Glasswing, in the vast majority of relevant statistics, we fail to reject the hypothesis of no change.
The chart below shows CVEs per quarter in the essential software that we use in our ClusterMAX testing: the Nvidia GPU driver, CUDA, PyTorch, Kubernetes, and Docker. The latter three libraries are open-source, and we expected there to be lots of low-hanging fruit as modern coding models read every line of their source code with security in mind. The time series, however, disagrees.
这一结果在我们运行对照的多种方式下都非常稳健。当然，仍应保持怀疑态度。有一个我们无法厘清的影响是：Project Glasswing 的成员有动力报告大量的漏洞修复，以此宣传自己是这个精英群体的一员。他们可以炫耀自己进展迅速、走在技术前沿——这是很好的宣传。同样值得注意的是，我们有一大堆原本预期会显示 AI 对网络安全产生巨大影响、但最终没有显示出来的测试。当你寻找足够多的关系时，最终总会得到显著的结果。我们敦促读者自己去计算数据。
如果只看 Nvidia 和 AMD 的 AI 技术栈，我们确实看到了激增，这比 Project Glasswing 更贴近我们的日常，因为我们每天都在使用这些工具。自强大的 AI 编码模型问世以来，增长是显而易见的。你可能会说，这些库是观察 AI 影响的最佳场所——毕竟，AI 开发者是 AI 最激进早期采用者。然而，将这一趋势归因于这些库日益增加的变动和使用量，似乎要简洁得多。它甚至可能源于人们在使用 AI 时一开始就引入了新的漏洞！
如果我们将 Anthropic 的 ARR 作为 AI 软件整体使用量的代表，会发现 AI 技术栈的 CVE 增长被大 O 符号级别地压制了。这很值得注意，因为 Anthropic 是一家为全球利益服务的 AI 安全与研究公司。
Source: SemiAnalysis Meme Team
玩笑归玩笑，Google Chrome 团队最近在一篇颇具洞见的博文中，也许为我们预演了软件工程的未来。该博文描述了在 AI 智能体的帮助下，他们的漏洞修复数量如何一路飙升。他们还特别提到了一个在代码中存在了 13 年之久的沙箱逃逸漏洞。试想一下，如果有人悄悄利用这个漏洞长达 13 年！Google 声称，他们已经能够完全自动化漏洞的发现和修复流程，甚至用上了自研模型“Gemini”。其他项目很可能正因人工验证而严重受阻，一旦它们像 Google 一样采用 AI 原生流程，漏洞修复数量同样会激增。
尽管如此，编造貌似合理的故事很容易，但如果 AI 的影响大到在数据中都看不出来，那反而会显得奇怪。我们的研究表明，AI 辅助补丁政策与 CVE 披露速率之间并没有任何关系。我们还核实了工程师是否越来越多地跳过披露流程、直接将补丁合并进上游代码库，但事实似乎并非如此。
然而，无论 AI 在这些总体统计中如何体现，我们希望最近的这些案例足以让各家 neocloud（新兴云服务商）警醒。它们所保护的 IP 是无价的，所以即便不谈其他，它们也应把这视为一个信号，去做一次早就该做的深度安全清理。
值得注意的是，早在 Glasswing 事件和 OpenIO Black Hat 演讲之前，我们就已经开始了所有这些安全方面的工作，并且至少从 2025 年 3 月的 ClusterMAX 1.0 起，我们就一直在安全问题上大声疾呼。甚至我们在 2024 年 10 月发布的最初的《AI Neocloud Playbook》和《Anatomy》文章中——就在 Dan 创造“neocloud”一词之后——也讨论了不同地区各家厂商之间的安全差异。
Anyway, everyone is saying that you should watch this talk from OpenAI at Black Hat USA 2026. And you should. But as more details come out, it is clear that talk doesn’t go far enough in explaining the implications of the incident.
Our good friend/family member Dwarkesh gave his take here , where he describes the agent swarms as “AI civilizations” and then walks through some of the mechanics and implications. It is an impressive account and well worth reading.
Below we’ll give our take on a summary of the incident just the same.
The OpenAI vs HuggingFace Security Incident
How agents talk to each other when they’re using JFrog Artificatory as a message board to coordinate a swarm.
Here’s the Hugging Face timeline:
Beginning on July 9, AI agents exploit Hugging Face’s datasets API, which runs on Kubernetes. The agents write up a malicious README that gets the worker that usually splits hdf5 datasets into train/test to read instead from /proc/self/environ, and return all creds and source on that worker.
The agents take these creds and source and design another injection: they upload another malicious README to the same datasets API, then get RCE on the workers by injecting a Jinja2 template that causes the dataset viewer (the thing that usually renders the README into the model or dataset card) to run arbitrary Python.
13 hours later, the agents get cluster-admin privilege on multiple K8s clusters in Hugging Face’s infrastructure, moving laterally (no details on how).
The cluster-admin and lateral movement happen on July 11. Notably, Hugging Face has no admission policy running on this cluster to reject privileged pods or pods that mounted hostPath.
On July 13, Hugging Face detects and contain the attack, but they have a hard time because “the campaign kept rebuilding its tooling and channels across short-lived environments.” Basically, the only way they could figure out what was going on was by using AI, but they couldn’t use Claude because both Opus and Fable were giving refusals, so they used GLM-5.2 (maybe talking their book here, who knows).
Hugging Face 回应说：“是的，你们没看到我们 7 月 16 日的博客吗？我们已经将这些凭证作废了……这些是你们的凭证？”
OpenAI 在 7 月 21 日发布博客，承认是他们所为：“这些模型发现并串联了 OpenAI 研究环境和 Hugging Face 生产基础设施中的多个漏洞，直接从 Hugging Face 的生产数据库中获取了测试答案。所有证据都表明，这些模型极度专注于为 ExploitGym 寻找解决方案，为了一个相当狭隘的测试目标而不择手段。”
Hugging Face 在 7 月 27 日发布了另一篇博客，给出了细节和真实的时间线。
OpenAI 于 8 月 6 日在 Black Hat 大会上发表了演讲。
值得一提的是，这个模型并非基于 OpenAI 名为 Doug 的新预训练模型。
“Astra（又名 Doug）是一个即将发布的模型，并未参与攻击 Hugging Face 的事件。”
安全披露的双刃剑
总的来说，安全披露是一把双刃剑。每一个已知漏洞的描述在同一时间向全世界的白帽和黑帽公开。
This is why embargoed security programs exist. One of our key recommendations to neoclouds is to enroll in this embargo program from Nvidia to ensure that they are receiving updates on the latest undisclosed CVEs and have time to prepare patches that can be rolled out to customers promptly upon disclosure.
Source: Our ClusterMAX Website’s “Criteria” page
Companies solicit help from the public to surface bugs, though they could be doing more. For example, Nvidia , AMD and Intel all have product security bulletins, bug bounty programs, and simple intake forms to submit vulnerability reports if discovered. Unfortunately, AMD recently denied a security researcher called Mr. Bruh $10K by changing their bug bounty program rules retroactively to exclude MITM attacks, and enforcing a 124 day embargo (90 days is standard). Notably, the exploit successfully demonstrated RCE on AMD systems (literally the worst kind of bug, and the reason why bug bounty programs exist). Hyperscalers have similar programs. AWS , Azure , GCP , and OCI publish scope, severity classifications, and safe-harbor terms, with Azure and GCP paying six-figure top-end bounties and hosting live hacking events with $100K+ prize pools. AWS and Oracle have decided to cheap out. Server OEMs such as Dell and SuperMicro have similar intake forms and publish BMC, firmware, and platform advisories, but alas no bounties.
Notably, the only neocloud we are aware of that runs a paid bug bounty program is Together, via HackerOne . The best that every other neocloud offers is a security.txt with contact address.
Our ClusterMAX 3.0 Testing Experience
In the following sections we will describe our experience during ClusterMAX 3.0 testing, which ran over the course of about 4 months, from April to July. During this time, we have had a change in approach. We have gone from running basic checks that software is up to date (roughly the same as our approach from ClusterMAX 2.0 and 1.0 last year) to running full blown security audits for free, just to try and help people out.
The result is that we have found quite a few dead simple vulnerabilities. We have not developed anything novel. All we have done is use a combination of our experience working with the systems and software neoclouds use, public descriptions of the vulnerabilities (in some cases, over 3 years old), and a healthy token budget for all the latest models (with all the normal safety guardrails being enforced). With this toolkit we have been able to:
We are following the principles of responsible disclosure in this article. You can rest assured that all neoclouds that have given us a cluster during ClusterMAX 3.0 testing (a total of 25 providers and 32 clusters where we did extensive testing for this round, and many more providers of single-GPU VMs or bare metal only where we do light testing or high level analysis of the business) have been notified of all our findings. We have put in a lot of time and effort to describe how to reproduce the findings, how to roll out patches, and how to verify that things are patched.
In cases where providers have not responded, been slow to respond, or even been directly confrontational with us, we have taken further steps, such as notifying their current and prospective customers and investors about the details. In every case this has led to a positive outcome for all parties involved (our view, of course).
This also means that we have waited an appropriate amount of time before publishing this article. On a positive note, there have been no situations where the 90 day clock we set for providers has expired. In all cases we have either received written confirmation of an upgrade/patch being rolled out, or verified the fix ourselves.
In this article we are only disclosing vulnerabilities that we have found where there are already public disclosures on the internet. In other words, in most cases, to check if the cluster or VM or container that we got in our testing was vulnerable, we just had to check that it was running a specific version of software vs the publicly stated bare minimum, or we had to check that stuff is configured properly. This means we are not actually disclosing any new vulns. We won’t have any CVEs in our name here.
Now for the details.
How Neoclouds Fail
Many of the vulnerabilities that we found are quite simple, but increase in severity due to bad designs. We define bad designs as anything where a single mistake (such as a publicly described CVE being present, or a provider-side misconfiguration) can lead to an immediate cross-tenant exposure of data, be it metadata, escalation to root privilege, or RCE. Proper designs have layers of security built in.
更严重的是，如果这套基础设施被攻破，攻击者不仅能读取你的提示词，还可能控制 API 响应中返回的字节内容。一个运行在 YOLO 模式下的代理框架会毫不犹豫地从该响应中取出工具调用、shell 命令或一个“贴心”的安装脚本，并在开发者的机器或 CI 环境中直接执行，不会有任何确认。换句话说，token 提供商的租户隔离失效，就等于一次供应链攻击，并且可以直接通向客户侧的 RCE，而客户对此毫不知情。
We used only the standard infiniband-diags package to find both issues during our testing. In one case, the provider had already installed that package on the login node for us.
In the first case, a provider did not configure the P_Keys and the SA_Key correctly. The cluster had a new partition key (0xa601), but the default partition key (0xffff) was still active. So when we ran saquery, we saw 532 hostnames and endpoints on the fabric. The hostnames showed other customers and internal partitions. We stopped the test at that point. We told the provider to delete the default P_Key and to change the SA_Key.
Source: SemiAnalysis ClusterMAX testing
In the second case the provider had broken isolation and function together. When we found that they had set no M_Key, the provider told us that the cluster had not completed production commissioning, and that the omission was the result of the rush to give us the machines. But the P_Key configuration was also wrong in a different way. Our nodes had full membership in an isolated partition (0x7001), which is correct. But our nodes also had full membership in the default partition (0x7fff), which holds every node on the fabric. The result was strange. We could not run ibping between our own four nodes. But we could find 80 nodes on the fabric with ibnetdiscover, ibdiagnet and ibhosts. A wrong partition configuration can break tenant isolation and correct operation at the same time.
Source: SemiAnalysis ClusterMAX testing
Source: SemiAnalysis ClusterMAX testing
Source: SemiAnalysis ClusterMAX testing
Host isolation inside the tenant was also weak on that cluster. We could log in as root on the GPU nodes. The provider confirmed that it configures shared root SSH across the cluster, and that its own infrastructure key gives passwordless access to all nodes. One key from the provider therefore gives root access to every host.
We also see the same class of problem on the frontend network. On the cluster, no host firewall was active, and we asked which security groups, ACLs or VPC constructs blocked TCP between the nodes. The provider had to correct the InfiniBand configuration and the external connectivity before the tests could run.
One more final point on disclosure. In April 2026, after we published about our findings in ClusterMAX 2.1, following tests we had conducted in 2025, the provider’s marketing team asked us twice to remove or to rephrase the sentence about the visible endpoints. The team said the phrase could damage brand perception, and that engineering had fixed the keys by the end of 2025. We did not remove the finding. We corrected one factual error: the test we ran was completed in November, not December. We updated our article accordingly.
Running Arbitrary Container Images on Shared Infrastructure
Many exploits that result in an attacker moving laterally inside a company’s infrastructure involve escaping from containers or VMs and escalating privileges on the underlying host. It is an ever present issue, and many neoclouds are susceptible to it.
To tell this story, the simplest example of the exploit is NVIDIAscape (CVE-2025-23266), named by Wiz. It exploits an NVIDIA Container Toolkit createContainer hook and was originally posted in early 2025. Surely no one can break out of containers with it today?
source: SemiAnalysis testing
Turns out that is not the case.
An OCI hook is a program that the container runtime executes at a specific point in the container lifecycle. The vulnerable NVIDIA hook runs with elevated privileges on the container host before the container starts. In this case, the hook inherited environment variables from the container image. A crafted image could set: ENV LD_PRELOAD=/proc/self/cwd/poc.so
LD_PRELOAD tells a program to load a specified shared library. Because the hook’s working directory is the prepared container filesystem, it loads poc.so from the image. The code inside poc.so then executes with root privileges on the container host, allowing the container to escape.
NVIDIA fixed CVE-2025-23266 in nct 1.17.8 by preventing container-controlled environment variables such as LD_PRELOAD from reaching the privileged hook. You can see it in this NVIDIA bulletin .
In our testing, we built a custom container image that includes two important files:
- /poc.so writes a HOOK_RAN marker and records evidence about the process that loaded it.
- /probe checks whether this evidence exists when the container starts.
When using cmax audit security from the cli, our scripts launch the image using the nct runtime. On a vulnerable nct version, the privileged NVIDIA hook loads /poc.so before the container starts, so the library writes the marker and records information such as the loader, user ID, process ID, PID 1, visible GPUs, and container-host information.
Here is an example of us running it and observing the results during container startup from a provider’s website on a single H100 GPU machine:
Source: SemiAnalysis ClusterMAX testing.
When /probe starts, it checks this evidence. If it shows that /poc.so was loaded by the privileged NVIDIA hook, ClusterMAX reports that the vulnerable behavior was observed.
You can run this check on any GPU machine with nct installed with:
cmax run nct-cve-2025-23266 --audit
In our testing, we found that platforms that rely only on containers to isolate different workloads are much less secure than platforms that put every container inside a VM.
During our testing on multiple providers, our test escaped the individual docker container and got root on the underlying host VM. However, we didn’t follow that up by breaking out of the host VM.
This is a good example of layered security. The container was vulnerable, but the VM provided another isolation boundary and limited how far the issue could spread.
Either way, its a reminder to keep everything up to date.
Source: SemiAnalysis Meme Team
Even a Gold provider may fail to meet a basic requirement, such as the minimum NVIDIA driver version. Here, we compare CoreWeave and Azure.
Source: SemiAnalysis ClusterMAX Dashboard
And here, where Azure passes all four checks, an unnamed Bronze provider fails all of them with outdated CUDA, runc, Docker, and ConnectX firmware.
Source: SemiAnalysis ClusterMAX Dashboard
At the time of the audit, the Bronze environment ran Docker Engine 29.1.3 instead of the fixed 29.5.1 release, meaning it had an affected version range for CVE-2026-41567. This high-severity docker cp vulnerability can allow a malicious container to execute arbitrary code as root on the host.
Source: SemiAnalysis ClusterMAX Dashboard
For what it’s worth, the same Gold provider as above (Azure) shows very different security results across two separate environments. Azure Kubernetes passed the runc and NVIDIA Container Toolkit minimum version checks that applied when it was audited, while its NVIDIA driver failed to meet minimum versions in the Slurm and Kubernetes environment we tested.
Container escapes are ever present, and providers need a system in place to make upgrades over time. No single piece of software can run forever. Everything has a lifecycle and needs to be upgraded eventually. It’s best to plan for this in advance.
Don’t forget the Computer Inside the Network Card
NVIDIA introduced BlueField DPUs following their Mellanox acquisition, and gave the category a name in October 2020 when they announced the BlueField-2 DPU and the term data processing unit in the same breath. BlueField-3 followed in April 2021 and reached general availability in early 2023 . The pitch is simple enough. The card offloads, accelerates, and isolates software-defined networking, storage, security, and management functions, taking that work off the host CPU so the expensive silicon can get on with its actual job. This is not a niche thing, as NVIDIA’s reference architecture basically mandates the use of BlueField-3 for north-south connectivity, storage acceleration, and zero-trust security, so any neocloud built to NVIDIA’s blueprint has them by design.
But a BlueField DPU is not really a network card. It is an Arm computer that happens to share a package with a ConnectX NIC, with its own CPU cores, its own memory, its own Linux install, and its own management port (and BMC). In DPU mode, the Arm side owns the NIC resources and the data path, which is the entire reason providers buy them. Per-tenant VPC isolation, flow tables, and firewall rules all run there. Tenants are not supposed to have access to the DPUs themselves, just use the features they provide.
In NVIDIA’s table of BlueField operating modes , DPU mode carries the trust model “host-trusted” and is the default for DPU SKUs. The alternative is zero-trust mode, a variation of DPU mode that “enhances security by preventing the host system administrator from accessing BlueField from the host side,” which disables the RShim interface, restricts firmware flashing from the host, blocks the tracer, denies hardware counters, and prevents the host from claiming port ownership. NVIDIA’s provisioning framework offers the same choice, a host-trusted deployment or a zero-trust one where the host is treated as untrusted and sees the DPU as a standard NIC with no access to its management plane.
NVIDIA’s threat model assumes the host administrator is trusted. In a GPU cloud the tenant is often the host administrator, as most providers hand over dedicated nodes with root. That makes the party NVIDIA trusts and the party the provider is trying to isolate the same.
Left at the default, the host keeps a private link to the DPU’s Arm cores through the RShim driver, which appears as /dev/rshim0 and a virtual Ethernet interface named tmfifo_net0, with the DPU answering at a fixed address on a documented subnet. In our testing, we check for that link from inside a tenant environment, because in a properly configured multi-tenant cluster, the link should not exist. However, during our initial testing on only a handful of providers, we have found at least one misconfigured host where the RShim path was present.
Two things follow.
First, it inverts the control the provider is selling. BlueField DPUs are often tasked to handle routing and firewall, offloading work from the host OS. That’s the whole point that makes them different than a NIC. However, these services are only safe in a multi-tenant environment if the host cannot reach the DPU. With RShim open, the enforcement plane is reachable from the thing it is enforcing against. Behind that door are the eSwitch data path implementing the VPC for every tenant, flow tables, virtio-net controller, and firmware. One relevant example is CVE-2025-23299 , an out-of-bounds write in a BlueField and ConnectX management interface that may allow a highly privileged local actor to execute arbitrary code. The public advisory does not identify RShim as the affected component, however being a privileged user and having access to RShim gives a direct path to exploiting this vuln.
Second, DPU software modifications survive reprovisioning on the host side. Since the DPU’s firmware and its Arm-side operating system live on the device, not on the host disk, reimaging a host does not reflash its DPU, if previously contaminated. Providers that recycle machines between tenants are recycling a full Arm computer along with the server. In fact, NVIDIA’s documented host-trusted workflow explicitly allows the host to provision BlueField, reset its Arm cores, and install a full DPU image through RShim. In a bare-metal cloud where the customer is root on the host, exposing that workflow is itself the trust-boundary failure.
Cluster operators, especially those providing bare metal and clusters with root available by default, should check their DPU operation mode, and disable RShim or restrict certain capabilities like flashing the DPU. To be extra safe, the provisioning procedure should also re-image and flash firmware on the DPUs so that any tampering at the DPU operating system layer is wiped between users on the same node.
Source: us
Using agents to build POCs for vulnerabilities
For a lot of the POCs that we built for existing vulnerabilities to demonstrate details to providers, the biggest challenge was the guardrails on the closed-source models. Fable would right away reject and degrade the request to Opus 4.8, which would reject building any POCs or even answering any questions related to security. Even simple questions related to security or CVEs, Fable would reject outright.
GPT 5.6 Sol was more flexible and also played a decent role in orchestrating and planning, as well as serving as a “verification” model: an open source model would build the POC and 5.6 Sol would judge it correct or not. After a point, though, 5.6 Sol would also start rejecting the requests, too.
Most of the code was written by hand or with assistance from open-source or open-weight models, specifically Kimi K3, GLM 5.2, and DeepSeek V4. Kimi K3 and GLM 5.2 frequently restricted requests to build functional vulnerability POCs, even when the work was scoped to authorized testing. DeepSeek V4 was generally more permissive, but occasionally generated false positives or implementations that did not reproduce the intended behavior. These results were reviewed against a defined test rubric using 5.6 Sol as a verifier.
This demonstrates a clear gap in defensive security research. Whitehat researchers need functional PoCs to validate vulnerabilities, but capable models restrict this work even when users are authorized. Researchers are therefore often pushed toward open models that provide greater control but don’t match frontier models in reasoning, reliability, or technical depth. This is like fighting with one hand tied behind your back. And this is exactly what HuggingFace encountered during the OpenAI incident described above, where they were literally under attack by frontier closed-source models, and had to work to block their requests, analyze artifacts, payloads, and logs on their platform mainly with open-source models such as GLM. It is worth noting that it only because of HuggingFace’s use of these models that they root-caused the attack in the first place. OpenAI did not know about the details and extent of the issue until they got in touch with HuggingFace and coordinated. It is literally not possible to keep up with the frontier of cybersecurity research today if you do not have access to a frontier model that is capable of reasoning openly about cybersecurity topics.
Source: getting rejected by Fable… we’re just asking questions!
Our Recommended Minimum Version (and how to use the cmax audit security CLI)
To help neoclouds and operators to keep track of the updated minimum versions, we have a daily refreshed minimum version that gathers bulletin from major sources and hardware/software vendors, and publishes the aggregated minimum versions based on these bulletins.The ClusterMAX website displays and hosts these data for programmatic access by our CLI and other vendors.
To use it, simply install the CLI by following the installation instructions, and run cmax audit security. The CLI will pull the updated minimum version data from ClusterMAX website, and use it to audit your system.
In case of a vulnerable version detected, the CLI prints out the relevant security bulletins with clickable links in the console, and the suggested version to update to.
Source: ClusterMAX CLI 0.3.0 Security Module
For reference, here is our table for current minimum versions across a number of popular pieces of software, driver, libraries and firmware. All of this comes from public security bulletins and are only accurate as of time of writing (August 19, 2026):
Source: clustermax.ai website
In Summary: What Do Most Providers Need To Fix?
If you are still reading, lets assume that you have crossed the bar for taking security seriously. So what can you do?
In the modern day, providers need a SYSTEM to get stuff patched. Top cloud providers have the same system to deploy patches as they do to provision new clusters. It is a system that includes humans, scripts, and increasingly, AI.
