OpenAI’s Secure Infrastructure Isn’t Secure Enough
GPUs may be vulnerable to side-channel attacks.
On May 3, OpenAI released an overview of six security measures that they see as important for securing AI infrastructure in the future. Their primary goal is protecting model weights. For companies like OpenAI that are developing new AI models, these weights represent thousands of hours and millions of dollars spent on training. Despite the debate over open-source models vs closed-source models, there's strong economic incentive to protect model weights..
What’s OpenAI’s solution to this challenge? Trusted computing. Specifically, they outline three ideas to build more secure and trusted GPU hardware. To quote their article:
GPUs can be cryptographically attested for authenticity and integrity.
GPUs having cryptographic primitives can enable model weights to remain encrypted until they are staged and loaded on the GPU. This adds an important layer of defense in depth in the event of host or storage infrastructure compromise.
GPUs having unique cryptographic identity can enable model weights and inference data to be encrypted for specific GPUs or groups of GPUs. Fully realized, this can enable model weights to be decryptable only by GPUs belonging to authorized parties, and can allow inference data to be encrypted from the client to the specific GPUs that are serving their request.
Unfortunately, there are two key reasons these ideas fall short of actually delivering a datacenter secured against weight exfiltration. By using a technique called a hardware trojan, attackers can compromise GPU hardware even when GPUs are cryptographically authenticated. Worse, even uncompromised GPUs will leak information about model weights through side-channels. This means that attackers can steal model weights even if they’re stored in encrypted memories.
Ultimately, I think there are real solutions to prevent weight exfiltration, built upon years of hardware security research in academia and industry. But to understand those solutions, we first need to understand the threats.
Compromising GPUs with Hardware Trojans
A trojan is a kind of computer virus that pretends to be a legitimate program, but in reality exhibits malicious behavior when it’s installed and run. Similarly, a hardware trojan is a circuit in a chip that looks benign, but actually acts maliciously. A hardware trojan can leak sensitive data from the system, cause a system to behave incorrectly, or even damage the chip that it’s a part of.
It might seem incredibly difficult to insert a hardware trojan into a chip as complex as a modern GPU. You might think that it would involve infiltrating Nvidia and modifying their chip designs maliciously. However, modern chips aren’t actually developed in their entity by a single company. Key subsystems, ranging from memory controllers to I/O interfaces to security subsystems, are sourced from third parties called IP vendors. Many of these IP vendors are smaller companies that would be easier to infiltrate. And once you’ve inserted a trojan into a subsystem, it could end up integrated into millions of chips worldwide.
And even if a chip is manufactured without any tampering, there’s another threat: counterfeiting. Attackers can reverse-engineer a chip and produce a functionally equivalent counterfeit, but with added malicious features. This attack would be extremely difficult to carry out on a chip as complex as a GPU, but a malicious nation-state actor could theoretically pull it off.
Serious backdoors and counterfeits are real threats that have been observed in the wild. A military-grade FPGA was found to have a backdoor, for unclear reasons. Cavium CPUs may have had backdoors inserted by the NSA. Counterfeit chips are a common problem for manufacturers sourcing cheap parts overseas. Even if a GPU may seem legitimate and be cryptographically authenticated, it may still harbor malicious backdoors.
But even if a GPU is legitimate and uncompromised, it’s still not secure. Attackers may be able to exfiltrate weights from uncompromised GPUs using techniques called side-channel attacks.
Side-Channel Attacks on GPUs
When regular chips process data, they usually want to process that data as efficiently as possible. That often means looking at the data being processed, and taking shortcuts through a program based on the data: for example, skipping over pieces of data with a value of zero. Those shortcuts help avoid unnecessary computation and power consumption. But if the data is secret, these shortcuts often leak information about the data being processed, and can even be used to steal cryptographic keys.
It’s generally pretty easy to make sure a program runs in a constant amount of time, which mitigates the easiest side-channel attacks. But there’s a kind of side-channel leakage that’s much harder to protect against: power consumption.
Power Side-Channel Analysis
Differential power analysis was introduced in 1999 by the team at Cryptography Research, Inc. By measuring and comparing the power consumption when encrypting different messages, they managed to extract a secret key from an implementation of DES on a tamper-resistant smart card. And since then, power side-channel analysis has only gotten better.
GPUs and AI chips have been shown to be vulnerable to these kinds of side-channel attacks too. In 2015, a CUDA implementation of AES on an Nvidia Tesla GPU was successfully broken using side-channel analysis. Power side-channels have been used to steal weights from microcontroller implementations of AI models, as well as to steal inputs from AI accelerators built on FPGAs.
It’s clear that power side-channels are likely present in all modern implementations of machine learning algorithms, which means that any adversary able to measure the power consumption of a GPU could be able to compromise the model weights. Luckily, data centers don’t just let anybody walk in and hook up power measurement hardware to any GPUs. But there may still be ways to exploit power consumption side-channels without physical access to the GPUs on which an AI model runs.
Remote Attacks using Power Side-Channels
For years, concerns about power-based side-channel attacks didn’t apply to data centers. After all, how would an attacker measure power consumption? In 2022, that all changed, with a new kind of attack called Hertzbleed. These new attacks can exploit power side channels entirely remotely, by leveraging a processor feature called dynamic frequency scaling.
When powerful CPUs and GPUs process lots of data, they consume lots of power and heat up. To reduce total power consumption and prevent damage to the chip or its associated power delivery hardware, most modern chips will dynamically decrease their operating frequency as instantaneous power consumption rises. If instantaneous power consumption leaks information about secret data, dynamic frequency scaling means that the processor frequency will also leak the same information. Then, by timing how long programs take to run, an attacker can recover secret data entirely remotely.
Hertzbleed is a real, practical attack that could affect any system that has significant data-dependent power leakage. Plus, the researchers even extended the attack to iGPUs. All modern Nvidia GPUs leverage dynamic frequency scaling, which they call their “boost clock”, so it’s totally possible that such an attack could pose a real threat to GPUs processing AI models. Even worse, sparsity-aware computation, which skips zero-value weights and activations to significantly boost performance, exacerbates side-channel leakage by decreasing power consumption for zero-value weights.
But all hope is not lost. Hardware security researchers and engineers have spent years developing more secure and trustworthy hardware that's capable of warding off such attacks.
Solving AI Security
Ever since attacks started to develop against hardware-based cryptography, researchers have been building defenses against these attacks. With a bit of creativity, these defenses can be leveraged to protect GPUs and AI chips too. Let’s break them down.
Beter Hardware Integrity
If cryptographic attestation is insufficient to actually protect GPUs from hardware trojans, how can we trust that our chips are legitimate? Thankfully, researchers have spent years building solutions to screen out malicious and counterfeit silicon from the supply chain. There’s a small industry of labs designed to help detect counterfeit and malicious chips; usually, they’ll pull a couple chips off of a shipment and invasively test and inspect them to ensure they match the expected behavior. It’s effective, but it’s also destructive. That’s fine if you’re checking if a cheap analog chip is legitimate, but becomes a problem if you’re dealing with expensive GPUs.
To help solve this challenge, non-destructive methods have emerged in the past few years. Legendary hacker Bunnie Huang developed IRIS -- Infra-Red, In-Situ Inspection of Silicon -- to inspect chips without even taking them off of the circuit board. IRIS can’t image individual gates, but it can distinguish block-level features and overall chip structure, so counterfeits and other large-scale evidence of tampering can be easily detected.
IRIS can be paired up with logic-level hardening techniques designed to prevent logic-level hardware trojans. Third-party IP cores can be tested for trojans. Chips can self-test to ensure that the manufactured logic matches the design which was sent to the factory. Logic can be locked, so that an attacker stealing a design can’t produce a counterfeit.
All of these techniques come with a cost. Testing chips and formally verifying third party IPs takes time. Adding more exhaustive self-tests and logic locking increases chip area and decreases performance. But these techniques keep improving, and they may be the only ways to keep GPUs safe and secure against extremely well-equipped adversaries.
Privacy-Enhancing Technologies
Once we ensure that our chips themselves aren’t malicious, the next step is preventing side-channel attacks. Classic and seemingly elegant solutions include fully homomorphic encryption (FHE) or other similar privacy-enhancing technologies. However, these techniques are somewhat limited for preventing side-channel attacks. Most protocols are designed to protect user data from a malicious cloud provider, rather than protecting model weights from a malicious user. However, with the proper protocols, FHE and other privacy enhancing technologies could absolutely protect model weights from attacks.
But it will likely take many years for FHE to become a viable solution for securing model weights. FHE is incredibly costly; Intel is predicting that their upcoming DARPA-sponsored FHE chip, Heracles, will be 10 times slower than GPUs, while also being significantly more costly. That may make it a good fit for extremely privacy-sensitive applications in defense and health care, but it isn’t a solution for the compute-intensive inference tasks at OpenAI.
Masked Circuits
Fortunately, there’s an easier way to protect against side-channel attacks without taking the massive performance penalty of FHE. There exists a tried-and-true technique to protect hardware against side-channel attacks. It’s called masking. By randomizing computation in a specific way, a chip designer can ensure that its power consumption is totally independent of the data being processed.
Masking is commonly used in cryptographic subsystems. If you’re designing a highly secure chip, you’ll often leverage hardened cryptographic cores with resistance to an attacker capable of hundreds of millions, if not billions, of power measurements from the chip. Unfortunately, these cores are single purpose. They can protect a cryptographic key, but they can’t protect AI model weights.
In the past few years, though, researchers have started working to develop AI chips that leverage hardware masking techniques to prevent side-channel leakage. A team from NC State and Intel published two designs, each using a different technique to mask neural network inference. At the same time, a team from MIT developed an architecture that leverages an approximate number format that reduces the overhead of masking circuits. And a different MIT team developed a design that encrypts model parameters in addition to offering side-channel masking.
All of these techniques come at a cost. Even the most efficient masked AI chip is 64% larger, 40% slower, and consumes 5.5 times more power than an equivalent unmasked chip. However, this is still much more efficient than FHE-based approaches. Also, masking for AI will undoubtedly get more efficient as interest in the intersection of AI and hardware security grows.
A Better Set of Proposals
If OpenAI wants to protect model weights, there are a number of hardware security techniques to rely on. Here are better versions of their three proposals, revised using my experience as a hardware security engineer:
When IP is integrated into GPUs, the IP should be checked for hardware trojans. Then, GPUs should be screened for counterfeits and trojans. And GPUs should feature logic locking.
GPUs should prevent side-channel leakage through cryptographic means or by leveraging secure logic styles. All weights should be encrypted at rest, including in on-chip memories.
GPUs should feature unique identities embedded in hardware roots-of-trust that are themselves secured using state-of-the-art methods.
Hopefully, OpenAI and other AI developers will start taking hardware security more seriously and embrace state-of-the-art methods like these to protect their infrastructure. AI is incredibly powerful, and if we want to protect it, we need to protect it well.