Tandon team to design novel microchips so data to be processed without being decrypted

A research team at the NYU Center for Cyber Security (CCS) at NYU Tandon are participating in a major initiative in collaboration with data security company Duality — supported by a $14 million grant from the Defense Advanced Research Projects Agency (DARPA) — to design a revolutionary new microchip (codenamed “Trebuchet”). The microprocessor, to be designed by CCS researchers Mihalis Maniatakos, research assistant professor of electrical and computer engineering; and Brandon Reagen, assistant professor of electrical and computer engineering and computer science and engineering, will accelerate and enable practical applications of an emerging type of encryption, called fully homomorphic encryption (FHE). 

Typically, data encryption protects data in transit: it’s locked in an encrypted “container” for non-secure transit, then unlocked by the other party for analysis. But outsourcing to a third-party is inherently insecure. FHE is an advanced cryptographic technique, widely considered the “holy grail of encryption,” that enables multiple users to process encrypted data while the data or models remain encrypted, preserving data privacy throughout the analytics process. 

The hardware Maniatakos and Reagen are developing as part of the 3.5 year project is of particular value to AI systems because it allows data scientists to train some of the most advanced machine learning (ML) models on encrypted data, enabling organizations to leverage greater amounts of diverse sensitive data for training. 

“With increased privacy concerns and tightening data protection regulations, organizations across industries are looking for secure computing methods to train and execute AI models without exposing sensitive or confidential data,” said Maniatakos.

FHE also secures outsourcing to a third party for processing, without that entity being able to “unlock” that data. One example of secure data outsourcing is the use of privacy-preserving machine learning, critical for such applications as personal DNA sequencing to scan for predisposition to some diseases.

“FHE lets you encrypt your genome, send it to a service that predicts your predisposition to diseases, and you get the result back and decrypt it. The outsourced service cannot see your genome and cannot use it against you in any way,” said Maniatakos.

Over the course of the project, Reagen will focus on hardware architecture, and Maniatakos the logic units for a feature called Large Arithmetic Word Sizes (LAWS) which can be used to accelerate FHE to practical speeds.

The Hardware to handle the numbers 

The underlying mathematics behind FHE, developed in 2009, involve operations on polynomials of very high degrees (in the order of thousands) and with very big coefficients, requiring resource- and time-consuming computation. The solution lies in hardware that can natively process these very big numbers. Reagen, a computer architect who designs chips that can do very specific things to make them run much faster than general purpose hardware, is tasked with laying the groundwork for this chip. 

“To enable homomorphic encryption, my job is to take the building blocks of this unit — not just memory, but a lot of parallel functionality as well as the LAWS units Mihalis is designing — and customize, organize and allocate them on chip for a given area to maximize performance, which, at the end of the day, is the fundamental inhibitor of this encryption process. In fact, if performance were not an issue, everyone would be using FHE for the simple reason that you would never have to decrypt data. My job is to reclaim that performance.” 

Function-specific hardware (think of a graphics card, for example, or processing units designed for machine learning operations) is nothing new. For FHE, the key is in the bits a computer uses to represent numbers for the rapid calculations it performs. A typical processor’s architecture, organized for 32 bit operations, would take far too long and use too many resources to process the huge numbers required for FHE, explained Maniatakos. 

“You could do it, but it would be incredibly slow. It would be six orders of magnitude slower than unencrypted computation. Our task is to bring down the slowdown to maximum one order of magnitude.”

At 32 bits, he explained, the largest number a chip can process is two to the power of 32 — four billion —which is fine for most applications. The problem is that with FHE the numbers processed are much bigger than 32 bits, in the order of thousands of bits and can’t be represented natively, so computer has to split these numbers up, which incurs a lots of overhead.

“Imagine we have a list of numbers, say one through 10,” explained Reagen. “Let’s say that we first encrypt this data in such a way that this very small list of numbers turns into two lists where each element is 10 to 100 times as big. And suddenly, right off the bat, you increase the number size by a factor of 20 to 200. Now if you want to execute, say, multiplication functions on this bloated data you are not just going to be doing multiplication, you have to do many other functions that transform representations of the data. These meta-functions, in fact, take longer to run than the actual multiplication.” 

He added that designing customizations such as parallel hardware helps vastly increase computation speed by allowing multiple processes to happen at once at maximum throughput. Working as a team is important.

“The key challenge in designing effective architectures is how to balance all of these processes and keep them fed with data, and collaboration with someone designing LAWS for the functional hardware units is critical to making sure performance is not left on the table.”

The CCS team, in 2018, was actually the first to develop an early version of a homomorphic encryption chip that accelerated partial HE, allowing expression of a limited set of algorithms. This earlier version, making CCS the first to create a homomorphic encryption ASIC accelerator, allowed such applications as vote aggregation, and smart grid data applications. 

For this project, the NYU Tandon team, after designing the chip, will hand the baton to collaborators at the University of Southern California Information Sciences Institute, Carnegie Mellon University, SpiralGen Inc., Drexel University, and TwoSix Labs Inc. for integration, prototyping, fabrication and other processes.

The project at NYU Tandon will also fund four graduate students: Deepraj Soni, under the guidance of CCS Director Ramesh Karri; Homer Gamil, a research assistant under Maniatakos, who begins his Ph.D. work at NYU Tandon in September; and Karthik Garimella and Benjamin Heyman, doctoral and master’s students, respectively, under Reagen’s guidance.