# NRG1 Report - ZKEmail.nr & Z-Imburse
Mach 34 is excited to bring our final report on Noir Research Grant round #1! We've spent the past few months working to bring ZKEmail to the powerful Noir language. Furthermore, we've demonstrated a production-grade integration of ZKEmail.nr with Z-Imburse - an Aztec native application for self-authorized reimbursements using email receipts.
Two of the industry's top auditing teams, [Consensys Diligence](https://diligence.consensys.io/) and [Veridise](https://veridise.com/), partnered with Aztec on the NRG#1 to audit Z-Imburse and ZKEmail.nr.
You can view the audits here
* [Consensys Diligence Audit](https://github.com/zkemail/zkemail.nr/blob/main/audits/v1/consensys_diligence_v1.pdf)
* [Veridise ZKEmail.nr Audit](https://github.com/zkemail/zkemail.nr/blob/main/audits/v1/veridise_v1.pdf)
{%youtube 2tbIQwgzGSo %}
You can read the [previous NRG#1 half-way report from Mach 34 here.](https://hackmd.io/@IQZ-5dJ4QGGu4K6oX71X7w/SyOjrd30R/edit)
## [ZKEmail.nr](https://github.com/zkemail/zkemail.nr)
If you're unfamiliar, [ZKEmail](https://prove.email/) is a tool for proving the authenticity and content of emails using the DKIM signatures placed on most of all emails.
[ZKEmail.nr](https://github.com/zkemail/zkemail.nr) is on track to be the default solution for client-side proving of emails in zero knowledge! In Circom, a browser-based prover can *barely* verify a DKIM signature over a small header - forget checking the body or header fields and say hello to a massive proving key and proving cost. Meanwhile, our benchmarks for [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) demonstrate clear efficacy:
* M1 Mac 16 gb memory
* 222,783 gate circuit (default [2048-bit DKIM verification circuit](https://github.com/zkemail/zkemail.nr/blob/main/examples/verify_email_2048_bit_dkim/src/main.nr) - 512 byte header & 1024 byte body)
| | Plonk (Browser) | Honk (Browser) | Plonk (NodeJS) | Honk (NodeJS) | Plonk (Native) | Honk (Native) |
| -------------------------- | --------------- | -------------- | -------------- | ------------- | -------------- | ------------- |
| Cold Start (Single Thread) | 89.01s | 19.63s | 88.26s | 20.14s | N/A | N/A |
| Cold Start (Multithreaded) | **22.10s** | **6.18s** | **21.51s** | **6.47s** | N/A | N/A |
| 10x avg (Single Thread) | 59.50s | 16.63s | 59.37s | 16.90s | **6.81s** | **2.34s** |
| 10x avg (Multithreaded) | **12.59s** | **4.23s** | **12.72s** | **4.43s** | N/A | N/A |
| Witcalc | .75s | <--- | .88s | <--- | .90s | <--- |
You can check these results for yourself [with the benching repository](https://github.com/Mach-34/zkemail.nr-bench).
:::info
The 'Cold Start' refers to the fact that a common reference string (CRS) must first be retrieved in order to prove. Additionally, the wasm executable must be retrieved and instantiated.
'Cold Start' proving would refer to a situation where you want to immediately start running a proof on page load - you could circumvent cold starts if your UX can hide instantiation.
Native (desktop/ mobile) does not include a cold start benchmark as the CRS is pre-downloaded and the prover is compiled.
:::
### Use
The library exports the following functions:
dkim::RSAPubkey::verify_dkim_signature - for verifying DKIM signatures over an email header. This is needed for all email verifications.
headers::body_hash::get_body_hash - constrained access and decoding of the body hash from the header
headers::email_address::get_email_address - constrained extraction of "to" or "from" email addresses
headers::constrain_header_field - constrain an index/ length in the header to be the correct name, full, and uninterrupted
partial_hash::partial_sha256_var_end - finish a precomputed sha256 hash over the body
masking::mask_text - apply a byte mask to the header or body to selectively reveal parts of the entire email
standard_outputs - returns the hash of the DKIM pubkey and a nullifier for the email (hash(signature))
Here's a minimal circuit verifying the DKIM signature and email body:
```rust
use dep::zkemail::{
KEY_LIMBS_1024, dkim::RSAPubkey, get_body_hash_by_index,
base64::body_hash_base64_decode, standard_outputs
};
use dep::std::hash::sha256_var;
// Somewhere in your function
...
// verify the dkim signature over the asserted header
pubkey.verify_dkim_signature(header, signature);
// extract the body hash from the header
let signed_body_hash = get_body_hash(header, dkim_header_sequence, body_hash_index);
// compute the sha256 hash of the asserted body
let computed_body_hash: [u8; 32] = sha256_var(body.storage, body.len() as u64);
// constain the computed body hash to match the one found in the header
assert(
signed_body_hash == computed_body_hash,
"SHA256 hash computed over body does not match body hash found in DKIM-signed header"
);
...
```
The repository additionally comes with the JS package `@zkemail/zkemail-nr` which is used just like the original circom sdk:
```js
// example of generating inputs for a partial hash
import { generateEmailVerifierInputs } from "@zk-email/zkemail-nr";
const zkEmailInputs = await generateEmailVerifierInputs(emailContent, {
maxBodyLength: 1280,
maxHeadersLength: 1408,
shaPrecomputeSelector: "some string in body up to which you want to hash outside circuit",
});
...
```
### Learnings from ZKEmail.nr V1
Throughout the process of developing ZKEmail.nr, we've picked up on a few ideas, patterns, and tricks to use in Noir (and ZKEmail)! Here are some of the most interesting things we think you should know:
#### Conditional Array Access & Unconstrained Functions
Arithmetic circuits have weird quirks - seemingly normal operations like if statements or division can cause extreme blowups in circuit constraints. Using unconstrained functions in tandem with constrained operations is one of the best ways to mitigate these occurences.
Take an example where we'd like to pack bytes together, perhaps for a [SHA256 hash](https://hackmd.io/Rt01fpVAQ7WvcLIISIzzHg?both#Output-Packing):
```rust
global SIZE_1: u32 = 64;
global SIZE_2: u32 = 128;
fn main(
item_1: [u8; SIZE_1],
item_1_length: u32,
item_2: [u8; SIZE_2],
item_2_length: u32
) -> pub ([u8; SIZE_1 + SIZE_2], u32) {
let mut packed = [0; SIZE_1 + SIZE_2];
let mut index = 0;
for i in 0..SIZE_1 {
if i < item_1_length {
packed[index] = item_1[i];
index += 1;
}
}
for i in 0..SIZE_2 {
if i < item_2_length {
packed[index] = item_2[i];
index += 1;
}
}
(packed, index)
}
```
This pattern looks unassuming, but actually results in over 640,000 constraints! This is due to conditional writing to RAM tables (mutable arrays). Instead, we can use an unconstrained function to pack the array, and simply check the output:
```rust
fn main(
item_1: [u8; SIZE_1],
item_1_length: u32,
item_2: [u8; SIZE_2],
item_2_length: u32
) -> pub ([u8; SIZE_1 + SIZE_2], u32) {
pack(item_1, item_1_length, item_2, item_2_length)
}
unconstrained fn __pack<let N: u32, let M: u32>(
item_1: [u8; N],
item_1_length: u32,
item_2: [u8; M],
item_2_length: u32
) -> [u8; N + M] {
let mut preimage = [0; N + M];
let mut index = 0;
for i in 0..N {
if i < item_1_length {
preimage[index] = item_1[i];
index += 1;
}
}
for i in 0..M {
if i < item_2_length {
preimage[index] = item_2[i];
index += 1;
}
}
preimage
}
fn pack<let N: u32, let M: u32>(
item_1: [u8; N],
item_1_length: u32,
item_2: [u8; M],
item_2_length: u32,
) -> ([u8; N+M], u32){
// unconstrained construction of array
let packed= unsafe { __pack(item_1, item_1_length, item_2, item_2_length) };
// check packing of element 1
for i in 0..N {
let out_of_bounds = i > item_1_length;
let matched = item_1[i] == packed[i];
assert(out_of_bounds | matched);
}
// check packing of element 2
for i in 0..M {
let out_of_bounds = i > item_2_length;
let idx = i + item_1_length;
let matched = item_2[i] == packed[idx];
assert(out_of_bounds | matched);
}
// set length
let length = item_1_length + item_2_length;
// return
(packed, length)
}
```
We've turned this operation into ~6000 constraints (of which over half are initializing the range check lookup table)!
You can see more about this specific practice, including demonstrations with both arrays and BoundedVec, [in this demo repository](https://github.com/Mach-34/noir-conditional-ram-best-practices).
We chose this example as we find conditional writes to arrays ubiquitous and unavoidable - but as you go about developing circuits, keep an eye out for situations where you can leverage constrained checks on unconstrained functions to optimize your circuits.
#### Output Packing
One small trick when settling on the EVM is to pack all of a circuit's outputs into a SHA256 hash. This trade adds client proving cost, but each public output increases the verifier work (and therefore gas cost). SHA256 is chosen for its efficiency in the EVM.
The strategy is:
1. Decompose all outputs into bytes in circuit
2. Pack bytes into a single array in circuit
3. SHA256 hash the packed bytes in circuit
4. Output the hash as a single "payload" public output from the circuit
5. The prover should know the preimage of the hash and supplies the values as function arguments to a contract call
6. The contract should reconstruct the hash using `sha256(abi.encodePacked(...arguments))`
7. The contract should pass the reconstructed hash to the verifier as the only public output to constrain the unpacked outputs
#### Multithreaded Proving in Browsers
The JS proving backends for Noir can take advantage of `wasm_bindgen_rayon` which uses web workers and `SharedArrayBuffer` to simulate multithreading in the browser. Depending on the client's hardware, you can easily cut 50% off of the proving speed!
Ostensibly, this is very easy:
```js
// in the browser
...
const threads = window.navigator.hardwareConcurrency;
const backend = new UltraHonkBackend(circuit.bytecode, { threads });
...
```
What's the catch? SharedArrayBuffer requires the server response headers to contain cross-origin isolation policies:
```
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
```
This issue is not unique to Aztec/ Noir and you can find plenty of information about integrating this. One thing we haven't tried to futher mitigate this cost is hidden iframes...
#### Profile Your Circuits
Flamegraphs are a great way to see what parts of your code are the most resource intensive. When trying to optimize Noir code, the profiler is an incredibly useful way to analyze where your circuit can be optimized.
Take the example above - we can see that `sha256_var` accounts for a significant potion of our circuit. Sha256 in Noir is a black box function that we can't really optimize any further, and trying to optimize this circuit further would be like trying to squeeze blood from a stone. At least we know that the cost of this circuit is pretty rigid! In other examples, you might find your circuit is spending significant resources on something that doesn't make sense, useing this lead to whittle down the proving cost.
Install the profiler with these instructions:
```console
git clone https://github.com/noir-lang/noir && cd noir
cd tooling/profiler
cargo install --path .
# Now you should have the profiler installed with:
noir-profiler
```
#### Email Circuits Are Hard... For Now
As we mention in the Z-Imburse section, we originally intended to support far more emails. Unfortunately, the process of writing the pattern matching logic by hand for each email template proved to be very time consuming. For an idea like Z-Imburse, we could expect to see hundreds of circuits for different receipt types. The technical debt here is frankly insurmountable.
As we mention in the "[Future of ZKEmail.nr](https://hackmd.io/Rt01fpVAQ7WvcLIISIzzHg?both#Future-of-ZKEmailnr)" section, we integrating regex and supporting the ZKEmail templating SDK. The templating will allow us to define the patterns we expect to extract from a circuit and quickly generate a circuit that meets the expectations of the product. Like Circom, this will make the process of quickly developing email verifier circuits in Noir attainable for even novice circuit developers.
### [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) Audit
We asked the auditing teams to focus mostly on [ZKEmail.nr](https://github.com/zkemail/zkemail.nr). After all, the SDK will see widespread use by developers who should expect to build on top of rugged and trustworthy code. While Veridise took on a balanced review of both Z-Imburse and [ZKEmail.nr](https://github.com/zkemail/zkemail.nr), Consensys Diligence focused exclusively on auditing the [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) codebase.
The [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) audits were successful in turning up a host of deficiencies and edge cases in our first production iteration of the SDK. Some of the most egregious exploits identified include:
* Sneaking malicious body hashes into the DKIM header
* Creating multiple nullifiers from the same DKIM signature
* Using "simple-canonicalized" (instead of "relaxed") header encoding to juke header pattern matching and overflow constrained header sequences
Of course, these types of issues are *critical* - while ZKEmail.nr could allow private, no-relayer key recovery and authentication for AA, ZKEmail.nr can't be trusted to secure major economic value while these exploits are around.
We've implemented the prescribed fixes by Veridise - a few issues require a second attempt to resolve, but in general these vulnerabilities have been addressed. The Consensys Dilligence fixes are still WIP and will be included as [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) in the immediate future. It is important to note that many issues relating to pattern matching will be deprecated by the integration of ZK-Regex (as mentioned in "[Future of ZKEmail](https://hackmd.io/Rt01fpVAQ7WvcLIISIzzHg#Future-of-ZKEmailnr)").
### Future of [ZKEmail.nr](https://github.com/zkemail/zkemail.nr)
As mentioned, [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) offers unrivaled client-side proving capabilities. However, there is one very important part missing from [ZKEmail.nr](https://github.com/zkemail/zkemail.nr): regex pattern matching. Currently, [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) has a rigid and limited API for checking header fields, with a specific tool for extracting email addresses. However, it is up to the developer to manually do pattern matching beyond this using [noir_string_search](https://github.com/noir-lang/noir_string_search) or even with manual checks. Integrating regex means that [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) will be at **full feature parity** with the Circom version, giving developers the same tools to easily check for expected content in an email. This also fortifies the security profile of [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) - while Noir regex now needs to be audited, we can rely on all of the Circom-based auditing and test vectors to lock down the vast majority of the edge cases and exploits that were encountered in the audits.
What is especially interesting about feature parity with Circom is the fact that we'll be able to integrate with the official [ZKEmail SDK](https://registry.prove.email/). The SDK prescribes a set of "parameters" that describe the entire circuits:
This is a massive unlock for multiple reasons:
1. Any existing template already uploaded to [registry.prove.email](https://registry.prove.email) can be retrieved for your own reuse - *including* templates originally built for ZKEmail in Circom
2. Instead of having to manually write circuits to verify emails, developers can just describe the parameters of the email verifier and automatically codegen the circuit
We can attest to the fact that this extension of ZKEmail would have saved us considerable time on Z-Imburse. While we labored over an email verifier for the United receipt, the template registry already has the verifier template (shown above). With regex and template code-gen, we could have made small changes to the pattern for our own needs over a few hours rather than spending days lining up pattern matching!
We're aiming to get [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) fully integrated in production in the ZKEmail SDK by EthDenver (end of February 2025). Future audits will likely follow, along with updates as Noir v1 reaches ossification.
### Engaging with [ZKEmail.nr](https://github.com/zkemail/zkemail.nr)
You can open issues/ pull requests in the [ZKEmail.nr](https://github.com/zkemail/zkemail.nr) github repository! This is the best central repository for raising serious issues or proposing changes.
If you have general inquries or are seeking assistance with ZKEmail.nr, you can join the official [ZKEmail telegram group](t.me/zkemail).
## Z-Imburse
Work in the crypto ecosystem often presents situations where grantees pay for costs upfront, and some company or foundation eventually reimburses them for the funds. This slow, reactive, and bureaucratic process is an ironic antithesis of the infrastructure these organizations are building. We propose ZImburse: an Aztec-based web app for instant self-authorizing reimbursement dictated by organizational policies.
We build this application from a perspective of a real problem we and many others face- just see DaoHaus's Cookie Jar extension. If ZImburse can achieve a reasonable UX, the likely adoption would lead to drastically reduced friction and overhead for organizations and contributors in web3 alike.
### Under the Hood
#### Claim process
1. The organization sets up a Z-Imburse contract instance and funds the escrow with USDC
2. The organization creates an entitlement type - for instance, up to $100/ month that can be claimed from the escrow using an AWS Receipt sent via email
3. The organization gives this entitlement to a recipient address
4. The recipient address can upload their email into the Z-Imburse web app, prove authenticity, and export the amount they paid to AWS that month
5. The Z-Imburse contract constrains the timeframe and allows the recipient to withdraw USDC, up to the max value set by the escrow manager
#### Shared State
One of the most ambitious (and disappointing) aspects of Z-Imburse is the shared state of entitlements. An entitlement is created by an escrow admin and given to an escrow participant/ claimant.
Originally, we attempted to hijack the native, protocol-level nullifiers used to "delete" notes. The objective:
1. create two different copies of the same note controlled by two different users
2. have both notes create the same nullifier
3. when one user nullified the note, the other user's PXE would pick up the state change and understand their copy of the note was now spent
Though 1 and 2 are trivial, 3 proved impossible due to the current PXE implementation. The PXE contains a database of valid and nullified notes. The PXE will find transactions initiated by accounts it can control and nullify those notes as it synchronizes, and the PXE will nullify notes immediately when it creates a transaction that will nullify the note. The problem is that the PXE currently is incapable of looking at nullifiers from transactions with foreign origins to associate them with notes it currently stores.
This is a solvable problem - the PXE will synchronize every time it sees a new block, so the PXE could be updated to just check all of its notes. However, this is a relatively niche edge case for the PXE to be checking *every block*, and it could seriously deteriorate the performance of the PXE. We could ship our own fork of the PXE, but this practice should likely be shunned as the PXE holds the most sensitive data of Aztec users.
Therefore, we reverted to a primitive, app-level solution to nullification - a simple immutable mapping of nullifiers to booleans. A transaction emitting a nullifier must enqueue a public call to add the nullifier to this mapping, and any transaction that requires checking this nullifier can do so in the private context.
From a soundness perspective, this is no less private or secure than protocol-level nullifiers. The only transactions emitting nullifiers are ones that also withdraw money from the escrow, meaning the transaction would already be associated with the escrow contract. Enqueued public function calls therefore do not leak any information.
However, we lose the key benefit of being able to easily prune invalid entitlement notes from PXE's. We have events notifying the PXE of nullified notes; a slightly more advanced integration could simply hide invalid notes from the user in the UI. However, we elected to add a function call for the user to just nullify their invalid notes. This is not a desirable pattern and we have ideas on better solutions, but for the sake of shipping the MVP we cut this corner.
#### Large Emails
Another issue we had to confront was the size of some emails. Below are example United and Linode receipts:
<div style="display: flex; justify-content: center; gap: 20px;">
<figure>
<img src="https://hackmd.io/_uploads/BJhaDFfDye.jpg" alt="Linode Receipt" style="width: 150%;">
<figcaption style="text-align: center;">Linode</figcaption>
</figure>
<figure>
<img src="https://hackmd.io/_uploads/rk9oOFGPyg.jpg" alt="United Receipt" style="width: 150%;">
<figcaption style="text-align: center;">United</figcaption>
</figure>
</div>
The entirety of the Linode receipt is shown on the left - less than 800 bytes with no HTML or CSS makes it a model candidate for ZKEmail proving and can be readily done in around 250k constraints. Meanwhile, the first 25% of the United receipt on the right is a nightmare. It is filled with (unshown) terms of service text and has a lot of HTML/CSS formatting the email. From our data set, these emails can be anywhere from 40kb to 100kb. In order to access data from the body of the email, we need to hash the asserted text and match it to the signed body hash in the header. While we can use partial hashing to precompute up to the start of the text we need from the body of the email, we have to hash *everything* after that. In the case of United, this can easily become a 10 million constraint circuit.
The "correct" solution here is to offload part of proving the hash to the server. Just like partial hashing involves precomputing the hash outside the circuit and picking up the state inside the circuit, we can compute some rounds of the hash on the client and some on the server and recursively verify. Ideally, the client will hash all sensitive data and the server is just responsible for hashing mundane content like an appended terms of service and any HTML/CSS. While this leaks *what* kind of email someone wants to prove, it doesn't leak any other real information.
In practice to land this sort of computation on the Aztec L2, we would need to create a complex auto-scaling API with multiple concurrent workers proving chunks of the hash then recursively combining proofs in a tree-like structure to reduce the proof down to a single succinct proof of the final hash state for the client to consume. This system was beyond what we were prepared to support for a proof of concept, and we chose to cheat a little bit here for the Z-Imburse PoC.
We built the basic circuitry for chunking hashing into 2048-byte blocks and simply relied on recursive contract function calls to produce the hash without overflowing a memory limit. This works - it takes 20-30 seconds to simulate this witness on the browser, and it proves that the logic can be constrained. However, while this works in the simulated sandbox, trying to prove this is prohibitive.
### Z-Imburse Audit
Issues Veridise found with Z-Imburse can be classified in be classified as either "app level" or "protocol level" problems. Application-level issues have to deal with the spec of Z-Imburse itself, including issues like:
* Centralization risk around escrow admins and DKIM keys
* Lack of association between entitlements and recipient addresses
* Potential for certain email receipts to be reissued upon request.
Meanwhile, protocol-level issues centered around our attempt to hijack native nullification in Aztec. Many of the issues identified were actually consequences of the [faulty nullifier implementation](https://hackmd.io/Rt01fpVAQ7WvcLIISIzzHg?both#Shared-State) we attempted. Largely these issues were addressed by swapping out protocol-level nullifiers for app-level nullifiers.
### Future of Z-Imburse
We spoke to parties we thought might be interested in Z-Imburse at Devcon - while there is certainly demand for this kind of tool, there are a few problems:
- There are a WIDE range of email receipts that need to be supported to start offering enough value to use Z-Imburse
- Liquidity that would use this platform is concentrated on existing EVM chains, demanding either a re-write that sacrifices privacy or an adapter to use the L2 portal once it is live
- Not all types of receipts are accessible by email, and a ZKTLS solution would also have to be added in order to access remaining expenses that emails can't support
With substantial requirements to build Z-Imburse into a full product, a questionable servicable addressable market, and a span of time where Aztec L2 isn't yet deployed, sustained development of Z-Imburse would have diminishing returns. Instead, we will ensure Z-Imburse is deployed on the Aztec testnets as a demonstration. We are still primarily invested in driving onchain business logic with attested offchain data, and expect that elements of Z-Imburse reappear in our next projects.
Sign in with Wallet
Connect another wallet