## TOC
* [What is ZKML?](#What-is-ZKML?)
* [Definitions](#Definitions)
* [Privacy Approaches](#Privacy-Approaches)
* [Machine Learning](#Machine-Learning)
* [Quantization](#Quantization)
* [Why ZKML?](#Why-ZKML?)
* [What ZKML is Not](#What-ZKML-is-Not)
* [Applications of ZKML](#Applications-of-ZKML)
* [Fighting deepfakes](#Fighting-deepfakes)
* [Attested Camera](#Attested-Camera)
* [Attested Microphone](#Attested-Microphone)
* [Decentralized Prompt Marketplaces](#Decentralized-Prompt-Marketplaces)
* [Decentralized Bounty Programs](#Decentralized-Bounty-Programs)
* [Sybil Resistance](#Sybil-Resistance)
* [Intent Agents](#Intent-Agents)
* [Teams Building ZKML](#Teams-Building-ZKML)

# What is ZKML?

Zero knowledge machine learning, ZKML, refers to the process of running a machine learning model inside of a zero knowledge circuit, inputting machine learning data and model weights into a ZK circuit to generate a proof π. The outputted proof can be verified on a blockchain. For example, the proof may attest to the fact that a machine learning model ran on some input.

The use of Zero-Knowledge Proofs (ZKPs) in neural networks (NNs) provides a balance between privacy and transparency, depending on the desired use case.

* Private input public output - some situations call for the model weights to be public while keeping input data private, like in face recognition. Here, one might trust the model but wouldn't want to expose sensitive facial features.
* Public input data, private weights - Companies might want to preserve their proprietary models, but need to show consistency in using the same model repetitively. In this case, public input data ensures the correctness of data.
* Private inputs, public weights - as seen in biometric authentication or private image/data marketplaces.
* Public input, private model - like in monetization algorithms or data science competitions, where the methodology needs to be kept proprietary.

Indeed, there are many potential ZKML applications in the future based on the above combinations of privacy preferences. Today, the concrete use cases center on model integrity. Model integrity may refer to things like reproducibility, validity, and AI fairness. Future use cases of ZKML could include fighting deepfakes with attested sensor cameras and microphones. Further down the road AI *agents* will serve as human surrogates who act to satisfy human intentions. ZKML will be important to integrate so users have proof that agents acted in their interests while keeping their identities and information about intentions private.

In this article, we will focus on three aspects of ZKML.
* First we will review privacy approaches, machine learning, and quantization at a high level to gain an intuition for the technology.
* Thereafter, we will review the motivations and attempt to answer why ZKML? Including addressing any misconceptions about what ZKML is not
* Finally we will spend time on reviewing current and future applications.

### Caveats
It should be noted that as an experiment I wrote this article with the help of GPT-4. I didn't use GPT-4 to source any information, but rather to help me re-order my notes on the source materials in a readable digest. I've never done this before, but I thought it would be a fun experiment. That said I did edit every line in the article.

## Definitions

### Privacy Approaches

Similar to function privacy in blockchain architectures, there exist a diversity of approaches to achieve privacy in Machine Learning models. Colloquially, ZKML is often used as a catchall to describe multiple methods of achieving privacy for ML models, it is not technically precise. That being said, as memes propogate there is often value in the information they transmit. As such let's briefly review MPC, HE, ZKPs & TEEs.

* Multi Party Computation - MPC protocols share computation across multiple parties such that it is secure against malicious adversaries. In the ML setting this is impractical: it can cost up to 550 GB of communication and 657 seconds of compute per example on toy datasets (Pentyala et al., 2021)
* Homomorphic Encryption - HE allows parties to perform computations on encrypted data without first decrypting the data. HE can preserve privacy of model inputs but can not verify the integrity of the model. HE is also expensive
* Zero Knowledge Proofs - ZKML literature references both Groth 16 and Halo 2 as proving systems. Quantization is a challenge which we will discuss below
* Trusted Execution Environment - TEEs present their own set of risks eg. side channel attacks. This type of solution is likely unfit for privacy preserving ML applications operating in adversarial environments.

### Machine Learning

Machine Learning (ML) is an integral branch of artificial intelligence (AI) that empowers computers to learn and improve from data without explicit instructions. Serving as a cornerstone within the AI hierarchy, ML uses statistical techniques to interpret patterns, allowing computers to predict outcomes and evolve based on experiences. Deep Learning, a subset of ML, further enriches this process. Inspired by the human brain, it employs intricate neural networks that learn from unstructured data, pushing the boundaries of machine accuracy and learning capacity.

The AI *hierarchy* can be represented as a set of Matroyshka dolls, a reference to wooden dolls that are nested within each other decreasing in size with each new doll.

{PlaceHolder picture}

Three key aspect of machine learning include;

* *Pattern recognition*: ML algorithms sift through data to detect recurring trends and relationships. This is the basis for learning and making predictions or decisions.
* *Adaptability*: As they encounter new data, ML models refine their internal rules and parameters, thereby enhancing their performance over time without requiring additional programming.
* *Autonomy*: ML enables systems to make predictions and decisions independently. This capacity is particularly beneficial for tasks that involve complex or large-scale data that would be challenging for humans to process

### Quantization

One of the challenges that exists in ZKML compositions is creating an efficient interaction between the Machine Learning model and the Zero Knowledge proof generation. ML models deal with *floating point* numbers, numbers without a fixed number of digits before and after the decimal point eg. 3.14, 4.2069, 8.888888.

On the other hand Zero Knowledge proofs deal with fixed-point number arithmetic over finite fields. Fixed-point is a method of representing fractional numbers by storing a fixed number of digits of their fractional part.

Neural network inference is typically done in floating-point arithmetic, which is extremely expensive to emulate in the prime field of arithmetic circuits. Therefore there exists a helpful mathematical technique called quantization that allows for "conversion" of a floating input from a continuous or otherwise large set of values (floating-point) to a discrete set (fixed-point). This conversion from large numbers to small integers allows for a more efficient construction of a zk-SNARK.

## Why ZKML?

* ML provider may have some model weights they want to keep hidden
* API provider can use ZK techniques to prove model ran as expected
* Cloud provider/API provider hacked - relevant for medical predictions. If you send proof you know model is run correctly
* But there could be bugs in the model
* Model provider may be lazy to save money run a smaller model but not what you expect

Having algorithms that are biased in different ways. One nice way to fix this is having a social media company they would publish a hash of the program they use for spam detection, ranking, emphasis, deemphasis. They publish the hash and pre-commit to reveal what the code is, except for the weights. In the meantime, they provide ZKPs to you that what you are running in your browser is the output of that program.

In real time this maybe isn’t as helpful, but what you know in a couple of years the program will be revealed and then people can analyze it. Realistically, it has to be delayed because algorithms have to be closed source for some time. But delay them long enough.

For this to be possible, you need to prove what you as the user see is the output of running this code. Maybe a use case of validity ML and not so much ZKML.

* Trust and accountability - You know that a particular model which was agreed upon, even though you don't know the exact parameters of the model, you know this model is the one being run. For example, the twitter algorithm. Even though you don’t know the specifics of the model, you know it's the same one which continues to be used.
* User experience - When thinking about accountability for proof, how does the end user know and believe this green check mark they received they can know and trust. This is hard to do without blockchain. We expect on-chain organizations will need this.
* Verifiability - In the case of ML we have not had this before.

You may be wondering what's so special about ZKML, sounds nice in theory, but is it worth the scarce resource of attention? Indeed, here we will start with some motivations for ZKML. (cite Jason from EZKL.)

* Gives the blockchain eyes to perceive the physical world - make decisions about physical reality, satisfy human intents
* Makes it possible for a human, not a field element, to own digital assets
* Lets Smart contract exercise judgements - deal with any kind of ambiguous situation decide if a contract is satisfied

### What ZKML is Not

Now that we have an introductory level understanding, of ZKPs, ML, and ZKML, let's briefly review what ZKML is not. This is important because in an emergent area of research and design, there can be colloquial misconceptions.

* ZKML does not pertain to the domain of federated learning, where multiple parties train the same algorithm on different systems, later merging these insights at a central location. ZKML can potentially facilitate privacy in such federated settings, but it does not inherently enable distributed training.
* ZKML is not equivalent to a robust watermarking system for models like language models. Contrary to some misconceptions, a minor alteration to the output of a language model, for instance, won't pass ZKML verification, as it would with a watermarking system. Even the slightest modifications can cause the ZKML proof to fail.
* ZKML does not offer hardware-side verification independently. While it can prove the validity of computations, it does not safeguard against potential manipulations of the data source. To verify aspects that involve the physical world, supplementary measures beyond proof are necessary.
* ZKML isn't a standalone tool to confer complete privacy or security. For instance, in the context of decentralized social media, while it can help hide the weights of algorithms, a comprehensive solution would likely involve other privacy-preserving techniques as well.

## Applications of ZKML

The ability to discern AI from human actions and confirm which tasks an AI has been authorized to carry out on a human's behalf is becoming increasingly critical. Various applications of ZKML discussed in this article and those not yet derived (or mistakenly omitted) can help solve this problem by fighting misinformation including deepfakes.

* [Fighting deepfakes](#Fighting-deepfakes)
* [Attested Camera](#Attested-Camera)
* [Attested Microphone](#Attested-Microphone)
* [Decentralized Prompt Marketplaces](#Decentralized-Prompt-Marketplaces)
* [Decentralized Bounty Programs](#Decentralized-Bounty-Programs)
* [Sybil Resistance](#Sybil-Resistance)
* [Intent Agents](#Intent-Agents)

### Fighting deepfakes

Deepfakes are manipulated images or videos, a specific kind of synthetic media where a person in an image or video is swapped with another person’s likeness, with the intent to deceive humans. ^{<a href="#ref1" id="ref1link">1</a>} Deepfakes are becoming ubiquitous and sophisticated, contributing to the spread of misinformation. This misinformation can have significant impacts on society at large and cause disruptions with many negative externalities. Deepfake manipulations involve altering a person's appearance for social engineering purposes. For example Deepfakes can be used to
* Aid state actors in [disinformation campaigns](https://cset.georgetown.edu/publication/ai-and-the-future-of-disinformation-campaigns/) (propoganda)
* Trick businesses into sending money to the [wrong places](https://www.cnn.com/2021/02/18/business/citigroup-revlon-lawsuit/index.html)
* Deceive people into falling victim to [social engineering](https://www.pcmag.com/news/gpt-4-was-able-to-hire-and-deceive-a-human-worker-into-completing-a-task) campaigns (scams)

>The rise of deepfakes raises a critical question: how can we verify the authenticity of visual media in the face of malicious adversaries?
>
> -Kang, Hashimoto, Stoica, Sun in *ZK-IMG: Attested Images via Zero-Knowledge Proofs to Fight Disinformation*, 2022

With attested sensor cameras and microphones, humans can gain assurances about the authenticity of synthetically generated content, which they can use to make judgements about the legitimacy of the information presented to them.

Take a presidential election, for example. If Party A runs a deepfake video on Facebook that shows Party B's candidate saying things they did not say, many consumers of this content may react in a way they would not have if the deepfake did not exist. Requiring Party A to post a proof that attests to the authenticity of the audio and video used in the video would act as a filter for deepfake content.

This is particularly helpful during ongoing political campaigns, where dueling parties may claim “fake news” towards the other's marketing material. Average consumers need a tool which allows them to decipher the authenticity of the content they consume. This has implications beyond social media, but also could improve the credibility of local and national media outlets, whose credibility is often in question as they manufacture consent through salacious headlines in order to capture attention and control the masses.

#### Attested Camera
One potential countermeasure against deepfakes is the use of *attested cameras*, which incorporate hardware that signs pixels immediately upon capture (taking a picture). This digital signature authenticates the original image, ensuring its integrity and trustworthiness.

As an example image a bomb goes off in small town in Montanna. With a digital signature the camera could prove the authenticity of the image and potentially attest to its location and time the image was captured.

However, challenges emerge when users want to edit these attested images. For instance, to crop out certain information, without revealing the original image or compromising privacy. While it is virtually impossible to fully prevent physical alterations, the goal is to increase the difficulty and cost of producing convincing deepfakes, discouraging their creation purely through software manipulation.


In response to this challenge, a new technology called [ZK-img](https://ddkang.github.io/blog/2022/11/13/zk/) has been proposed, which attests to image edits securely and privately. [ZK-img](https://export.arxiv.org/abs/2211.04775v2) is a library for attesting to image transformations while hiding the pre-transformed image. Zk-img allows application developers to specify high level image transformation

ZK-img takes the signature of the original image as a hidden witness and outputs an edited image using ZK-SNARKs, enabling consumers to verify that the edits were done honestly without facial swaps or other deceptive manipulations. However, ZK-img isn't a standalone solution; it needs to be integrated with other technologies like decentralized signature tracking or image and proof registries.

A promising application is the creation of a Trustless Face ID system, where a user takes a photo with an attested camera, performs certain edits or cropping, and inputs this into a face ID model. The system then produces a proof that the edited face matches a previously uploaded image, ensuring authentication while preserving privacy and not revealing sensitive biometric information.

#### Attested Microphone

Attested Sensor Microphones can help fight against deepfakes by proving the provenance of a speakers voice. Also they could be used to prove than an AI agent is authorized to speak on behalf of a human. In an environment where AI-generated audio can mimic human voices flawlessly, we need a reliable chain of trust stretching from the initial capture of audio to its final playback.

Establishing a chain of trust from the audio capture to final playback requires trusting how the audio is captured and how the audio is edited. Attested sensor microphones have a hardware unit that cryptographically signs the audio signal as soon as it is captured. This cryptographic signature is unforgeable, even with AI tools. With this signature, anyone can verify that the audio came from a specific microphone. In order to verify that audio came from a specific individual, that person can publish the public key of the attested microphone.

There are many circumstances where you would want to privately edit the original audio from an Attested Sensore Microphone. For example, intelligence agencies can use background noise to identify your location, which compromises privacy. To preserve privacy, you may want to perform edits like removing the background noise or cutting out parts of a conversation that might contain sensitive information.

There are some Limitations. For example, attested microphones don’t allow you to perform edits on the audio, including edits like noise reduction or cutting out sensitive information. Also, attested microphones are still early in development. However there are some promising results coming out of the latest Eth CC hackathon!

### Decentralized Prompt Marketplaces

Prompts are the initial input to models, guiding their responses or actions. Creating effective prompts can be challenging. They require a clear understanding of the task at hand, as well as insights into how the AI model will interpret and respond to them. A poorly crafted prompt can lead to incorrect or irrelevant responses, while a well-thought-out prompt can drive optimal outputs. This underlines the value in generating and utilizing prompts effectively.

In order to enhance the value and provenance of prompts, the concept of 'Proof of Prompts' has been introduced. This involves taking hidden inputs and using these to generate a ZK-SNARK, which produces an output. This method ensures that the results derived from the prompt are valid and truthful without revealing the input information. You can imagine Prompt NFTs selling on an order-book based marketplace accruing speculative value based on their perceived usefulness.

Moreover, this approach adds a layer of versatility to prompt usage. Once a prompt has been purchased, it can be modified for a wide range of use cases, while still preserving the original intent and truthfulness. This adaptability broadens the applicability of a single prompt, maximizing its value. It also enables the users to tailor the prompt to their specific needs without worrying about compromising the integrity of the original prompt or the results derived from it. This unique combination of authenticity, privacy, and flexibility makes the 'Proof of Prompts' approach a promising development in the effective usage of prompts.

### Decentralized Bounty Programs

ZKaggle is a decentralized bounty platform for hosting, verifying, and paying out bounties, with the added benefit of privacy preservation.

### Sybil Resistance

WorldCoin etc.

### Intent Agents

It’s clear that even tech savy crypto users have struggled to use public blockchains. Many stop at a Centralized exchange because it’s just easier to interface with, provides some privacy, and often offers better liquidity than decentralized exchanges. Also, the trust assumptions are relatively straight forward. In order for crypto to scale to one billion users we need a better user experience onchain. One such emergent idea is LLMs as a wallet interface.

For example, ChatGPT gained 1 billion users faster than any modern technology in the history of the world. I think it’s clear that users like ChatGPT, in the same way they like soft-confirmations from a centralized rollup sequencer. Good UX is hard to argue with. Given the ease of use and path towards mass adoption, what would a ChatGPT type of interface look like and why would it need ZKML?

#### Toy Example

**User**: I’d like to swap 100,000,000 SHIB for ETH. Get me the best price. I’d like my funds stored in my Ethereum wallet.
**ChatGPT**: Do you have any specific execution preferences?
**User**: What?
**ChatGPT**: What price would you like me to purchase ETH at?
**User**: The best current market rate
**ChatGPT**: what is the most you are willing to pay in fees?
**User**: $0.01
**ChatGPT**: I will send your trade to Zk-Shib chain for execution and then move funds back to your Ethereum account via IBC
**User**: What?
**ChatGPT**: I will fulfill your request and ETH will be in your account ~ 2 seconds after you confirm. 

User: Great!
**ChatGPT**: Do you confirm?
**User**: Yes
**ChatGPT**: Your account now has 1000 ETH on Ethereum. For integrity review, here is a link to the block explorer. Here is a ZKP you check to ensure I have executed your tx as specified and fulfilled your order at the best available price. Here is a snapshot of your wallet

address:

------
**0xmyWallet69
account owner**: apriori0x

**Token Balances**: 1000 ETH. 2200 DAI, 500,0000,000 SHIB.
**NFT Balances:** 1 coolCat, 300 Bad Kids



------
**ChatGPT**: If there are any other intentions you have please let me know. I can also provide real time price data as well as simulate your intents. 


**User**: Thank you!

------

In our toy example the user is a new user who has some intention but is unclear how to express it. With a simple interactive conversation the User is able to gain confidence and express their preferences.

Some other abstractions that may help include Centralized exchange plug-in to help users reach the Security domain of choice.Also, fiat on-ramps which abstract away dealing with the users bank directly, simply having the user provide banking credentials and a proof of identity. An easier path to implementation maybe through something like Apple Pay or Cash App.

#### Requirements
* Need ZKPs to make this safe and preserve privacy
* Perhaps ZK/TEE medium term, ZK/FHE long-term
* Proof that Model is what user expects
* How to verify? 2FA style? Something better?
* Proof of execution that user can verify easily
* Proof that intent was executed at best available price
* Does this only work with edge computing where a user generates proofs locally with a local instance of an LLM?
* How easy would it be for an adversary to break?
* Social engineering and fishing attempts most likely?
* How fast can hardware solutions emerge?

It is clear that given the ease of use, this type of front-end user interface will likely win the market and onboard new users faster than anything we’ve seen to date in crypto.

## Teams Building ZKML

- ZKML, ZK-IMG
- PSE (Privacy, Scaling & Exploration)
- EZKL
- Modulus
- World Coin

## References
<a id="ref1" href="#ref1link">[1]</a>: Kang, Hoshimoto, Stoica, Sun, *[ZK-IMG](https://export.arxiv.org/pdf/2211.04775v2): Attested Images via Zero-Knowledge Proofs to Fight Disinformation*, [pg 2, section 2].