Micromix v2 implementation

tags: `micromix` `semaphore` `libsemaphore` `surrogeth`

feedback on working on Micromix V2

Supporting multiple networks and token

For V2 proper testing and handling, configuration and configuration usage needed to be rewriten that leeded to rewrite lots of code
Without this it will be complicate to spam one webserver and backend for each network and token tested

Configuration

Micromix V1 has hardcoded 1 network and 1 token

Configuration need to be addapted to support multiple network and token

Contracts

Micromix V1 Contract Test is dupplicated for ETH and token

Test is made generic, one suite that can be run for all the network and token setup in the configuration file

Micromix V1 Contract deployement deploy hardcoded 1 semaphore/mixer for eth and 1 for token

Contract deployement deploy as a batch the whole network / tokens defined in the configuraion file on one run

Frontend

Micromix V1 Frontend have hardcoded 1 eth and 1 token

Configuration is addapted to be managed in one file to abstract it
duplication of ETH and token are moved to one generic function when possible
user can chose the network and token (page reload)

Backend

Micromix V1 Backend has hardcoded configuration for network and token
Micromix V1 Backend test has hardcoded configuration for network and token

Configuration is addapted to be managed in one file to abstract it
duplication of ETH and token are moved to one generic function when possible

Upgrade the package and the node

By default, when I start working on a project I try to use the last versions of the dependencies when possible for many good reason.
I fist started to updates dependencies but it lead to unsolvables issues.
So I have done the first milestone with node 11
As I get block by some issues on the test I decided again to update the dependencies to make sure it didn't come from bug already solved.
Node 11 is not suppoted and should not be use in production and etherlime need node 11 https://github.com/appliedzkp/maci/pull/174

update ethers 4 to ethers 5

etherlime and react-web3 does nt support ethers 5

Add solc and ethers for contract compilation and deployement instead of etherlime as well as in frontend and in backend
Remove react-web3 and implement context in frontent

update semaphore and libsemaphore to the last version

I try to update but it is not compatible without modification

update other dependencies

Update packages in different modules except React that break too many things

Add cache for test and automate configuration for contract address

Mixer 1 was running with ganache and kovan that are fast (kovan 1 block/4s)
Using Rinkeby and Arbitrum that have 1 block/15s slow down the test and make it quite long to run the suite for all the network and token (1h ?).

Add address cache to deploy only contract that have not been deployed before
Add address cache to use deployed contract in test
Add address file for automatic configuration for frontend and backend when deployed

Contract improvement

Add mixerRegistry to share the list of mixer
Add Surrogeth and Mixer deployement from mixer registry to certify surrogeth and mixer when mixerRezgistry is certified
Fix and divide contract side adding semaphoreLibrary not to excess the mas contract size

The mixer registry address is stored on the git repository in contract/deployedAddresses to enable sharing of mixer threw the network and avoid duplicated mixer that will reduce the anonimity.
curent deployement:

Arbritrum
Kovan
Ropsten
Oasis Lab Paratime*
Skale*

* Some netowrk bug to fix

Surrogeth integration

Fork of surrogeth : https://github.com/jrastit/surrogeth

config file:

old version

SURROGETH_PRIVATE_KEY=0x.............................
LOCAL_RPC_URL=...............
KOVAN_RPC_URL=...............
MAINNET_RPC_URL=...............
SURROGETH_MIN_TX_PROFIT=...
SURROGETH_ERC20_MIN_TX_PROFIT='{"0x8CdaF0CD259887258Bc13a92C0a6dA92698644C0": 1000000000000000000}'
SURROGETH_FEE=...
KOVAN_ALLOWED_RECIPIENTS=0x............................. 0x.............................
MAINNET_ALLOWED_RECIPIENTS=0x.............................

new version

#file generated by mixer config with command npm run surrogeth-info
surrogethPort: 8124
surrogethIp: "127.0.0.1"
network:
    kovan:
        url: "https://kovan.infura.io/v3/5d37c494621a43558d77c90e368d4022"
        chainId: 42
        privateKeysPath: "kovanPrivateKeys.json"
        ForwarderRegistryERC20: "0x0CEFa2EA705192ba1349980025e61D25E9782B16"
        token:
            eth:
                feeAmt: 0.001
            tkn:
                address: "0x6fB66Fe3a00aFF2fD0a373223592D9Ebe21913eF"
                decimals: 18
                feeAmt: 2
            dai:
                address: "0x4F96Fe3b7A6Cf9725f59d353F723c1bDb64CA6Aa"
                decimals: 18
                feeAmt: 0.02
    ganache:
        url: "http://localhost:8545"
        chainId: 1337
        privateKeysPath: "ganachePrivateKeys.json"
        ForwarderRegistryERC20: "0xF12b5dd4EAD5F743C6BaA640B0216200e89B60Da"
        token:
            eth:
                feeAmt: 0.001
            tkn:
                address: "0xAa588d3737B611baFD7bD713445b314BD453a5C8"
                decimals: 18
                feeAmt: 2

Command to configure and start surrogethd from mixer

#generate the configuration file from mixer configuration
npm run surrogeth-info

#run the surrogeth daemon with mixer generated config
#It will automaticaly register on the registry at start 
#if not registered or if ip/port have changes
npm run surrogeth

UI Improvement

General improvement

Update documentation and instalation for Mixer
Merges changes in dependencies : libsemaphore, semaphore, surroget
Update docker intallation

Anonimity Level

The mixer 1 version based the requirement for the user to wait until the next day for withdraw. That is good in the case there are enought deposit the same day. With v2 and the support of multiple network and token it is most likely not to have enought deposit the same day. Calculation the expected time for enought deposit seams quite complicate. To make it simple I indicate the number of deposit to the user and the anonimity level to help him to wait longer if needed.

One side effect is that to get this information the user will ask the RPC server the number of deposit since the block he used to deposit till he trigger the withdraw throw surrogeth and it can lead for a malicious RPC server to geter some information to be able to link the deposit to the withdraw. This can be offuscated in the future if needed chosing a random block for the query

Issues to fix

LibSemaphore

Thread do not terminate and is reported after test execution, thread end is commented
lib semaphore version and supposed commit:
- '0.0.16': '2019-10-20T16:53:37.835Z' => 835ac67cc868c02350e62db1af8e70cfcaecb0cf
- '0.0.17': '2019-11-26T13:05:42.749Z' => 241df9961bc11136cec907172bc48f6291339f6a
- '1.0.0': '2019-11-29T10:13:31.310Z' => 1eb36ec7fac2fd2767a4cc3b3cc9238a99a60291

https://github.com/jrastit/libsemaphore/blob/v0.0.16/ts/utils.ts : 70 : //groth16.terminate()

It will be fix when remplecing websnark by wasmsnark in the same file

Semaphore

On Arbitrum and Skale only:
To reproduce the issue:
run the contract test: npm run test-withdraw
run the backend test on Skale: npm run test-eth or npm run test-token
message: 'the proof is invalid according to pre-broadcast checks'
the problem appen when calling the function: preBroadcastCheck on semaphore.sol that return false instead of true due to the subcall of verifyProof from verifier.sol that return false

verifier.sol is a generated file.
the proof is calculated as well by generated files.

The issue does not appen en Kovan, Rinkeby and local Ganache.

It imply mixer/semaphore/libsemaphore/snarkjs

running the contracts test verifyProof in generated contract verifier.sol reject the proof created by genproof from libsemaphore verifyProof

Surrogeth

gas limlit to cloose to transaction, make tranaction rejection

Future improvements

Security concern

Base on good practice, private key must be stored encrypted and critical data must be backup.

Crypt private key on the disk and ask for paraphrase on test, backend and surrogeth launch
Run the frontend and surrogeth with tor
Offuscate RPC call not to give information that can be linked to deposit when making the withdraw.
Crypt the ZK private key in the broswer storage
Ability to export mixe the private key to save it outisde of the broswer
Import the mixe private key inside the broswer

Ui improvement

Ability to chose the deposit to withdraw
Check if the proof have not been use
Mint token for test in the UI

Sorrogeth improvement

Deploying a smart contract to trick surrogeth will give the possibility to get money from surrogeth. A simple one using random will have a behavior that change between the surrogeth check and the real contract execution.

Restrict contract call to mixer only to avoid trick to stole money from surrogeth
Add http or https in surroget broadcaster url
Add surrogeth versioning on registering on smart contract to avoid incompatibility
Add docker for surrogeth

Avoid the need of Surrogeth for compatible network

Pay the block miner instead of the relayer
- transfer the fee to block.coinbase this is the current block miner’s address to execute a transaction without fees

Need to check with Arbitrum or Optimism if it can work
Will not work as it is on ethereum mainet
Let Barry also check with our BLS project because they would also benifit from this :)

Notification or transaction processing when enought anonimity

The broswer can notify the user when the anonimity level is ok
The fronted can process the withdraw automaticaly when anonimity level is ok

Auto adjust eth fee and token fee depending on it value and network fee

Relayer fee are static but the price to process a transaction can varry by 10x in a year.
With surrogeth, fee can be adjusted bu the client before sending the transaction in a negociation with the daemon but it mean that the receiver will receive less.
Surrogeth can adjust it fees daily or in real time depending on market value and gas price.

Efficient in-browser proof generation using WASM

It semas it is quite complex due to the number of resources and library used as well as depending on crypto algorytm optimisation.

Changes in libsemaphore for a dependency on wasmsnark instead of websnark
Generate proof in background, while waiting for enought anonimity
Reduce download size (actual circuit.json 111,99 Mo, proving key.bin (62.81 Mo), )
Using more recent version of snarkjs
Using more recent version of circomlib
optimise semaphore circuit definition
Generate circuit in ASM or zip it for download

Refactoring and code improvment

Make it celan

merge config file and deployedaddress.json for frontend
remove duplication of code for generating proof for withdraw and quick withdraw.
remove duplicated ABI and contract source
remove duplication and move contract related operation to contracts module
divide frontend big file to component (cutdown, quickWithdraw)

simplify modules

remove backend and backend-test and add them to contracts with surrogeth
remove the mixer-utils package (Used only for two function)
move the mixer-config to mixer-contract to as used only for configuration file
rename mixer-contracts to micromix and mixer-frontend to micromix-ui

improve test

add hook for typescript compile and test before commit
auto deploy surrogeth agent needed for test
Add UI automated test

dependency on old libraries

Semaphore version used does not seams to install with npm i with node 15

have semaphore install and run with node 15

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`	在筆記中貼入程式碼
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.