Modular ✨ generator design

# Modular✨ Generator Design A potential new modular generator design for the Kalypso generator in-order to facilitate the support of multiple generators within a single Oyster enclave. ## Limitations of the previous monolithic generator design 1. Restricted to only one language for generator executable. 2. Difficulty in integrating new generators into the existing generator without bloating the generator executable. 3. If one of the generators fails for any reason, all the generators need to be restarted. ## Components for the new generator design * **generator-client** (Already existing, but needs modification): An HTTP API server used to set up the generator config JSON file required by the generator (The JSON config file includes the contract addresses and other details). It is also used for starting/stopping the generators and listener. * **listener**: Listens to newly assigned ask events and forwards the ask inputs to the respective generator based on the port defined in the config file by the user during the config setup. The listener also submits the proof returned by the generator on-chain. * **generator**: Each generator is an HTTP server always listening for new ASK requests from the listener. The proof will be generated by the generator, and the generator proof will be returned to the listener to be submitted on-chain. `Note: In the previous design, the listener and generator were part of a single executable running on supervisord and managed by the generator client.` ![GenNewDesign](https://hackmd.io/_uploads/HyDqvztq6.png) ## Generator requirements Every generator must be an HTTP API server that provides the following two endpoints in order to generate proofs requested by the listener: 1. generateProof 2. benchmark ### API endpoints specification 1. `POST` `/api/generateProof` ##### Body (**JSON**) ``` { "ask": Ask, "private_input": Vec<u8>, "ask_id": u64, } ``` ##### Ask struct for reference: ``` { "market_id": U256, "reward": U256, "expiry": U256, "time_taken_for_proof_generation": U256, "deadline": U256, "refund_address": Address, "prover_data": Bytes, } ``` ##### Response `Note` : The data field will return the zero-knowledge proof along with status code *200* if the inputs are valid and signature along with status code *400* if the inputs are invalid. ``` { "message": String, "data": String } ``` 2. `GET` `/api/benchmark` `Note` : The data field should return the proof generation time in milliseconds along with a status code of *200*. ##### Resoponse ``` { "message": String, "data": String } ``` ## Setup process for the new generator 1. To begin, users generates individual HTTP servers for each generator, such as zkbob and noir. These generators accept both the circuit's public and private inputs and subsequently transmit the proof to the listener upon generation. Users must run the generators as separate programs under supervisord. ##### superivsord.conf example for zkBob generator ``` [program:zkbob-generator] command=/app/zkbob_generator autorestart=true autostart=false environment=PORT=6000 stdout_logfile=/dev/stdout stdout_logfile_maxbytes=0 stderr_logfile=/dev/stdout stderr_logfile_maxbytes=0 ``` 2. When a user operates a generator, they must complete configuration procedures with the assistance of the generator-client operating within the enclave. Initiated by a `/generatorConfigSetup` API call to the generator-client, this process involves providing JSON data as specified below. ``` { "generator_config": [ { "address": "0xbCa21b37A139723F5546b5951f00B42e8E9a7D85", "data": "Some data", "supported_markets": [ "0" ] } ], "runtime_config": { "ws_url": "wss://arb-sepolia.g.alchemy.com/v2/***************", "http_url": "https://arb-sepolia.g.alchemy.com/v2/***********", "private_key": *******************, "start_block": 10970349, "chain_id": 421614, "payment_token": "0x01d84D33CC8636F83d2bb771e184cE57d8356863", "staking_token": "0xdb69299dDE4A00c99b885D9f8748B2AeD1Fe4Ed4", "attestation_verifier": "0x3D116255C2b06D7672a9512958d9a3FFD7Aea50c", "entity_registry": "0xFa1004E359fd8Cb76a7D5F32b954eF5020Ea033c", "proof_market_place": "0x27FDcb086Cdb0bCFa40638376CD3CbF5B8c69197", "generator_registry": "0x008d842cA209690D9Da5431a94e193A7B93aC105", "ivs_url":"http://49.215.157.47:3030/checkInput" "markets":{ "0":{ "port":"6000", "ivs_url":"******************" } } } } ``` In the above JSON, the markets field helps the listener to map the ASK request to a particular generator. For example if the ASK has a market ID of 0, the inputs of the ASK will be redirected to the generator running on port 6000. 3. After setting up the config file, the user has to register the generator, stake tokens, join the market and update their ECIES key by making smart contract calls using the Kalypso SDK. 4. Once the configuration setup and registration are done, the user can start the generator using the generator client's [`/startProgram`](https://github.com/marlinprotocol/generator-client/tree/modular-generator?tab=readme-ov-file#end-point-startprogram) API. ## Proof generation flow ![flow](https://hackmd.io/_uploads/ry1LLM9c6.png) ## Benchmarks ### amd64 instance Instance type : c6a.2xlarge Enclave configuration : 4cpu, 8GB memory **Proof generation time** : 7.949997439s **Enclave logs** ``` [2024-02-22T07:29:46Z INFO kalypso_listener] Need to generate proof for ASK ID : 24 [2024-02-22T07:29:47Z INFO kalypso_listener::listener] Finding the logs from block 16440453, for ask: 24 [2024-02-22T07:29:47Z INFO kalypso_listener::listener] Logs found: 1 [2024-02-22T07:29:47Z INFO kalypso_listener::listener] Secret input found [2024-02-22T07:29:47Z INFO kalypso_listener::listener] Took 326 ms for fetching the secret inputs for the ask [2024-02-22T07:29:47Z INFO kalypso_listener::listener] Forwarding inputs for market ID : "0" to the generator running on port "6000" [2024-02-22T07:29:47Z INFO kalypso_listener::listener] endpoint : http://0.0.0.0:6000/api/generateProof [2024-02-22T07:29:47Z INFO zkbob_generator::zkbob_generator] Proof generation started... [2024-02-22T07:29:55Z INFO zkbob_generator::zkbob_generator] Proof generation time: 7.930536941s [2024-02-22T07:29:55Z INFO zkbob_generator::zkbob_generator] Proof verification status : true [2024-02-22T07:29:55Z INFO zkbob_generator::zkbob_generator] proof: Bytes(0x0799911a549fcd574256a620106a758bfd70e31afcc1953ed105becf5becaa0905ed29faa6abb4b08e3f2df731a5a14419d38e7d6b05b50f717f559ef6bee1360de03f066b6d4b698154b3d4db0d94691083df0fd38d796d1f4a1b3b9538dd622e3c4e31ab9e9019e58724451cb779937decd3073e2232199186814d89407fde0a3d728786644be4dd14015d72ba056bf2fc5ee7058d6ae907552b41d38dbd581a81b33f7b776bdc25c9b351816f646c34c4c602f9d11f4f8abd065c6d4568290d379d3cada3c77192f62cf44b9c2456233fe82bfbe79b3bd28682a2312f9195071c829a2532c47deb7ba6ebf2fae2481b7f3cf6a0418ac0bc53b0676c8c10a5) [2024-02-22T07:29:55Z INFO kalypso_listener::listener] "Proof generated" [2024-02-22T07:30:01Z INFO kalypso_listener] Submitted proof for New ask with id : 24 via transaction 0x7a71230434986aa32de2d245db55edb47f21384ae6bf8fda0edf9aaf576da4ec ``` <hr/> ### arm64 instance Instance type : c6g.xlarge Enclave configuration : 2cpu, 5GB memory **Proof generation time** : 22.36704265s **Enclave logs** ``` [2024-02-22T08:30:29Z INFO kalypso_listener] Need to generate proof for ASK ID : 25 [2024-02-22T08:30:29Z INFO kalypso_listener::listener] Finding the logs from block 16440453, for ask: 25 [2024-02-22T08:30:29Z INFO kalypso_listener::listener] Logs found: 1 [2024-02-22T08:30:29Z INFO kalypso_listener::listener] Secret input found [2024-02-22T08:30:29Z INFO kalypso_listener::listener] Took 335 ms for fetching the secret inputs for the ask [2024-02-22T08:30:29Z INFO kalypso_listener::listener] Forwarding inputs for market ID : "0" to the generator running on port "6000" [2024-02-22T08:30:29Z INFO kalypso_listener::listener] endpoint : http://0.0.0.0:6000/api/generateProof [2024-02-22T08:30:30Z INFO zkbob_generator::zkbob_generator] Proof generation started... [2024-02-22T08:30:52Z INFO zkbob_generator::zkbob_generator] Proof generation time: 22.36704265s [2024-02-22T08:30:52Z INFO zkbob_generator::zkbob_generator] Proof verification status : true [2024-02-22T08:30:52Z INFO zkbob_generator::zkbob_generator] proof: Bytes(0x195e0c25b5df1e4e525a22b9e90cdeb0bdf5e9bd977cfcbcadae0f0c958143162b7fc09a12b34f54e5c25178c26907cf4a57033dd532a7a894c6c724134646a02c3f7d61d3eee76a18de4dcb1ead3a8857379fcdac99ba0fa4e03f957ef5b08b01ff873b3c3f8769d73e527c5a66045a17ff618ba1e1230113304aede2a0f2d911b0558f8f984b93aeeae2ebd0e4943dee223eab06d43da2af7f8dcd94c918cc10ead999ac1233923d69222356206e2dc4ea3cdabd7e645ebee5f029ba3165c4211408f1fa20ff39a16c9f3106b561dfae6e9e4f6c488143b565daf6d79ddbfe0c4dfb8d778cd45df59d8fde5f41d58045a0614f13a64d1d2070fff0822d426a) [2024-02-22T08:30:52Z INFO kalypso_listener::listener] "Proof generated" [2024-02-22T08:30:57Z INFO kalypso_listener] Submitted proof for New ask with id : 25 via transaction 0xed59ec3d18fb84460a59ba087fe09c1d3dfef65c24afd7cb9c24a59628d72653 ``` <hr/> Instance type : c7g.large Enclave configuration : 1cpu, 2.5GB memory **Proof generation time** : 9.507246227s **Enclave logs** ``` [2024-02-22T09:48:17Z INFO kalypso_listener] Need to generate proof for ASK ID : 28 [2024-02-22T09:48:17Z INFO kalypso_listener::listener] Finding the logs from block 16440453, for ask: 28 [2024-02-22T09:48:18Z INFO kalypso_listener::listener] Logs found: 1 [2024-02-22T09:48:18Z INFO kalypso_listener::listener] Secret input found [2024-02-22T09:48:18Z INFO kalypso_listener::listener] Took 414 ms for fetching the secret inputs for the ask [2024-02-22T09:48:18Z INFO kalypso_listener::listener] Forwarding inputs for market ID : "0" to the generator running on port "6000" [2024-02-22T09:48:18Z INFO kalypso_listener::listener] endpoint : http://0.0.0.0:6000/api/generateProof [2024-02-22T09:48:18Z INFO zkbob_generator::zkbob_generator] Proof generation started... [2024-02-22T09:48:27Z INFO zkbob_generator::zkbob_generator] Proof generation time: 9.507246227s [2024-02-22T09:48:27Z INFO zkbob_generator::zkbob_generator] Proof verification status : true [2024-02-22T09:48:27Z INFO zkbob_generator::zkbob_generator] proof: Bytes(0x16c4ad0d7fcf794be72133e2fe50bc54de90cdfe3fca4005c20bbed9472719d31733f81632f858d76edd5029e7058b50bfd6970959de5ef7273895c4a9fa09b518e3d4e3e26b556feb7046865a128b6e45971d823f228c1704d12c361b10e13e29bae7e11c99dc2cc1065138d57aa5e5bac2900a848e95c74f053596c4ffc40a0f4a45ac8f30f812d4b5c58c785e1df7ab7c0c3dfffcca9b9b414aa2c7b5af2d02da623ef1d7d74bd2c6195ee8c56a858f2768627728f1a2d26aa06ee602fa0914dcc4efaeff9b6beb25933fe20f7bcaa7098d83009329b820694b2fb01e504f14f3d383ca5896b6333f249312b4f7686c4105b9a93c6e96bd2a75776febe571) [2024-02-22T09:48:27Z INFO kalypso_listener::listener] "Proof generated" [2024-02-22T09:48:35Z INFO kalypso_listener] Submitted proof for New ask with id : 28 via transaction 0x9d05b22e31af3171edd9409ef4ad1c470905408f733aa7edaf9193a294a08bec ``` ## Github repos: listener : https://github.com/marlinprotocol/kalypso-listener zkbob generator : https://github.com/marlinprotocol/zkbob-generator generator client : https://github.com/marlinprotocol/generator-client enclave setup : https://github.com/marlinprotocol/kalypso-enclave-setups/tree/master/amd64-modular-generator