# Kalypso ↔️ ZKsync

In the previous [blog](https://hackmd.io/_T4pqRmBTSaWk0_McOBlSA), we explored the intricacies of proof generation within the ZKsync ecosystem. Today, we'll examine an initial design for integrating ZKsync's proof generation with Kalypso.


## The proof generation process

Let's take a high-level look at how the proof generation process works:

1. The master node receives the witness_input.bin file for a particular batch from the gateway.
2. The witness generator picks-up the witness_inputs.bin and starts generating proofs for the first round of the proof generation (Basic circuit).
3. The job details are updated in the database and the witness artifcats are uploaded to the GCS bucket.
4. The witness vector generator in the slave node pick jobs based on the following order:
1) pick the lowest batch
2) within the lowest batch, look at the lowest aggregation level (move up the proof tree)
3) pick the same type of circuit for as long as possible, this maximizes GPU cache reuse.

This is the query that is currently used by the light witness vector generator to fetch the next job :

    UPDATE prover_jobs_fri
                SET
                    status = 'in_progress',
                    attempts = attempts + 1,
                    updated_at = NOW(),
                    processing_started_at = NOW(),
                    picked_by = $3
                WHERE
                    id = (
                        SELECT
                            id
                        FROM
                            prover_jobs_fri
                        WHERE
                            status = 'queued'
                            AND protocol_version = $1
                            AND protocol_version_patch = $2
                            AND aggregation_round != $4
                        ORDER BY
                            l1_batch_number ASC,
                            aggregation_round ASC,
                            circuit_id ASC,
                            id ASC
                        LIMIT
                            1
                        FOR UPDATE
                        SKIP LOCKED
                    )
                RETURNING
                prover_jobs_fri.id,
                prover_jobs_fri.l1_batch_number,
                prover_jobs_fri.circuit_id,
                prover_jobs_fri.aggregation_round,
                prover_jobs_fri.sequence_number,
                prover_jobs_fri.depth,
                prover_jobs_fri.is_node_final_proof

`Note`: The heavy witness vector generator prioritizes node aggregation rounds first and processes other rounds only if no node jobs are present. The query remains largely the same, except for selecting the aggregation round.

5. The slave node starts generating proofs and forwarding the generated proofs to the verifier node and verifier nodes run the boojum verifier and generates the attestation signature.
6. After the attestation signature is generated, the verifier node uploads the proof to the GCS bucket and updates the master node DB table `prover_jobs_fri` with the attesation signature and the pod name(generator address) against the job completed.
7. The same steps are followed for all the subsequent rounds and at the end the compressor on the master node generates the final proof and uploads the final proof to the GCS bucket and the proof generation for the particular batch is marked as complete in the database.
8. The final generated proof is then forwarded to the gateway and subsequently submitted to the network which marks the proof generation flow as completed.

## Challenges

**Access Control:** Restricting unauthorized modifications to the database and object store is crucial. Slave nodes should not have permission to mark jobs as completed or upload proofs directly, as this could compromise the integrity of the proof generation process. A robust access control mechanism is necessary to enforce role-based permissions.

**Compensation Model:** Since tracking all ASKs via smart contracts is infeasible due to the large number of jobs (approximately 17,000 per batch), a scalable and efficient method to compensate slave nodes for their contributions is required. Ensuring fair and transparent compensation is key to maintaining a reliable proof generation network.

**Slashing:** Since jobs are not tracked with ASKs made through smart contract calls, how will the generator be slashed if they fail to generate proofs?

## Possible Solutions

**Role-Based Access Control (RBAC):** Implementing a fine-grained permission model ensures that only verifier nodes have the authority to finalize jobs and upload proofs. By restricting access, we can maintain data integrity and prevent unauthorized modifications.

**Payment System:** The number of proofs generated in a particular round by a specific slave node can be retrieved using SQL queries from the database. At the end of the proof generation process, compensation can be determined based on these metrics.

**New orchestration:** Modify the witness generator and other components to fit the Kalypso model of tracking jobs through ASKs by dividing each round into smaller batches. For example, if there are 17 available slave generators, divide the jobs into smaller batches (1,000 jobs per batch) and track them using ASKs. (Expect this to be toilsome).

## Conclusion

This initial design lays out a framework for integrating Kalypso with ZKSync's proof generation system. While the proposed architecture addresses key challenges around access control, compensation, and orchestration, it will require significant refinement before implementation. The path forward involves balancing decentralization with efficiency, optimizing job distribution, and developing robust security measures.