Try   HackMD

CC-Sector Update Circuit

General Notation

Fq
A prime field element (i.e. an integer
mod q
). Filecoin uses the BLS12-381 scalar field as its prime field.

Fq|{0,1}
A field element whose value is constrained to either
0
or
1
.

Fq[n]
An array containing
n
field elements.

x:Fq
A variable
x
which has type
Fq
.

(x1,x2)
An array containing two values.

0..n
The range of values
(0,1,,n1)
.

arr[m..n]
A slice (subarray) of an array
arr
containing the values
(arr[m],arr[m+1],,arr[n1])
.

y=some_gadget()
Denotes
y
as a being a value allocated in a constraint system by a gadget
some_gadget
.

Circuit Notation and Constants

N={230if sector-size=32 GiB231otherwise if sector-size=64 GiB
The number of nodes per sector.

C=2200
The number of challenges per sector (one proof is generated for per update CC-sector).

c0..N
A Merkle challenge (i.e. the index of a node in a sector of size
N
). There are
C
number of challenges
ci
generated per proof.

ρ:Fq
A random value generated for each challenge
c
.

bitlen(digest)=bitlen(Fq)=log2(q)=255
The bit-length of a Poseidon digest. Our Poseidon hash function is instantiated over a
256-bit field (BLS12-381's scalar field
Fq
), thus targets a security level of 128 bits.
q
is the Poseidon field modulus.

bitlen(c)=log2(N)={30if sector-size is 32GiB31otherwise if sector-size is 64GiB
The bit-length of a challenge
c
. Each challenge
c
is the index of a node in a sector
c0..N
.

bitlen(digest)bitlen(c)=255log2(N)=8
The number of challenges generated using a single Poseidon digest, i.e. the
jth
digest is used to generate the challenges
(cj8,cj8+1,,cj8+7)
.

C/8=275
The number of Poseidon digests required to generate
C
number of challenges, i.e. a single Poseidon digest generates
8
of the required
C=2200
challenges.

\boldsymbolH=(27,28,29,210,211,212)
An array containing the possible values of
H
.

H\boldsymbolH
Specifies the number of bits extracted from a challenge
c
's binary representation
c_bits
and appended onto
CommDNew
to generate the preimage for
c
's random value
ρ
.
H
is chosen from
\boldsymbolH
using the
HSelect
public-input.

HSelect:Fq{20,21,22,23,24,25}
Selects
H\boldsymbolH
via
H=\boldsymbolH[log2(HSelect)]
. Note that each possible value of
HSelect
has either no bits set or exactly one bit set, thus
log2(HSelect)
is a unique integer in
0..6
. Within the circuit we allocate
HSelect
as 6 bits
h_select_bits
and extract the
log2(H)
high bits from each challenge
c
using the linear combination
ck0..6h_select_bits[k]2log2(N)log2(\boldsymbolH[k])
.

CommDNew:Fq
Commitment to the sector's plaintext data after updating; the root of
TreeDNew
.

CommROld,CommRNew:Fq
Commitments to the replicated sector before and after updating;
CommR=poseidon2((CommC,CommRLast))
.

CommRLastOld,CommRLastNew:Fq
Roots of the sector's
TreeR
trees before and after updating.

CommC:Fq
The original/unupdated sector's
CommC
. Note that
CommC
is not changed when updating a CC-sector.

TreeDNewProofc:struct {leaf:Fq, path:Fq[1][log2(N)]}
TreeROldProofc,TreeRNewProofc:struct {leaf:Fq, path:Fq[7][log8(N)]}

Three Merkle proofs (1 bintree, 2 octtree) are generated for each challenge
c
: one in
TreeDNew
, one in
TreeROld
, and one in
TreeRNew
. The
leaf
field is the Merkel challenge
c
's (leaf) label. The field
path
is the Merkle path generated for
c
.

Summary

Generate

C number of challenges using
C/8
Poseidon digests, i.e. one digest generates
8
challenges.
j0..C/8:digestj=poseidon((CommDNew,j))

Each challenge is a

log2(N)-bit unsigned integer, thus split each digest into
8
,
log2(N)
-bit, partitions interpreting each as the little-endian bits of a challenge
c
.
i0..8:c8j+i=digestj[log2(N)i..log2(N)(i+1)] as Fq

Isolate the most-significant

H bits of
c
denoted
c_high
where
H
is chosen from
\boldsymbolH
using the public input bit-array
HSelect
. Calculate each challenge's random value
ρ
using
c_high
.
ρc=poseidon((poseideon(CommDNew,CommROld),c_high))

Verify each challenge

c's Merkle proofs in
TreeDNew
,
TreeROld
, and
TreeRNew
(i.e. the trees built over the new plaintext data, old replica, and new replica).

Verify each

c's new replica label
Rc
using
c
's old replica label
Rc
, new plaintext label
Dc
, and
ρc
.
Rc=Rc+Dcρc

Circuit

Public Inputs:

CommDNew,CommROld,CommRNew,HSelect

Private Inputs:

CommC,CommRLastOld,CommRLastNew,c(c1,,c2200):TreeDNewProofc,TreeROldProofc,TreeRNewProofc

Circuit: SDR CC-Sector Update  
1.10.  comm_d_new=alloc_pub(CommDNewpub)

2.10.  comm_r_old=alloc_pub(CommROldpub)

3.10.  comm_r_new=alloc_pub(CommRNewpub)

4.10.  ϕ:Fq=poseidon_gadget2((comm_d_new,comm_r_old))

5.10.  h_select=alloc_pub(HSelectpub)

6.10.  h_select_bits:Fq|{0,1}[6]=le_bits_gadget(h_select,6)

7.10.  comm_c=alloc(CommCpriv)
8.10.  comm_r_last_old=alloc(CommRLastOldpriv)

9.10.  comm_r_last_new=alloc(CommRLastNewpriv)

TODO: do we need to prove

CommRLastOld using
CommROld
?
10.10.  comm_r_old_calc:Fq=poseidon_gadget2((comm_c,comm_r_last_old))

11.10.  assert_eq(comm_r_old_calc,comm_r_old)

10.10.  comm_r_new_calc:Fq=poseidon_gadget2((comm_c,comm_r_last_new))
11.10.  assert_eq(comm_r_new_calc,comm_r_new)

12.10.  for j0..C/8:
13.10.  j=alloc(j)

14.10.  digest:Fq=poseidon_gadget2((comm_r_new,j))

15.10.  digest_bits:Fq|{0,1}[8log2(N)]=le_bits_gadget(digest,8log2(N))

16.10.  for i0..8:
17.10.  c_bits:Fq|{0,1}[log2(N)]=digest_bits[log2(N)i..log2(N)(i+1)]

18.10.  c_high=lc_gadget(k0..6h_select_bits[k](c_bits(log2(N)log2(\boldsymbolH[k]))))

19.10.  ρ:Fq=poseidon_gadget2((ϕ,c_high))

20.10.  (label_d_new,path_d_new)=alloc_bintree_proof(TreeDNewProofc,priv)
21.10.  (label_r_old,path_r_old)=alloc_octtree_proof(TreeROldProofc,priv)

22.10.  (label_r_new,path_r_new)=alloc_octtree_proof(TreeRNewProofc,priv)

23.10.  comm_d_new_calc=bintree_root_gadget(c_bits,label_d_new,path_d_new)
24.10.  comm_r_last_old_calc=octtree_root_gadget(c_bits,label_r_old,path_r_old)

25.10.  comm_r_last_new_calc=octtree_root_gadget(c_bits,label_r_new,path_r_new)

26.10.  assert_eq(comm_d_new_calc,comm_d_new)
27.10.  assert_eq(comm_r_last_old_calc,comm_r_last_old)

28.10.  assert_eq(comm_r_last_new_calc,comm_r_last_new)

29.10.  rho_label_d_new=mul_gadget(ρ,label_d_new)
30.10.  label_r_new_calc=add_gadget(label_r_old,rho_label_d_new)

31.10.  assert_eq(label_r_new_calc,label_r_new)

Gadgets

alloc(x) allocates
x
in the constraint-system and does not add constraints.

alloc_pub(x) makes one public and one private allocation in the constraint-system and adds one constraint asserting equality between the two.

alloc_bintree_proof/alloc_octtree_proof(leaf,path) allocate a challenge
c
's Merkle leaf label and its Merkle path elements and doe not add constraints to the constraint system.

assert_eq(x,y) adds one constraint asserting the equality
x==y
.

add_gadget(x,y) allocates the sum
x+y
and adds one constraint asserting the correctness of the allocated value.

mul_gadget(x,y) allocates the product
xy
and adds one constraint asserting the correctness of the allocated value.

lc_gadget(iconstiprev_alloc_valuei) makes one allocation for the output of the linear combination ("lc") and adds one constraint asserting that the allocated value is consistent with the sum of (scaled) previously allocated values. Note that each constant
consti
is unallocated.

poseidon_gadget2((x,y)) calculates the arity-2 Poseidon digest of the preimage array
(x,y)
.

le_bits_gadget(x,n) allocates
x
as
n
bits in the constraint system, i.e. makes
n
allocations, add a boolean constraint for each allocated bit, and adds
1
constraint asserting that the allocated bits are the correct binary representation for
x
:
assert_eq(i0..n2ibits[i],x)
.

bintree_root_gadget/octtree_root_gadget(c_bits,leaf,path) calculates the root of a Merkle bintree/octtree given a challenge
c
's Merkle leaf label and Merkle path. Bintrees use arity-2 SHA256 and octtrees use arity-8 Poseidon.

Circuit (Updated)

const P=16
const A=128

const C=86

const Hs=(7,8,9,10,11,12)

const PartitionTreeHeight=log2(P)=4

const ApexTreeHeight=log2(A)=7

config SectorSize{32 GiB,64 GiB}
config k0..P

config HsIndex{0,1,2,3,4,5}

type TreeD=BinTree

type TreeR={OctTreeif SectorSize=32 GiBOctBinTreeif SectorSize=64 GiB

Nodes={230if SectorSize=32 Gib231if SectorSize=64 Gib

input RoldFq[Nodes]
input DnewFq[Nodes]

input RnewFq[Nodes]

ChallengeBits=log2(Nodes)
PartitionBits=log2(P)

RandChallengeBits=ChallengeBitsPartitionBits

HSelect=2HsIndex
H=Hs[HsIndex]=Hs[log2(HSelect)]

TreeROld=TreeR(Rold)
TreeDNew=TreeD(Dnew)

TreeRNew=TreeR(Rnew)

CommROld=TreeROld.root
CommDNew=TreeDNew.root

CommRNew=TreeRNew.root

TreeDHeight=log2(Nodes)

TreeRHeight={log8(Nodes)if SectorSize=32 GiBlog8(Nodes/2)+1if SectorSize=64 GiB

ApexRootsRow=TreeDHeightPartitionTreeHeight
ApexLeafsRow=ApexRootsRowApexTreeHeight

ApexLeafs=TreeD[ApexLeafsRow][kA..(k+1)A]
ApexTree=TreeD(ApexLeafs)

ApexRoots=TreeD[ApexRootsRow]

ApexRoot=ApexRoots[k]=ApexTree.root

PartitionBits=((ki)1ilog2(P))=le_bits(k)
PartitionPath=(TreeDnew[ApexRootsRow+i][(ki)+j]i0..ApexTreeHeight,j={1if is_even(ki)1if is_odd(ki))=TreeDNewProofc[TreeDHeightPartitionTreeHeight..]

PubInputs=(k,CommROld,CommDNew,CommRNew,HSelect)
PrivInputs=(CommC,RootROld,RootRNew,ApexLeafs,ChallengeProofs)

Circuit: SDR Empty Sector Update  
1.10.  k=alloc_pub(kpub)

2.10.  comm_r_old=alloc_pub(CommROldpub)

3.10.  comm_d_new=alloc_pub(CommDNewpub)

4.10.  comm_r_new=alloc_pub(CommRNewpub)

5.10.  h_select=alloc_pub(HSelectpub)

6.10.  k_bits=le_bits_gadget(k,255)[..log2(P)]
7.10.  h_select_bits=le_bits_gadget(h_select,255)[..6]

8.10.  phi=poseidon_gadget((comm_d_new,comm_r_old))

9.10.  comm_c=alloc(CommCpriv)
10.10.  root_r_old=alloc(RootROldpriv)

11.10.  root_r_new=alloc(RootRNewpriv)

12.10.  apex_leafs:Fq[A]=(alloc(apex_leaf)apex_leafApexLeafspriv)

13.10.  partition_path:Fq[1][PartitionTreeHeight]=((alloc(el))elPartitionPathpriv)

14.10.  comm_r_old_calc=poseidon_gadget((comm_c,root_r_old))
15.10.  assert_eq(comm_r_old_calc,comm_r_old)

16.10.  comm_r_new_calc=poseidon_gadget((comm_c,root_r_new))

17.10.  assert_eq(comm_r_new_calc,comm_r_new)

18.10.  apex_tree:(Fq[A],Fq[A/2],,Fq[1])=bin_tree_gadget(apex_leafs)
19.10.  apex_root=apex_tree[ApexTreeHeight][0]

20.10.  comm_d_new_calc=bintree_root_gadget(k_bits,apex_root,partition_path)

21.10.  assert_eq(comm_d_new_calc,comm_d_new)

22.10.  rand_bits:Fq|{0,1}[RandChallengeBits][C]=gen_challenge_bits_gadget(comm_r_new,k)
23.10.  for c_rand_bitsrand_bits:

24.10.  c_high_and_zeros:Fq[6]=()

25.10.  for h,h_select_bitzip(Hs,h_select_bits):

26.10.  c_high_bits=c_rand_bits[RandChallengeBitsh..]

27.10.  c_high=lc_gadget(i0..h2ic_high_bits[i])

28.10.  c_high_or_zero=mul_gadget(h_select_bit,c_high)

29.10.  c_high_and_zeros.push(c_high_or_zero)

30.10.  c_high_selected=lc_gadget(i0..6c_high_and_zeros[i])

31.10.  rho=poseidon_gadget((phi,c_high_selected))