# Zcash Wallet Interop Project

(random notes from signal channel https://signal.group/#CjQKINBnm98HKYowL-et3NIixcb2CPPDYLy7LmiWwVPDlVCyEhBN9bcaR9zSOot_Gfga44ui )

**The main goal as it was described in the last deprecation would be to create an extensible format that can accommodate today’s wallet.dat from zcashd and leave room for continuous improvements onward.**

## Proposal in progress from Blockchain Commons

@Christopher Allen: Shannon is working on a ZF proposal, full draft to be shared here next Monday or Tuesday (we both are on vacation starting Wednesday).

Here is the outline that Shannon is working from for the ZF proposal. We could also do it if there is sufficiently Github Sponsors patronage from the Zcash community: https://github.com/sponsors/blockchaincommons

a) update Gordian Seed Tool iOS with preference option to default to Zcash (currently has option to set to defaults to Bitcoin, Ethereum, and Tezos, or any derivation but no addresses.) The reason why is this will allow reference of gordian envelope based seed recovery (both self-sovereign and sskr (shared secret key reconstruction). We'll need to implement zip32 in swift (or maybe wrap rust crate - TBD) to demonstrate creation of arbitrary zcash addresses. The main thing missing is that there isn't a standard for the equivalent of bitcoin descriptors in the zcash community (so to be deferred to later). The current version of the reference app is at https://apps.apple.com/us/app/gordian-seed-tool/id1545088229

b) Survey of metada requirements of zcashd's wallet.dat and other wallets, first pass on open issues (like how to handle multisig without bitcoin-style descriptors), and some meetings with wallet implimentors.

c) once we have a good survey, and decide what items are only supported by one wallet (which still can be backed up with 'attachments' feature), we'll document and register the cbor tags for all the metadata (or at least 80/20 or 90/10) and create some reference interchange format files.

d) Once we have the examples done and reviewed, create a reference serializer / deserializer in rust that supports them. If funding available, we could do other languages, but it Rust is easiest for us today.

e) Work with others to export zcashd https://github.com/zcash/zcash/issues/6873 data. One key issues is they don't support zip32, only master keys, so any wallet importing a zcashd import will need to do some extra work as there is no seed.

f) From there (not in proposal) will be if the wallet community wants to support other gordian standards, such as encrypting the exported information, secret sharing (SSKR), gordian depos (social key recovery), animated QRs, etc

We have had the greatest success with these standards in the airgap Bitcoin wallet community, with 13+ wallets using one or more of our standards, and Ledger recently released/approved an official SSKR app for Ledger, which enables a large group of people using Ledger today to do seed recovery. Ledger hasn't committed to CSR (Collaborative Seed Recovery = SSKR + metadata recovery + secure transmission over unsecure channesl aka GSTP).

There are some ethereum and tezos wallets that use our L0 standards, but we've not had broad adoption on those communities.

(We also will be continuing our FROST meetings, next will be December 4th (10am PT). We’ll have presentations on how to implement FROST in wallets. Be sure to sign up for our Gordian Developers announcements-only mailing list and/or Signal channel to receive the invites for these meetings https://www.blockchaincommons.com/subscribe/)

We have extensive documentation on our stack of L0 libraries and reference apps at https://developer.blockchaincommons.com and a lot of videos at https://youtube.com/@blockchaincommons

@Pacu: An aspect not to be neglected is that the priority of the Zcash ecosystem is to move away from Zcash and that needs to happen rather quickly. **The main goal as it was described in the last deprecation would be to create an extensible format that can accommodate today’s wallet.dat from zcashd and leave room for continuous improvements onward.** V1 of the format would mostly be dedicated to migrating full node wallets from Zcashd 6.0 to the new thing 1.0 and nothing else

@ChristopherA: We can make focus the initial survey and the v1 extensible format on zcashd's wallet.dat. However, as far as I know no other zcash wallets supports master keys as the root of account (they instead support zip32), just the export is insufficient. Thus there likely needs to be some additional utility for a wallet to commit to the changes needed to support master keys without seeds, so we need to do more than just zcashd's wallet.dat. It doesn't do any good to have an export format unless someone can fully support importing it.

@pacu: Yes that's correct. I'm referring specifically to the overall scope and what is a priority in the short term which will be what ZCG will focus their funding efforts for the time being. I think there's a clear consensus on your expertise on the subject so as long as the proposal aligns with the scope and priorities for the next 5 months you'll be in a good shape for sure.

@zancas: ZingoLabs would like to provide and test a reference implementation. I suppose would could being testing the ingestion of seed-tool exported format immediately.

## Re: HD Master-Key vs Seed Wallets

@ChristopherA: We've run into the problem of master-key-only vs seed-based wallets in the bitcoin community before. The solution we proposed was a dedicated "sweep" app, which would sweep funds from a master-key-only account to a seed-based account of choice. It would transfer all funds on all master-key accounts to a single seed based one, but doesn't do all the other wallet functions (only sweeps).

The sponsor that was interested in supporting this ultimately decided it wasn't worth supporting master-keys after all, so it was never fully developed, but we did do a command-line version of it.

Our cli version is at https://github.com/BlockchainCommons/sweeptool-cli

It leveraged the bdk library for the L1 features. We generally avoid doing L1+ code.

My advice for zcashd deprecation is to intially just do something like the sweeptool-cli.

## Misc.

@ChristopherA: I think the biggest thing that we'll need to puzzle out is exporting the shielded transaction data from wallet.dat. Online discussions say they can be pretty large. I don't see a lot of detail on how that data differs from other wallets, but I presume the cryptographic parts are defined in a ZIP?? somewhere, but likely any non-cryptographic metada is different.

If someone has an spent zcashd wallet.dat that has a variety of transaction and addresses types, that they can contribute as a reference, it would very helpful. wallet.dat is a Berkeley DB file, so we can use Berkeley DB utility db_dump to at least look inside the file.

Another question for a zcashd is related to master seeds. Apparently bitcoind v0.11 (which is what zcashd forked from) did have some experimental support for HD keys, in addition to the pool of keys of pre-0.11. The question would be if zcashd only used the mature pool of keys, or if they cherrypicked some HD support and moved it forward in zcashd.

Hmm, it looks like at some point HD keys were used for shielded transactions rather than a key pool. Wonder if we have to support both {sigh}. I had presumed 9falsely) that zcashd only used the older pool of keys.

## Metadata in `zcashd`'s `wallet.dat` Format

The `wallet.dat` file in Zcash's `zcashd` node software contains a wide variety of metadata necessary to manage transparent and shielded addresses, keys, and transactions. Zcashd's wallet format is derived from Bitcoin Core version 0.11, but with significant extensions to support Zcash's privacy features, particularly shielded transactions using zk-SNARKs.

### 1. Transparent Address Metadata

#### Private Keys
- **Private Keys for Transparent Addresses**: Stores the private keys needed to spend Zcash from transparent addresses (t-addresses).

#### Public Keys and Addresses
- **Public Keys and Bitcoin-style Addresses**: Corresponding public keys for all private keys stored in the wallet.
- **Address Labels**: Labels associated with transparent addresses for easier reference (e.g., "Savings" or "Car Payment").

#### Keypool Metadata
- **Keypool**: A set of pre-generated transparent keys to ensure addresses are always available without delay.
- **Key Creation Timestamps**: Metadata indicating when each key was generated.

### 2. Shielded Address Metadata

#### Shielded Keys
- **Shielded Spending Keys**: Private keys for spending funds from shielded addresses (z-addresses). These are essential for making shielded transactions that use zk-SNARKs for privacy.
- **Full Viewing Keys**: These keys allow users to monitor shielded transactions without spending capability. Useful for auditing or viewing balances.
- **Key Metadata**: Includes details like key creation timestamps and whether the key is from the Sprout or Sapling protocol.

#### Shielded Address Labels
- **Labels for Shielded Addresses**: Labels for shielded addresses, similar to the transparent address labels, for better management and organization.

### 3. Hierarchical Deterministic (HD) Wallet Metadata

#### HD Seed
- **Master Seed for HD Wallets**: Zcash supports HD wallets, allowing deterministic generation of both transparent and shielded keys. The HD seed is stored and is used to derive all subsequent child keys.
- **Derivation Paths**: Paths used to derive transparent, Sprout, and Sapling addresses from the master seed.

### 4. Shielded Transaction Metadata

#### Notes and Commitments
- **Notes**: Shielded transactions in Zcash create "notes" instead of traditional UTXOs. These notes represent encrypted representations of value, managed privately within the wallet.
- **Note Commitments**: Metadata representing commitments that are added to a Merkle tree, enabling shielded transactions.
- **Nullifiers**: Nullifiers are used to indicate that a note has been spent, while ensuring that its origin remains private.

#### Memo Fields
- **Memo Data**: Shielded outputs include optional memo fields (up to 512 bytes) that can be used to add custom information, such as messages or transaction references.

#### Transaction Data
- **JoinSplits (Sprout Transactions)**: Details on Sprout JoinSplits, which allow value to move between transparent and shielded pools. This includes zk-SNARK proofs, shielded inputs and outputs, and transferred values.
- **Sapling Actions**: Metadata related to Sapling shielded transactions, which replaced the JoinSplit model for better efficiency. This includes inputs, outputs, and zk-SNARK proof data.
- **Anchor Points**: Anchor points used in zk-SNARK proofs to verify that notes are part of the valid note commitment tree.

### 5. Transaction and Blockchain Metadata

#### Transaction History
- **Transactions**: Full transaction history for both transparent and shielded transactions, including TxIDs, input and output details, timestamps, and confirmation status.
- **Confirmation Metadata**: Block height, number of confirmations, and other details related to the confirmation state of each transaction.

#### Best Chain Pointer
- **Best Chain Information**: Stores the block that the wallet considers to be the current chain tip. This helps track which blocks have been processed.

### 6. Address Book and Reserve Keys

#### Address Book
- **Stored Addresses**: Information about external Zcash addresses stored in the wallet, including both transparent and shielded addresses.
- **Labels for External Addresses**: Similar to internal addresses, these labels help identify saved external addresses.

#### Reserve Keys
- **Reserve Keypool**: A collection of pre-generated keys reserved for future use, ensuring new addresses are always available as needed.

### 7. Encryption Metadata

#### Encrypted Private Data
- **Encryption of Private Keys**: If the wallet is encrypted, all private keys (including shielded spending keys) are stored in an encrypted form. This includes both transparent and shielded keys.
- **Encryption Scheme**: Enhanced encryption to cover shielded data, requiring a passphrase to access and decrypt sensitive key information.

### 8. Payment Disclosure Metadata

#### Payment Disclosure
- **Payment Disclosure Data**: Metadata for optional payment disclosure features, which allow a user to reveal shielded transaction details to a third party. This can include proof data, transaction IDs, and memo information for compliance or audit purposes.

### 9. Sprout and Sapling Differentiation

#### Address Type Identification
- **Sprout vs. Sapling Addresses**: Metadata indicating whether shielded keys and addresses belong to the older Sprout protocol or the newer, more efficient Sapling protocol. This helps the wallet manage different types of shielded transactions separately.