owned this note
owned this note
Published
Linked with GitHub
---
robots: noindex, nofollow
---
# Zcash Wallet proposal
## Title
Zcash Extensible Wallet Interchange Format (ZeWiF)
## Applicant Name
Christopher Allen
## Pitch: A one-liner elevator pitch version of your proposal(required)
To support the deprecation of zcashd, we will create an extensible wallet interchange format for Zcash that supports historic wallet.dat, such as that found in zcashd, and leaves rooms for expansions and improvements in the future.
## TOTAL REQUESTED
$63,600
## DETAILS
### Applicant Background
Christopher Allen is the coauthor of the TLS and DID standards and is currently working with the IETF to advance dCBOR and the Envelope data format for standardization. Recognized as a “Top 100 Influencer in Identity,” he was Principal Architect at Blockstream, Vice President at Blackphone, and CTO of Certicom. Today, he is the Executive Director of Blockchain Commons.
Blockchain Commons has been working for over five years on bringing together wallet developers across the blockchain ecosystem to create secure & interoperable systems that protect the human dignity of their participants. This project will allow us to extend that expertise to the Zcash ecosystem.
Some of Blockchain Commons' major achievements for wallet design include the introduction of an interoperable Animated QR specification (https://developer.blockchaincommons.com/animated-qrs/), formulation of the Lifehash visual hash specification (https://developer.blockchaincommons.com/lifehash/), creation of the security-tested Sharded Secret Key Reconstruction library (https://github.com/BlockchainCommons/bc-sskr), and development of the dCBOR (https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/) and Envelope structured data formats (https://datatracker.ietf.org/doc/draft-mcnally-envelope/) for the IETF. More information on Blockchain Commons' work for wallet design can be found in the 2023 Yearly Overview (https://www.blockchaincommons.com/quarterlies/Yearly-2023/) as well as overviews from previous years (https://www.blockchaincommons.com/tags/#yearly-overview).
### Description of Problem or Opportunity
The Zcashd reference client is being deprecated, and this means that funds must be exported from the client and converted to other wallets. The fact that there are now at least four popular wallets, each with their own formats for metadata about funds and transactions makes this transition challenging as does the number of Zcash address formats (transparent, sprout, sapling, and orchard) and the proliferation of key generation formats (master keys, seeds for HD keys, and BIP-39 compatible seeds for HD keys).
However, this also offers a major opportunity: we can create an interoperable wallet interchange format that not only serves the main purpose of exporting data from zcashd in a standardized manner, but that also supports the interoperability of the whole Zcash wallet ecosystem going forward, ensuring that users always have their choice of wallet and are never locked into a single provider out of fear of losing past transactions, addresses, seeds, or keys.
This helps to preserve the fairness and openness that are among Zcash's core values.
### Proposed Solution
Blockchain Commons will: collect and survey existing wallets (with a focus on zcashd); design an interoperable and extensible wallet interchange format that can preserve and secure data from zcashd and other wallets; and create Rust crates that import data into and export data from that interchange format.
The primary users of this solution will be current and future wallet developers. Zingo Labs has already agreed to work with us and we've also reached out to Andrew Arnott and other members of the Zcash ecosystem. We expect to reach out to all of the other major wallet developers, including ECC (Zashi) and YWallet.
As wallets adopt the extensible wallet interchange format, wallets customers will become users too, and they should see big wins in the ability to make their own choices on which wallets to use.
### Solution Format
Blockchain Commons has this project laid out in 3 steps:
1. WALLET SURVEY. Examine current wallets to enumerate all stored information on seeds, keys, addresses, and transactions including labels, notes, unmined transactions, and other wallet-specific data, with highlighting of information that is not legacy and that is stored by at least two different wallets, and therefore the highest priority for inclusion in an extensible wallet format. The goal here is to be thorough in recording all off-chain data stored in wallets (and perhaps even on-chain data that is replicated in wallets).
* Deliverable #1.1: One or more meetings for wallet developers (Month 1).
* Deliverable #1.2: Final report on all surveyed data, including highlight of data intended to be in a extensible wallet interchange format (Month 1).
2. WALLET INTERCHANGE SPECIFICATION DESIGN. Create a format that will allow the export of zcashd data as well as data from other major Zcash wallets and future expansions.
* Deliverable #2.1: Interchange design document, describing a format focused on well-known data (used for two or more wallets). The goal of this document is to provide what's needed to export data from popular wallets and for partners such as Zingo Labs to import any exported data (Month 2).
* Deliverable #2.2: A developer's how-to document describing use of Envelope attachments for data not included in basic interchange format (Month 2).
3. RUST IMPORT & EXPORT LIBRARIES. Release an open-source reference importer and exporter Rust library for the new extensible wallet interchange format.
* Deliverable #3.1: A Rust crate to import a wallet into an intermediate in-memory representation, using API inputs (Month 3).
* Deliverable #3.2: A second Rust crate to convert that intermediate representation to the serialized Gordian Envelope format as an export. By dividing this into two crates, we give developers the ability to convert into other serialized formats if they prefer (Month 3).
* Deliverable #3.3: A third, small crate and/or CLI and/or integration test suite to import a specific wallet format as a reference and test case (Month 3).
* Deliverable #3.4: A best practices document on importing & exporting data.
The extensible interchange format and the Rust crates together comprise the technical heart of this project.
The two best practices documents are meant to supplement that.
The phase 2 document, on using Envelope attachments, will lay out which data was incorporated into the core interchange format, which was delegated to attachments, and the general reasons for doing so (as support for future extensions). This will include discussions on legacy vs. modern formats and how they're supported differently to ensure the interchange format isn't saddled with technical debt from the start. The document will also provide guidance on attachment formatting and usage as well as how to access an archival copy of the original wallet data.
The phase 3 document then talks about best practices for importing and exporting data. This will discuss ways that legacy data can be imported or converted (primarily through sweeps from legacy keys) and also what an importer should do if it is unable or unwilling to import all data.
## Technical Approach
### Technical Approach: Dive into the how of your project. Describe your approaches, components, workflows, methodology, etc. Bullet points and diagrams are appreciated!(required)
This project will take advantage of the extant Gordian Envelope technology, which is a data storage and exchange format intended for sensitive and confidential data, exactly like this. Gordian Envelope (see https://developer.blockchaincommons.com/envelope/) is privacy-focused and supports encryption and elision without destroying signatures. It can even be used over untrusted or unreliable transports, all of which should be to the benefit to wallet interchange (though the best benefits might await a later stage than the more pragmatic need to get wallets input and output, which is the heart of this proposal).
With those technologies the approach should be fairly simple:
* Survey existing wallets and collected wallet.dat files & talk with wallet designers
* Spreadsheet all possible data elements
* Highlight all repeated data (used by at least two wallets)
* Register CBOR tags for highlighted data
* Write specification for Envelope/CBOR-based extensible wallet interchange format
* Write how to incorporate other data into Envelopes as "Envelope Attachments"
* Write Rust crate that supports import of data into intermediate in-memory format.
* Write Rust crate that supports export of in-memory format to serialized Gordian Envelope format.
* Write mini-crate and/or CLI and/or integration test to read a specific wallet format and call other Rust crates to convert it into Gordian Envelope format, as test suite and reference app.
We expect the extensible interchange format to encode data in three ways:
1. Widely used data that we expect to continue to be used well into the future will be encoded to explicit specifications with CBOR tagging.
2. Less common data or data types that have been deprecated will be encoded as "attachments", which are essentially "blobs of data" with the format referenced in conjunction with a specific vendor. This allows preservation of data without creating technical debt for the data format for little-used or older content.
3. The entirety of the original wallet file will also be preserved, probably in a compressed, encrypted form with metadata revealing where it came from. This allows its review in the future if anything is determined to be missing.
Because Gordian Envelope supports encryption, data that is stored (such as Zcash keys, addresses, seeds, and of course the archived data) can be encrypted with a symmetric key, leaving it protected on disk or in transport. Because of the standardization of this encryption method, this data remains fully interoperable. An exporting wallet could even sign the data and that signature would remain valid after the encryption, if proof was desired of the origin and validity of the seeds, keys, and other data. This is all an crucial improvement over the use of older formats such as JSON, which do not natively support encryption, and so do not offer it in a standardized way.
Other than the test suite, this project is focused on the creation of the extensible interchange format and the development of libraries to read to and from the format. It does not include CLI apps to do that actual conversion (except for the test suite), nor sweep apps (which we think will be necessary for older addresses not based on BIP-39 seeds). We plan for the people most familiar with individual wallet.dat formats to do that final integration work.
Future Possibilities
This proposal does not include long-term support of the new extensible interchange format. We think that upgrades, bug fixes, improvements, training workshops, and more comprehensive documentation on the format might all be desirable. We're happy to work with wallet companies on their individual needs or to write a future proposal for longer term support if needed.
This proposal also does not include best practices or code support to make it *easier* to manage encryption of content during storage and transmission. (We think that a password-to-encryption-key method is likely the optimal way to manage easy encryption with less concern of loss, but our goal here is to focus on creating a forward-looking interchange format that can quickly support the zcashd deprecation.) This is another place where we are happy to write a new proposal in the future to improve this support as usage of the format matures.
We also find value in reference applications that can help designers and developers to test out formats, to ensure compatibility, and to see best practice. We've written an iOS app called Gordian Seed Tool and a few CLI apps, including seedtool and keytool, that offer such support focused on Bitcoin. We'd like to bring these over to Zcash, but such a project could be better specced out once the extensible interchange format is finalized, so that it can be fully integrated.
We hope to advance some of these possibilities in the future.
### Collaboration and Upstream Dependencies: Describe any dependencies or collaborations with other individuals, teams, or organizations that are required to successfully integrate, maintain, or upstream your changes.(required)
We have already begun work with Zingo Labs and with other principals working in the Zcash ecosystem. More collaboration will be required to achieve the best success. We feel that it's especially important to get buy-in from ECC because of their central position in maintaining librustzcash and zcashd.
However, we believe we're on the path to success here, both because we've already begun these discussions and because we've worked with a variety of wallet developers before in the larger blockchain ecosystem, leading to the successful deployment of formats such as Animated QRs, Uniform Resources, Sharded Secret Key Reconstruction, and Envelope itself.
The other major dependency is making sure that we can properly support the the zcashd deprecation plan. The current goal is for zcashd to close out in April 2025, which is obviously very near. To support this, we'd like to be able to have the format fully available for use and testing by the start of April. To meet that deadline, our three months of work would need to be January, February, and March. An end-of-the-year decision on this project would allow us to fulfill that timeline. Otherwise, things will get pushed back, which would endanger the April deprecation date.
### Execution risks: What obstacles do you expect? What is most likely to go wrong? Which unknown factors could jeopardize success? Who would have to incorporate your work in order for it to be usable?(required)
The biggest challenges will be adoption, and adoption that's rapid enough to meet those deadlines. We would like all of the major wallet developers to create true interoperability by incorporating the extensible wallet interchange format in advance of the zcashd deprecation date, but we're well aware that might not be sufficient time depending on each team's priorities.
The other major issue is the fact that older wallets, including zcashd, may have keys not based on seeds or keys based on seeds that were not originally BIP-39 compatible. For many modern wallets, this will require either an adjustment to how they store keys or else a sweep function to import master-key-based funds. We've built a similar sweeptool-cli for Bitcoin (https://github.com/BlockchainCommons/sweeptool-cli) and can highlight it as a reference. We'll also discuss in our best practices ways to accomodate these older keys that reduce Zcash's technical debt without losing funds.
The biggest unknown threat is not knowing what data might be in ancient wallets. We're hoping things are pretty clean because the Zcash ecosystem has had just more than eight years to really create differentiation within their wallets, but it's still an unknown possibility.
### Unintended Consequences: What are the negative ramifications if your project is successful? Consider usability, stability, privacy, integrity, availability, decentralization, interoperability, maintainability, technical debt, requisite education, etc.(required)
Any standardization implicitly creates limits. By saying how things should be done, you might be limiting the innovation of doing new things in new ways. We'll do our best to prevent this through our planned survey (which should catch all the major ways things are being done currently) and through our best practices for use of Envelope attachments (which will describe how to incorporate data other than that which is explictly included in the wallet interchange standard).
### Evaluation plan: What metrics for success will you share with the community once you’re done? In addition to quantitative metrics, what qualitative metrics will you commit to report?(required)
Obviously, we can report out the specification for the wallet extenstible wallet format itself.
Beyond that, our major metric for success will be how many wallets have incorporated the format. We'll report that out a month or two after zcashd is deprecated and at the one year mark, by which time we expect everyone who's planned to incorporate the new format has.
## Budget
Hardware/Software total budget:(required)
$
USD
Please write "N/A" if not applicable.
Please provide justification for the total hardware/software budget: (required)
N/A
Please write "N/A" if not applicable.
Services total budget (cloud, hosting, etc.):(required)
$
USD
Please write "N/A" if not applicable.
Please provide justification for the total services budget: (required)
N/A
Please write "N/A" if not applicable.
Compensation total budget:(required)
$63,600
USD
Please write "N/A" if not applicable.
> Please provide justification for the total compensation budget: (required)
This budget is based on rates for Blockchain Commons' contractors: $125/hr for our technical writer, $150/hr for our lead engineer & researcher.
It breaks down as follows:
Phase 1:
* Research into Zcash address derivations & wallet file formats as well as review of current Zcash-related Rust crates (3 research days: $3,600)
* Survey of Zcash wallets (6 writing days: $6,000)
Phase 2:
* Interchange specification (15 research days: $18,000)
* Attachment best practices (3 writing days: $3,000)
Phase 3:
* Wallet importer & exporter library (25 engineering days: $30,000)
* Importing & Exporting best practices (3 writing days: $3,000)
> Schedule and Milestones:
## Milestone 1
1/31/25
$9,600
### Deliverable 1.1
Survey of Zcash Wallets
## Milestone 2
2/28/25
$21,000
### Deliverable 2.1
Interchange specification
### Deliverable 2.2
Best practices for incorporating legacy or single-use wallet data as Envelope attachments
## Milestone 3
3/31/25
$33,000
### Deliverable 3.1
Import Rust crate; Gordian Envelope Export Rust crate; test suite (or CLI or crate)
### Deliverable 3.2
Best practices doc for importing & exporting
## How was the project timeline determined? (required)
Consultation with engineers and writers on project. In order to meet the deprecation date for zcashd, it assumes a decision on the grant by 12/31/24, which we're aware is tight. Pushing back the decision would push back the milestones similarly.
## How did you learn about Zcash Community Grants?(required)
Long familiarity with the blockchain ecosystem, and discussion with some Zcash ecosystem principals.