owned this note
owned this note
# Migrations RFC: v2
###### tags: `update` `migrate` `URI`
## Context
- Migrations: v1 https://hackmd.io/uuzBdsOKRRqS1m2Q4wyeCg?both
- Go proto update: https://medium.com/holochain/holochain-progress-update-migrations-80ef5290fb67
## Light touch spec
- Decomplect "cleanup/setup" and "update discovery" and "agency transfer"
- Keep core functionality to a minimum and encourage community driven design/governance patterns in zome/client code
- Historical gets are a phase 2+ concern
- Not dealing with holochain core changes yet
### Zome API functions
#### Relay
Attempts a relay (see below).
Triggers relay validation.
Commits relay system entry on successfull validation.
Arguments:
- `Entry::Relay`
### Validation Callbacks
#### Accept THE official DNA update
Validates that a DNA has been accepted per the built-in validation rules for software updates.
The built-in validation rules are implemented as a callback in the zome.
There can be only one.
Hence THE official update.
??Watch this space for singleton CRDT flows??
Arguments:
- `Entry::DNAUpdate`
Example validation logic:
- Signed by main developer's key
- M of N signatures of dev committee
- Signed decision proof from bridged "voting" app
#### Validate Relay
Validates that a relay can be committed pointing to _either_ the official DNA or a agent-provided relay.
Returns true if all setup/cleanup (e.g. accounts are tallied and closed) is valid and so the relay can be finalised.
Relay can be incoming or outgoing. zome validation must cover both.
Arguments:
- `Entry::Relay`
- `Entry::DNAUpdate`
- Local source chain
Example validation logic:
- Accounting is opened/closed correctly
- One relay per chain
- Official updates only
### Data structures
#### Official DNA Update
System entry variant `Entry::DNAUpdate`.
Represents a signed official update.
Various governance models can be implemented with the validation callback (above).
As the official DNA update is a system entry we should implement network behaviour that treats the DNAUpdate as 100% replicated. (not MVP tho)
Everyone needs the data eventually, might as well push it to them!
Even though `Entry::DNAUpdate` is a system entry variant, it DOES have a callback for validation (i.e. exposing system entry to the conscious, see above).
```rust
type Proofs = Signatures
type DNAAddress = Address
enum Entry {
// ... everything else ..
DNAUpdate(Proofs, DNAAddress)
}
```
#### Relay
System entry variant `Entry::Relay`.
```rust
enum RelayDirection {
Incoming,
Outgoing,
}
struct DNAAddress(Address)
enum Entry {
// ... everything else here ...
Relay (DNAAddress, RelayDirection),
}
```
This is missing all the extra data like value, sources, etc. from previous iterations because (as above) "account close" style entries should be app entries + app validation logic.
A relay is only valid if all cleanup/setup tasks are already complete and committed successfully to the local chain.
Even though `Entry::Relay` is a system entry variant, it DOES have a callback for validation (i.e. exposing system entry to the conscious, see above).
#### Conventions:
**Closing sequence:**
1. closing entry (summary of state when applicable)
2. capabilities token for app bridging
3. relay entry
**Opening sequence:**
1. relay entry
2. capabilities token for app bridging
3. opening entry (summary of transferred state which must match closing entry)
### Broad comments re: previous/updated thinking:
- Branching histories are probably too far future to build towards now
- WASM performance/resourcing is just too heavy RN to support many parallel zomes for full historical GETs
- Can always be simulated client side if needed
- Don't have clear use-cases from real world usage to use for planning
- Merging histories are definitely in the too hard basket for now
- At least branching histories have a linear history from each branch, merged histories don't even have that
- Decomplect the ideas of references between DNAs and closing/opening accounts
- Closing/opening is an accounting process
- This is an app entry as accounting is always domain-specific
- Should prevent further accounting transactions once a chain is closed
- Referencing/migrating is a social/technical process
- This is a system entry as moving through DNAs is a holochain managed process
- Does not prevent accounting transactions itself but may not be valid without a prior valid account closing entry (see below)
- Need validation callbacks for the DNA hashes themselves
- E.g. Hit this URL from the developer and get the hash back as a "checksum"
- Allows for pre-migration governance
- Want a better name for Migrate entries to be clearer about what is happening
- Need validation callback against the system migrate entries + local source chain + DNA hash
- E.g. don't stick a migration reference in the local chain before closing the account!
- GET should NOT run through all histories _by default_
- _get_ should just be for one zome at a time
- _getHistorical_ can get until we find a match + provide a DNA
- _getArcheological_ can get all matches across all DNAs (probably too slow with current performance characteristics)
- queries (chain-only, no live DHT) could be for all historical data potentially as this just whips through the CAS following migration references
- everything other than the default/single get needs to return the DNA hash alongside the found entries
- High level approach is "light touch" core + community driven patterns
- Wordpress/Drupal/Saas CMS -> Zome
- Zome -> Zome
- Holochain Migrator Agent + Migrator CMS adapter
### Open questions
- Types of branching:
- single branch / closing entry --> permanent move (301)
- for now? permanent move can be achieved with the right callback logic
- multi-branch --> also find me here (302)
- opening entry
- Can a "Migrate From" entry be added anywhere other than genesis?
- e.g. balance transfer
- Singleton nature of official updates
- Encourages community coherence
- Doesn't allow amicable/equal splitting of communities (e.g. always need to pick a "real" version)
- Requires CRDT for global singleton behaviour
- Simplifies UI/UX
- Phishing tradeoffs
- Stronger for official update
- Weaker for potentially legit forks
- How to handle changes to holochain core?
- How much change is "breaking"?
- Zomes are hashed so break immediately on any change but core can arbitrarily chug along underneath?
- Even "non breaking" changes can totally be breaking for someone
- e.g. node version 8.14.0 vs. 8.14.2 breaking in cyprus
- e.g. "edge cases" in constantinople ETH delays due to performance improvements that threw cpu/gas price usage out of alignment leading to weird vulnerabilities
- Sem ver is insufficient for what we are trying to do
- Can developers maintain/publish a list of core commits/hashes/versions that zomes are compatible with?
- What is the process/ramifications for tracking core lifecycle vs. zome lifecycle?
- parallel versions of holochain running?
- relays?
- data migrations?
### Holochain URI / URN / XDI format
- In a multiDNA world, what is the full URI to CAS, because it isn't just the address
- `hc:[<Agent-pubkey>@]<DNA-hash>/<CAS-hash>[?Captoken=09jj0shs8e]` (For context and type information, you may want to reference the header for an entry rather than direct reference to the entry itself.) Technically, this is performing an implicit GET.
- Also we should think about [web DIDs](https://w3c-ccg.github.io/did-spec/#dfn-did-method) for identifiers
- https://tools.ietf.org/html/rfc3986
### Solid inter-operability
> [name=Nicolas Luck] After looking into Solid and all those W3C specs and ontologies, I believe it makes a lot of sense to equate Holochain DNAs/DHTs with Solid's pods and use their link annotation spec.
>
> If we make the pod/DHT URLs match real DNS names on Holo, we could have (gateway-public) Holochain data inter-operable with Solid apps and other web apps / front-ends.
[Solid](https://solid.inrupt.com) is mainly about having standards and ontology for resources that live on their own central servers, called PODs. At the core of how inter-operability works with Solid are [links](https://solid.inrupt.com/docs/intro-to-linked-data).
Example from their docs:
```url
<https://mypod.solid/comments/36756>
<http://www.w3.org/ns/oa#hasTarget>
<https://yourpod.solid/photos/beach>.
```
Imagine we would build our equivalent to a chain-explorer into Holo
that could be activated for Holo hosted apps,
such that entries could be retrieved with a link that includes
the DNA and entry address:
```url
<http://holo.host/<DNA-hash>/entries/<CAS-hash>[?Captoken=...]>
```
We could then use Solid's [Turtle syntax](https://solid.inrupt.com/docs/expressing-ld-with-turtle) or the [Web Annotion syntax for links](https://www.w3.org/TR/annotation-vocab/#linking) to represent links
between DNAs.
```url
<http://holo.host/<DNA-hash>/entries/<CAS-hash>[?Captoken=...]> a oa:Annotation ;
oa:hasTarget <http://holo.host/<DNA-hash>/entries/<CAS-hash>> ;
rdf:type <http://holo.host/<DNA-hash>/link_types/comments>
oa:motivatedBy oa:linking .
```
**Benefit**: Holochain DNAs could link to external Solid data and vice versa.
