# Record content indexing

Two needs:
1. Typing of record attributes
2. Annotation of the record attributes type

## Typing of record attributes
1. Convert attributes type from `string` to protobuf `any.Any`
2. Add new "TypeID" field to record
* For _schema records_ this field will be empty but for _content records_ it will contain a CID that references the _schema record_ which contains the typing information for this record's attributes
3. Register _schema record_ that contains in its attributes the FileDescriptorProto set for a _content record's_ attributes
4. Use [DynamicMessage](https://protobuf.dev/reference/java/api-docs/com/google/protobuf/DynamicMessage) type to construct Go type from referenced [FileDescriptorProto](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/descriptor.proto#L61) set to unmarhsal _content record_ attributes into
5. Look into using https://github.com/ipld/go-ipld-prime/tree/master/node/bindnode to IPLDize the Go types constructed above

## Annotating of record attributes
1. Annotation needs to be in separate schema
1. To tell which feilds you want to index against
2. Qualifiers which indicate a field cannot be some set of values (null, zero, etc)
2. Add addition "AnnotationID" field to records and create new _annotation record_ type
1. For _schema records_ this points to the "defualt" annotation for that type.
2. For _content records_ it can be empty, in which case the default in the referenced _schema record_ is used, or it can contain a reference to an _annotation record_ itself which will override the default

# Keep it simple for now
Hand code solution for specific records we need to supprot at this time

## Concrete types
Figure out what are the concrete types we need to support

* Runtime binaries
* Go releases
* Git repository
* Do we want to register the entire dependency graph?
* Not necessary for the first pass
* Just top-level, standalone binaries
* Full indexing node stack
* Geth node modified with statediffing ETH IPLD ETL service (for head tracking): https://github.com/vulcanize/go-ethereum/tree/v1.10.15-statediff-3.0.1/statediff
* Standalone statediffing service for offline levelDB : https://github.com/vulcanize/eth-statediff-service
* Mempool/pending state indexing service: https://github.com/vulcanize/eth_probe
* ETH IPLD DB Schema: https://github.com/vulcanize/ipld-eth-db
* ETH IPLD data APIs: https://github.com/vulcanize/ipld-eth-server, https://github.com/vulcanize/tracing-api
* ETH IPLD state snapshotting tool: https://github.com/vulcanize/ipld-eth-state-snapshot
* Trailing validation code (WIP): https://github.com/vulcanize/ipld-eth-db-validator
* Snapshot validation code: https://github.com/vulcanize/eth-ipfs-state-validator
* IPLD ETL service for beacon chain: https://github.com/vulcanize/ipld-eth-beacon-indexer
* Beacon IPLD DB Schema: https://github.com/vulcanize/ipld-eth-beacon-db
* Watcher stack
* TS watcher: https://github.com/vulcanize/watcher-ts
* Uniswap V3 Watcher Overview: https://github.com/vulcanize/uniswap-v3-info-watcher
* Watcher that can run subgraphs: https://github.com/vulcanize/graph-watcher-ts
* Attributes:
* Name
* Reference to repository
* Description?
* Binaries
* Watcher stack
* FIN (full indexing node stack)
* Attributes:
* Raw multicodec
* Raw binary (reference to the raw binary as stored on IPFS)
* Targeted arch
* Runtine version e.g. go 1.18
* Reference to repository (reference to the nameservice record registering that repository)
* Version information
* Docker images
* Own file format
* Attributes
* Image ID
* Docker image binary (reference/CID to the binary stored in IPFS)
* Version number
* Reference to repository (reference to the nameservice record registering that repository)
* Watcher registration (watcher metadata/artifact)
* Note: we need to write IPLD schemas and register multicodecs for watcher metadata/artifact
* Attributes
* Watcher metadeta
* Talk to Ashwin
* Version
* Reference to the chain registrations for the the chains that are producing the data being consumed by this watcher
* Reference to git repository that contains the watcher source code (reference to its registration record)
* Compiled WASM binary of the watcher (a reference to it stored on IPFS)
* Build meta data
* WASM compiler version
* Execution engine version
* SDK generates the record
* Laconic has an endpoint for registering watchers, where you provide the unique set of attributes
* Responder contract schema
* We need to write IPLD schema and register multicodec
* Anyone can make an ask for a set of service providers to run watcher instances of a specific configuration
* Result of an auction
* We need an ask compiler which takes human readable input to generate the auction
* The responder contract record should be produced by the SDK as a result of a service provider auction
* Attributes
* Reference service providers (their registration records)
* Reference to the auction
* Reference to watchers (their registration records)

* Service provider registration
* Attributes
* Bond/Vault ID
* https://github.com/cerc-io/laconicd/blob/main/x/bond/types/bond.pb.go#L77
* Vault and Bond need to be merged
* Laconic ID
* DIDs
* Need to support multiple types of addresses (ethereum, cosmos)
* Treat as simple string for now
* Reference to legal identity
* Specifies location
* Use x.500
* Look into how/if it is hashlinked, can/should we provide explicity IPLD support (schemas + multicontent types + codecs)
* commonName - common name of a person, e.g., "Susan Jones"
* Usually the only field used
* Usually a DNS host name
* organizationUnit - small organization (e.g, department or division) name, e.g., "Purchasing"
* organizationName - large organization name, e.g., "ABCSystems, Inc."
* localityName - locality (city) name, e.g., "Palo Alto"
* stateName - state or province name, e.g., "California"
* country - two-letter country code, e.g., "CH"
* Npm packages
* For self hosting
* Attributes
* Reference to git repository that contains the npm source code (reference to its registration record)
* npm package (tar or w.e.; reference to it stored on IPFS)
* Version information
* Websites
* laconic.com
* mobymask's website
* our uniswap.info website
* Our endpoint which supports the uniswap.info front end
* Attributes
* url
* Reference to git repositor(ies) (reference to their registration records)
* Some type of package/build artifact for the website (uniswap.info is stored on IPFS?)
* TLS cert
* Ideally the orgs here match the orgs in service provider registration
* Chain registry
* Attributes
* Name
* Use an existing chain registry
* Starname
* List of IPLD types and their codec or schemas
* Fill in the rest....

## Start by supporting the below record types:

type ServiceProviderRecord struct {
    BondID string
    LaconicID string
    X500 struct {
        CommonName stirng
        Etc ....
    }c
}

type WebsiteRegistrationRecord struct {
    Url string
    RepoRegistrationRecordCID cid.CID
    BuildArtifactCID cid.CID
    TLSCertCID cid.CID // download cert, store on IPFS
}

// and then
type WatcherRegistrationRecord struct {
    ...
}

## Next steps
0. Update tendermint version in laconicd
* Is there a version of the v2 Multistore/SMT-SDK that works with tendermint 0.34.21?
* If not, either need to refactor our fork to work with 0.34.x or revert to using stock SDK
* Use stock SDK: https://github.com/cosmos/cosmos-sdk/releases/tag/v0.45.8
2. Write proto types for the above
3. Write the logic in the nameservice record registration/put methods to peform type introspection of the record.Attributes (resolve to one of the above protobuf types)
* https://github.com/cerc-io/laconicd/blob/main/x/nameservice/keeper/keeper.go#L243
4. Write the logic in the nameservice record put method (?) to write new secondary mapping between specific fields in the attributes type and the record type on disk
* Using the approach outlined by Murali
5. Testing
6. Modify the get methods in the keeper to leverage these new indexes

## Later
### Chain nameservice (research)
* We need to do research into which chain nameservice to use
* DIDs from fission???
* TendermintDNS?
* https://github.com/tendermint/cns
* https://github.com/ChainAgnostic/CAIPs/blob/master/CAIPs/caip-2.md
* https://github.com/cosmos/chain-registry
* IBC relayer registration?
* IBC directly to the beacon chain?