---
# System prepended metadata

title: Record content indexing

---

# Record content indexing

Two needs:
1. Typing of record attributes
2. Annotation of the record attributes type


## Typing of record attributes
1. Convert attributes type from `string` to protobuf `any.Any`
2. Add new "TypeID" field to record
    * For _schema records_ this field will be empty but for _content records_ it will contain a CID that references the _schema record_ which contains the typing information for this record's attributes
3. Register _schema record_ that contains in its attributes the FileDescriptorProto set for a _content record's_ attributes
4. Use [DynamicMessage](https://protobuf.dev/reference/java/api-docs/com/google/protobuf/DynamicMessage) type to construct Go type from referenced [FileDescriptorProto](https://github.com/protocolbuffers/protobuf/blob/main/src/google/protobuf/descriptor.proto#L61) set to unmarhsal _content record_ attributes into
5. Look into using https://github.com/ipld/go-ipld-prime/tree/master/node/bindnode to IPLDize the Go types constructed above

## Annotating of record attributes
1. Annotation needs to be in separate schema
    1. To tell which feilds you want to index against
    2. Qualifiers which indicate a field cannot be some set of values (null, zero, etc)
2. Add addition "AnnotationID" field to records and create new _annotation record_ type 
    1. For _schema records_ this points to the "defualt" annotation for that type.
    2. For _content records_ it can be empty, in which case the default in the referenced _schema record_ is used, or it can contain a reference to an _annotation record_ itself which will override the default


# Keep it simple for now
Hand code solution for specific records we need to supprot at this time 


## Concrete types
Figure out what are the concrete types we need to support

* Runtime binaries
    * Go releases
* Git repository
    * Do we want to register the entire dependency graph?
        * Not necessary for the first pass
            * Just top-level, standalone binaries
    * Full indexing node stack
        * Geth node modified with statediffing ETH IPLD ETL service (for head tracking): https://github.com/vulcanize/go-ethereum/tree/v1.10.15-statediff-3.0.1/statediff
        * Standalone statediffing service for offline levelDB : https://github.com/vulcanize/eth-statediff-service
        * Mempool/pending state indexing service: https://github.com/vulcanize/eth_probe
        * ETH IPLD DB Schema: https://github.com/vulcanize/ipld-eth-db
        * ETH IPLD data APIs: https://github.com/vulcanize/ipld-eth-server, https://github.com/vulcanize/tracing-api
        * ETH IPLD state snapshotting tool: https://github.com/vulcanize/ipld-eth-state-snapshot
        * Trailing validation code (WIP): https://github.com/vulcanize/ipld-eth-db-validator
         * Snapshot validation code: https://github.com/vulcanize/eth-ipfs-state-validator
         * IPLD ETL service for beacon chain: https://github.com/vulcanize/ipld-eth-beacon-indexer
        * Beacon IPLD DB Schema: https://github.com/vulcanize/ipld-eth-beacon-db
    * Watcher stack
        * TS watcher: https://github.com/vulcanize/watcher-ts
        * Uniswap V3 Watcher Overview: https://github.com/vulcanize/uniswap-v3-info-watcher
        * Watcher that can run subgraphs: https://github.com/vulcanize/graph-watcher-ts
    * Attributes: 
        * Name
        * Reference to repository
        * Description?
* Binaries
    * Watcher stack
    * FIN (full indexing node stack)
    * Attributes: 
        * Raw multicodec
        * Raw binary (reference to the raw binary as stored on IPFS)
        * Targeted arch
        * Runtine version e.g. go 1.18
        * Reference to repository (reference to the nameservice record registering that repository)
            * Version information
* Docker images
    * Own file format
    * Attributes
        * Image ID
        * Docker image binary (reference/CID to the binary stored in IPFS)
        * Version number
        * Reference to repository (reference to the nameservice record registering that repository)
* Watcher registration (watcher metadata/artifact)
    * Note: we need to write IPLD schemas and register multicodecs for watcher metadata/artifact
    * Attributes
        * Watcher metadeta
            * Talk to Ashwin
            * Version
            * Reference to the chain registrations for the the chains that are producing the data being consumed by this watcher
        * Reference to git repository that contains the watcher source code (reference to its registration record)
        * Compiled WASM binary of the watcher (a reference to it stored on IPFS)
            * Build meta data
                * WASM compiler version
                * Execution engine version
    * SDK generates the record
        * Laconic has an endpoint for registering watchers, where you provide the unique set of attributes
* Responder contract schema
    * We need to write IPLD schema and register multicodec
    * Anyone can make an ask for a set of service providers to run watcher instances of a specific configuration
    * Result of an auction
    * We need an ask compiler which takes human readable input to generate the auction
    * The responder contract record should be produced by the SDK as a result of a service provider auction
    * Attributes
        * Reference service providers (their registration records)
        * Reference to the auction 
        * Reference to watchers (their registration records)
        
* Service provider registration
    * Attributes
        * Bond/Vault ID
            * https://github.com/cerc-io/laconicd/blob/main/x/bond/types/bond.pb.go#L77
            * Vault and Bond need to be merged
        * Laconic ID
            * DIDs
                * Need to support multiple types of addresses (ethereum, cosmos)
                * Treat as simple string for now
        * Reference to legal identity
            * Specifies location
            * Use x.500
                * Look into how/if it is hashlinked, can/should we provide explicity IPLD support (schemas + multicontent types + codecs)
                * commonName - common name of a person, e.g., "Susan Jones"
                    * Usually the only field used
                    * Usually a DNS host name
                * organizationUnit - small organization (e.g, department or division) name, e.g., "Purchasing"
                * organizationName - large organization name, e.g., "ABCSystems, Inc."
                * localityName - locality (city) name, e.g., "Palo Alto"
                * stateName - state or province name, e.g., "California"
                * country - two-letter country code, e.g., "CH"
* Npm packages
    * For self hosting
    * Attributes
        * Reference to git repository that contains the npm source code (reference to its registration record)
        * npm package (tar or w.e.; reference to it stored on IPFS)
        * Version information
* Websites
    * laconic.com
    * mobymask's website
    * our uniswap.info website
        * Our endpoint which supports the uniswap.info front end
    * Attributes
        * url
        * Reference to git repositor(ies) (reference to their registration records)
        * Some type of package/build artifact for the website (uniswap.info is stored on IPFS?)
        * TLS cert
            * Ideally the orgs here match the orgs in service provider registration
* Chain registry
    * Attributes
        * Name
            * Use an existing chain registry
                * Starname
        * List of IPLD types and their codec or schemas
        * Fill in the rest....



## Start by supporting the below record types:

```go=
type ServiceProviderRecord struct {
    BondID string
    LaconicID string
    X500 struct {
        CommonName stirng
        Etc ....
    }c
}

type WebsiteRegistrationRecord struct {
    Url string
    RepoRegistrationRecordCID cid.CID
    BuildArtifactCID cid.CID
    TLSCertCID cid.CID // download cert, store on IPFS
}

// and then
type WatcherRegistrationRecord struct {
    ...
}
```

## Next steps
0. Update tendermint version in laconicd
    * Is there a version of the v2 Multistore/SMT-SDK that works with tendermint 0.34.21? 
    * If not, either need to refactor our fork to work with 0.34.x or revert to using stock SDK
        * Use stock SDK: https://github.com/cosmos/cosmos-sdk/releases/tag/v0.45.8
2. Write proto types for the above
3. Write the logic in the nameservice record registration/put methods to peform type introspection of the record.Attributes (resolve to one of the above protobuf types)
    * https://github.com/cerc-io/laconicd/blob/main/x/nameservice/keeper/keeper.go#L243
4. Write the logic in the nameservice record put method (?) to write new secondary mapping between specific fields in the attributes type and the record type on disk
    * Using the approach outlined by Murali
5. Testing
6. Modify the get methods in the keeper to leverage these new indexes

## Later
### Chain nameservice (research)
* We need to do research into which chain nameservice to use
* DIDs from fission???
* TendermintDNS?
* https://github.com/tendermint/cns
* https://github.com/ChainAgnostic/CAIPs/blob/master/CAIPs/caip-2.md
* https://github.com/cosmos/chain-registry
* IBC relayer registration?
* IBC directly to the beacon chain?