# ethPM V3

# Agenda
- [x] introductions
- [x] review agreed upon breaking changes
- camelCase
- `"natspec"` -> `"userdoc"` / `"devdoc"`
- `"name"` & `"version"` optional (both are required if either is included)
- [x] `"manifest"` / `"version"` / `"name"`
- potential conflict w/ solc v1 metadata
- `{"manifest": "3"}` / `{"manifest": "ethpm/3"}`
- [ ] `"sources"`
- require keys to be valid paths that resolve to a virtual working directory?
- enforce formatting? (eg. `./` prefix in v2)
- how to encode checksums
- [ ] `"contractTypes"`
- how to identify a `"contractTypes"` respective source file?

        {
            "contractTypes": {
                "MyContract": {
                    "compiler": {
                        "compilationTarget": {
                            "./path/MyContract.sol": "MyContract"
                        }
                    }
                }
            }
        }

or

        {
            "contractTypes": {
                "./path.sol:MyContract": {
                    // sourcePaths = list of source files necessary to recompile this contract type
                    "sourcePaths": ["./path/MyContract.sol"]
                }
            }
        }

- why is `"compiler"` a subfield of `"contractTypes"`?
- [ ] solc standard json input / output compatibility?
- [ ] next steps
- update docs / json schema / libraries / tooling
- target eta for implementations?

# Goals
- Define a compatible JSON schema that satisfies requirements for Solidity/Vyper compilers and `ethPM` tooling
- Facilitate improved verification of smart contract txs via metadata ([source-verify](https://github.com/ethereum/source-verify))
- Automatically generated, published, well specified, language agnostic, code legos

# Use cases
- ethPM tooling / framework integrations
- [Compiler metadata](https://solidity.readthedocs.io/en/v0.6.2/metadata.html)
- Compiler JSON input/output (potentially)

# Stakeholders
This list is not exhaustive nor exclusive, feel free to invite anybody who would like to be involved.

- @chriseth (Solidity)
- @fubuloubu (Vyper)
- @iamdefinitelyahuman (Brownie / Vyper)
- @gnidan (Truffle)
- @njgheorghita (ethPM)
- @pipermerriam (ethPM)
- @loredanacirstea (Pipeline)
- @spalladino (zeppelin)

# Requirements

### ethPM
- Compatibility with existing tooling
- Supports [v2 use cases](http://ethpm.github.io/ethpm-spec/use-cases.html#id1)
- The ability for tooling to faithfully recreate the source tree from a package

### Solidity compiler
- The metadata can be generated from the inputs to the compiler for every bytecode that is generated without having to add any additional information.
- "source-verify" is centered around the metadata and the fact that it is hash-anchored into the bytecode. Sets of compiled contracts are out of scope, this is just about retrieving the natspec and source code of a contract on-chain for the purpose of showing additional information when signing a transaction
- It has to be possible to compute the original input to the compiler from the metadata and recompile resulting in exactly the same bytecode and metadata

### Vyper
- ...

# Breaking changes from V2
- `packageName` & `packageVersion` can be omitted, but both are required if either is included
- `manifestVersion` -> `version`
- `version` -> `packageVersion`
- `sources` obj. definition
- `contractTypes` obj. definition

# V3 Specification
Largely inspired by ethpm V2.
- [V2 human readable spec](http://ethpm.github.io/ethpm-spec/)
- [V2 repository](https://github.com/ethpm/ethpm-spec)

# `"version"`
- Defines the specification version that this document conforms to.
- Required: **Must** be `"3"`
- `{"version":"3"}`

# `"packageName"`
- Defines a human readable name for this package.
- Required if `"packageVersion"` is defined.
- Spec: [Same as ethpm V2](http://ethpm.github.io/ethpm-spec/package-spec.html#package-name-package-name)

# `"packageVersion"`
- Declares the version number of this release.
- Required if `"packageName"` is defined.
- Spec: [Same as ethpm V2](http://ethpm.github.io/ethpm-spec/package-spec.html#version-version)

# `"meta"`
- Defines package-level metadata (authors, links, license ...).
- Optional
- Spec: [Same as ethpm V2](http://ethpm.github.io/ethpm-spec/package-spec.html#package-meta-meta)

# `"sources"`

> - Defines a source tree that **should** comprise the full source tree necessary to recompile the contracts contained in this release. Sources are declared in a key/value mapping.
> - Paths **must** resolve to a path within the current virtual directory.
> - If the resulting document is a directory the key should be interpreted as a directory path.
> - If the resulting document is a file the key should be interpreted as a file path.

- Key: `"sources"`
- Value: `SourcesObject`

`SourcesObject`
- Key: (string) Compilation source file path.
- Value: `SourceObject`

`SourceObject`
- `"keccak256": string` (optional) keccak256 hash of the source file
- `"urls": List[string]` (required if no `"content"`) List of urls that resolve to the same source file - **must** be stored on a content-addressable filesystem (i.e. swarm, ipfs, github blobs).
- `"content": string` (required if no `"urls"`) Source string

ex.

{
    "sources": {
        "GnosisSafe.sol": {
            "keccak256": "0x123...",
            "content": "pragma ..."
        },
        "Owned.sol": {
            "urls": ["ipfs://"]
        },
        "zeppelin/ERC20.sol": {
            "keccak256": "0xabc...",
            "urls": ["ipfs://", "bzzr://"]
        }
    }
}  

# `"contractTypes"`
> - Packages **should** only include contract types that can be found in the source files for this package.
> - Packages **should** not include contract types from dependencies.
> - Packages **should** not include abstract contracts in the contract types section of a release.

- Key: `"contractTypes"`
- Value: `Object[contract_alias, ContractTypeObject]`

#### `contract_alias: string`
This is typically the contract name, unless there are duplicate contract names in a package, then an alias must be used. Every `contract_alias` must be unique within a given package.

- The contract alias must use one of the following naming schemes (as defined in ethpmV2)
- `<contract-name>`
- `<contract-name>[<identifier>]`
- The `<contract-name>` portion must be the same as the Contract Name for this contract type - conforms to the regex `[a-zA-Z][-a-zA-Z0-9_]{0,255}`
- The `[<identifier>]` portion must match the regular expression `\[[-a-zA-Z0-9]{1,256}\]`

#### `ContractTypeObject`
- ### `contractName`
Required: If [`contract_alias`](http://ethpm.github.io/ethpm-spec/glossary.html#term-contract-alias) (the key) is different from the contract name.
- **Must** conform to the regex `[a-zA-Z][-a-zA-Z0-9]{0,255}`

- ### `deploymentBytecode`
Required: No
Type: object
Format: [ethpmv2 bytecode object](https://ethpm.github.io/ethpm-spec/package-spec.html#the-bytecode-object)

- ### `runtimeBytecode`
Required: No
Type: object
Format: [ethpmv2 bytecode object](https://ethpm.github.io/ethpm-spec/package-spec.html#the-bytecode-object)

- ### `abi`
Required: No
Type: List
Format: Must conform to [Ethereum Contract ABI JSON format](https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI#json)

- ### `devdoc`
Required: No
Type: Object
Format: **Must** conform to the solc [devdoc format](https://solidity.readthedocs.io/en/v0.6.2/natspec-format.html#developer-documentation)

- ### `userdoc`
Required: No
Type: Object
Format: **Must** conform to the solc [userdoc format](https://solidity.readthedocs.io/en/v0.6.2/natspec-format.html#user-documentation)

- ### `compiler`
Required: No
Type: Object
Format: `CompilerInformationObject`

#### `CompilerInformationObject`
- `"compilationTarget": String` (optional) `<source_file>:<contract_name>`
- `"keccak256": String` (optional) Hash of the compiler binary which produced this output
- `"version": String` (optional) Version of the compiler
- `"language": String` (optional) Source code language
- `"settings": MetadataSettingsObject` (optional) The settings field defines any settings or configuration that was used in compilation. This **should** conform to the [Solidity Metadata "Settings" Description](https://solidity.readthedocs.io/en/latest/metadata.html).

# `"deployments"`
- The information for the chains on which this release has contract instances and other details for those deployments.
- Spec: [Same as ethpm V2](http://ethpm.github.io/ethpm-spec/package-spec.html#deployments-deployments)

# `"buildDependencies"`
- A key/value mapping of Ethereum Packages that this project depends on.
- Spec: [Same as ethpm V2](http://ethpm.github.io/ethpm-spec/package-spec.html#build-dependencies-build-dependencies)

# Custom fields
Custom fields may be included. Custom fields **should** be prefixed with `x-` to prevent name collisions with future versions of the specification.

# Formatting
- The document **must** be tightly packed, meaning no linebreaks or extra whitespace.
- The keys in all objects **must** be sorted alphabetically.
- Duplicate keys in the same object are invalid.
- The document **must** use UTF-8 encoding.
- The document **must** not have a trailing newline.

# Previous versions
### solc (metadata v1)

{
    "evm": {
        "bytecode": {...},
        "deployedBytecode": {...}
    },
    // metadata obj should be a json string but displayed here for visualization
    "metadata": {
        "compiler": {"version": "0.5.10+commit.5a6ea5b1"},
        "language": "Solidity",
        "output": {
            "abi": [...],
            "devdoc": {...},
            "userdoc": {...}
        },
        "settings": {
            "compilationTarget": {"GnosisSafe.sol": 'GnosisSafe'},
            "evmVersion": "petersburg",
            "libraries": {},
            "optimizer": {"enabled": False, "runs": 200},
            "remappings": []
        },
        "sources": {
            "GnosisSafe.sol": {
                "keccak256": "0xf529ce...",
                "urls":["bzzr://f0360...", "dweb:/ipfs/QmaLjUWc..."]
            }
        },
        "version": "1"
    }
}

### ethpm V2

{
    "package_name": "gnosis-safe",
    "version": "1.0.0",
    "manifest_version": "2",
    "contract_types": {
        "GnosisSafe": {
            "abi": [...],
            "deployment_bytecode": {
                "bytecode": "0x123"
            },
            "runtime_bytecode": {
                "bytecode": "0xabc"
            },
            "compiler": {
               "name": "solc",
               "version": "0.5.10+commit.5a6ea5b1",
               "settings": {
                  "optimize": True 
               }
            },
            "natspec": {...union of userdoc & devdoc...}
        }
    },
    "sources": {
        "./GnosisSafe.sol": "ipfs://QmaLjUWc..."
    }
}

# V3

Example solc metadata output with ethPM v3 formatting

{
    "version": "3"
    "contractTypes": {
        "GnosisSafe": {
            "abi": [...],
            "contractName": "GnosisSafe",
            "compiler": {
                "keccak256": "0x123..."
                "version": "0.5.10+commit.5a6ea5b1",
                "language": "Solidity",
                "settings": {
                    "compilationTarget": {"GnosisSafe.sol": 'GnosisSafe'},
                    "libraries": {},
                    "metadata": {},
                    "optimizer": {"enabled": False, "runs": 200},
                    "remappings": []
                }
            },
            "devdoc": {...},
            "userdoc": {...}
        }
    },
    "sources": {
        "GnosisSafe.sol": {
            "keccak256": "0xf529ce...",
            "urls":["bzzr://f0360...", "dweb:/ipfs/QmaLjUWc..."]
            "type": "jsonabi", // should be from predefined list
            "license": "MIT"
        }
    }
}

#### solc output

{
    'abi': [...],
    'devdoc': {...},
    'evm': {...},
    'ethpm': '{"contractTypes":{"GnosisSafe":{"abi":[...],"compiler":{"version":"0.5.10+commit.5a6ea5b1","language":"Solidity","compilationTarget":{"GnosisSafe.sol":"GnosisSafe"},"evmVersion":"petersburg","libraries":{},"optimizer":{"enabled":False,"runs":200},"remappings":[]},"devdoc":{...},"userdoc":{...}}},"sources":{"GnosisSafe.sol":{"keccak256":"0xf529ce...","urls":["bzzr://f0360...","dweb:/ipfs/QmaLjUWc..."]}},"version":"3"}'
}

# TODO
- Finalize & Freeze spec
- Create test cases around edge cases (i.e. deps of deps of deps)
- Update [ethpm-spec](https://github.com/ethpm/ethpm-spec) & [machine readable schema](https://github.com/ethpm/ethpm-spec/blob/master/spec/package.spec.json)
- Implement v3 into tooling
- Blogs / Tweets / Announcements

{
    "sources": {
        "http://abc.com": {
            "urls": ["ipfs://Qm..."]
        }
    },
    "contractTypes": {
        "http://abc.com:ABC": {
            "abi": ...
        }
    }
}

{
    "deployments": {
        "blockchain://123abc...": {
            "abcdeployment": {
                "contract_type": "abc:http://abc.com:ABC",
                "address": "0xC708..."
            }
        }
    },
    "buildDependencies": {
        "abc": "ipfs://Qmabc..."
    }
}