![](https://i.imgur.com/XIQ0XRn.png "oz logo" =200x)
# OZ-Upgrades Plugins - Solidity Libraries

## Rationale

The current versions of the [openzeppelin-upgrades](https://github.com/OpenZeppelin/openzeppelin-upgrades) library and plugins allow a user to deploy implementation contracts and proxies for any contract, **as long as they don't rely on Solidity libraries** with external functions that require them to be linked.

We need to design a solution that can **override** this restriction, allowing the user to deploy contracts and proxies with external linked libraries, if they have manually checked that the corresponding libraries are **Upgrade Safe**.

Github Issue: [#144](https://github.com/OpenZeppelin/openzeppelin-upgrades/issues/144).

---

## Current scenario

In the current scenario both plugins are **not behaving as expected** when trying to deploy or upgrade a contract that relies on external libraries. The outcomes and errors returned to the user in this operation do not match the required/implemented validations.

### Buidler:

* If a contract contains external linked libraries, the deployment **breaks** due to the plugin not identifying a previously **validated** version of that contract (by the bytecode hash). No warning/message related to the current validation is included.

Error: The requested contract was not found. Make sure the source code is available for compilation
    at getContractName (projects/openzeppelin-upgrades/packages/core/src/validate.ts:125:11)
    at Object.assertUpgradeSafe (projects/openzeppelin-upgrades/packages/core/src/validate.ts:150:24)
    at Proxy.deployProxy (projects/openzeppelin-upgrades/packages/plugin-buidler/src/deploy-proxy.ts:44:5)

### Truffle:

* If a contract contains external linked libraries, it is **deployed normally**, without notifying the user and enforcing the required validation.
* This occurs because Truffle does not replace the placeholder when **linking** a Contract to a Library. In the code below the bytecode extracted in `deployProxy` **does not contain** the referenced library.

const { deployProxy } = require('@openzeppelin/truffle-upgrades');

const SumLibrary = artifacts.require('SumLibrary');
const SimpleContractExtLib = artifacts.require("SimpleContractExtLib");

module.exports = async function (deployer) {
  deployer.deploy(SumLibrary).then(async () => {
    deployer.link(SumLibrary, SimpleContractExtLib);
    const instance = await deployProxy(SimpleContractExtLib, [1], { deployer });     
    console.log('Deployed', instance.address);  
  });
};

Deployment output:

Linking Contract: SimpleContractExtLib <--> Library: SumLibrary (at address: 0xA3E629dB246B8090A2AF82fabBeED3DC9C73c38e)

   Deploying 'SimpleContractExtLib'
   --------------------------------
   > transaction hash:    0x8ba3cb37ef619fc254e3c1923a350a062298f623279aa84a1c38276f239c6127
   ...

Deploying 'ProxyAdmin'
   ----------------------
   > transaction hash:    0x29cb8a171ebdca39a4ac62e80a849a0d8924aeed41c4942a55f0689d64ca9359
   ...
 
Deploying 'AdminUpgradeabilityProxy'
   ------------------------------------
   > transaction hash:    0x5a66f8367f07676c87faac47805f2fc621ac39bbd8b5b9940cbbe90c313c009c 
   ...
   
Deployed 0xC93850983C0a8ca968f5Ca97F48d03a1aA36DC9C

* In addition to this, the **`linkReferences`** section is not available for Truffle, because at the time of reconstructing the **`solc-output`** file we do not have the section in the generated artifacts. This causes the plugin not to be able to store the **`start` and **`length`** of the Placeholder in the validation process.

View code in [Github](https://github.com/OpenZeppelin/openzeppelin-upgrades/blob/e64368f0bea4e9c31ed833b75e75f49d3ae8e3d0/packages/plugin-truffle/src/validate.ts#L53)

**Note:** In both cases (Buidler and Truffle) the following error is displayed when the contract being deployed does not have the external libraries properly linked.

Error:  *** Deployment Failed ***
"SimpleContractExtLib" -- SimpleContractExtLib contains unresolved libraries. You must deploy and link the following libraries before you can deploy a new version of SimpleContractExtLib: SumLibrary.

---

## Proposed Solution

The proposed solution stores in the `validations` file the **`linkReferences`** output from the compiler that specifies the bytecode locations for linked libraries. This information is later used at **deployment** time to extract and replace the addresses from the linked contract.

This requires the user to first **deploy** the corresponding libraries, and **link** them to the contract. Then the user can call **`deployProxy`** or **`upgradeProxy`** with the linked contract factory as an argument.

In order to ensure a correct **validation**, the solution **extracts and replaces** the linked libraries addresses with the original **`placeholders`**, allowing to calculate the original/validated **version** (bytecode hash).

#### Process Diagram
The following **diagram** provides a high-level overview of the process. A detailed explanation of the changes is included next.


<br><br>

* 1 -The User **deploys** the required libraries and **links** the corresponding addresses to the implementation contract, before calling `deployProxy` and/or `upgradeProxy`.
* 2 -The User then calls **`deployProxy`** or **`upgradeProxy`** providing the implementation contract (and other parameters).
* 3-The process checks if there are `linkedReferences` for that contract, and **replaces** the linked addresses with the corresponding `placeholders`. This allows the plugins to identify the correct version (hash) of the contracts previously stored, and ensure all validations are properly checked.
* 4 - The process performs the required **validations**.
* 4-a) **NOT UPGRADE SAFE:** an error is displayed to the user preventing the depoyment/upgrade.
* 4-b) **UPGRADE SAFE:** (or has the appropiate override parameter) the contracts are **deployed/upgraded** using the linked `bytecode` and `linkReferences` information is carried over to the `manifest` file.

#### General Considerations

* **Validations**: The **`validations`** file/object can be created during compilation (Buidler) or when artifacts are processed (Truffle). This file will now include the **`linkReferences`** information, with containes the following fields: `sourceFile`,`libName`,`start`, `length`, and `placeholder` (required to recreate the original/validate `bytecode`).

* **Assumption**: In this process we assume users will **deploy** and **link** the corresponding libraries by themselves, providing a complete linked contract to the **`deployProxy`** and **`upgradeProxy`** functions.

* **Out of scope**: In this version we don't check the source code of the linked libraries to ensure they are **upgrade safe**. It is up to the user to set the appropiate override to proceed (`unsafeAllowLibraries`).

### Required changes

* Add a new deployment option **`unsafeAllowLibraries`**, allowing the user to **override** this check. When this flag is set to `false`, the process should **prevent** the deployment of contracts with external libraries and display the message **`Contract is not Upgrade Safe`**. When set to `true` it should allow the deployment/upgrade to proceed but showing a `warning` message to the user.

export interface DeployOptions {
  initializer?: string;
  unsafeAllowCustomTypes?: boolean;
  unsafeAllowLibraries?: boolean;
}

export function assertUpgradeSafe(validation: Validation, version: Version, unsafeAllowCustomTypes = false, unsafeAllowLibraries = false): void {}

* Store in the `validations.json` and `manifest.json` files the **`linkReferences`** information and bytecode positions for each external library, including the *sourceFile*, *libraryName*, *start*, *length*, and *placeHolder* attributes. This information allows us to **reconstruct** the original/validated bytecode if the contract being deployed or upgraded has links to external libraries.

 "linkReferences": [
      {
        "src": "contracts/SimpleContractExtLib.sol",
        "name": "SumLibrary",
        "length": 20,
        "start": 327,
        "placeHolder": "__$ef7ef997682fd8d4f4802da8e3f8a5a679$__"
      }
    ],
  ...  

* Create a new **`link-refs`** module that is in charge of providing interfaces and handling specific logic related to managing `LinkReferences`. It provides the following functions/interface:

import { SolcBytecode } from './solc-api';

export interface LinkReference {
  src: string;
  name: string;
  length: number;
  start: number;
  placeHolder: string;
}

// Get link references from SolcBytecode
export function extractLinkReferences(bytecode: SolcBytecode): LinkReference[]{...}

// Replace library addresses with placeholders
export function getUnlinkedBytecode(bytecode: string, linkReferences: LinkReference[]): string {...}

### Modified Files

List of required modifications by **`package/file`**:

### openzeppelin-upgrades Project

* `README.md`: Update FAQ for library management and include reference to the override option.
* `CHANGELOG.md`: Include override option and link to PR for the next release.
* `docs/modules/pages/*`: Update documentation based on the changes implemented.

### upgrades-core

* `validate.ts`

-`assertUpgradeSafe`: Add new parameter `unsafeAllowLibraries` and ensure validation is properly handled together with other override(s).
-`validate`: Ensure correct handling of validation. Store `linkReferences` information in `validatons` file.
-`errorInfo`: Adjust hint message suggesting use of `unsafeAllowLibraries`.
* `link-refs`: new file/module that handles `linkReferences` specific logic.
* `solc-api.ts`: Complete `SolcBytecode` with `length` and `start` fields.
* `manifest.ts`: Include `linkReferences` in `ManifestDataCodec`
* `stub-provider.ts`: Include `linkReferences` in `ImplDeployment`.
* `index.ts`: Export new module `./lib-refs`

### plugin-buidler

* `deploy-proxy.ts`
* Handle new parameter in `deployProxy` and include it in call to `assertUpgradeSafe`.
* Get `unlinked` bytecode so versions can be comparable and validations can be ensured.
* Include `linkReferences` in `manifest`.
* `upgrade-proxy.ts`:
* Handle new parameter in `upgradeProxy` and include it in call to `assertUpgradeSafe`.
* Get `unlinked` bytecode so versions can be comparable and validations can be ensured.
* Include `linkReferences` in `manifest`.

* `test/contracts/*.sol`: New contracts to test linked external libraries (TBD)
* `test/happy-path-with-ext-library` - Add tests for external linked libraries.
* `test.js`: Add new test files.
* `README.md`: Add explanation of new parameter `unsafeAllowLibraries`.`

### plugin-truffle

* `deploy-proxy.ts`
* Handle new parameter in `deployProxy` and include it in call to `assertUpgradeSafe`.
* Get `unlinked` bytecode so versions can be comparable and validations can be ensured.
* Include `linkReferences` in `manifest`.
* `upgrade-proxy.ts`:
* Handle new parameter in `upgradeProxy` and include it in call to `assertUpgradeSafe`.
* Get `unlinked` bytecode so versions can be comparable and validations can be ensured.
* Include `linkReferences` in `manifest`.

* `options.ts`: Add new parameter option for `unsafeAllowLibraries`.
* `validate.ts`: Manage link reference validations.
* `test/contracts/*.sol`: New contracts to test linked external libraries (TBD)
* `test/migrations/*.js` - Add migrations for new contracts.
* `migrations/happy-path-with-ext-library` - Add tests for external linked libraries
* `README.md`: Add explanation of new parameter `unsafeAllowLibraries`.

---

## Open Issues

1. **CLOSED** Should we keep the flag as **`unsafeAllowLibraries`** or make it more explicit as for example **`unsafeAllowLibraries`**.

Yes, process with `unsafeAllowLinkedLibraries` // Other possible option: `unsafeAllowExternalLibraries`

2. **CLOSED** Should I store the **`sourceFile`** in the **`linkReferences`** section in `validations` and `manifest` files or just the library name? Is it required for control in `upgradeProxy`?

 "linkReferences": [
      {
        "src": "contracts/SimpleContractExtLib.sol",
        "name": "SumLibrary",
        "length": 20,
        "start": 327,
        "placeHolder": "__$ef7ef997682fd8d4f4802da8e3f8a5a679$__"
      }
    ],
  ...  

Yes, may come up handy for displaying error messages.

3. **CLOSED** How can we avoid the **`contractName`** parameter to `deployProxy` and `upgradeProxy`? This allows us to get the contract `version` even when bytecodes don't match. One option is to **iterate** the `validations` file and find the contract that matches the bytecode hash once the placeholders are restored. This would allow to identify is the contract being deployed and enforce all validations.

Do the iteration process to find the correct contract in the `validations` object.

4. Do we need to store the **deployed** linked library addreses in the `manifest`?

For now there is no need to store that in the manifest. But may need to be revisited when analizing edge test cases.

5. **Truffle** Issues:
- 5.1- We do not have **`linkReferences`** as part of the compiler output artifacts.
* This is required to regenerate the `solc-output` file and handle external libraries. This causes the plugin not to be able to store the **`start` and **`length`** of the Placeholder in the validation process.

Check with Truffle. Open issue if not working properly.

- **CLOSED** 5.2- The **`link`** between the implementation contract and the external library is **not done at the bytecode** level.
* Should we make an **assumption** and require contracts to be **fully linked** at the bytecode level when calling `deployProxy` and `updateProxy`? How can we manage the Truffle linking issue? Should we have different behavior for Buidler and Truffle?

Identify where it is storing the linked address. Also try a script outside of migrations.

6. **Testing Case:** Analize the impact of the following case:
- A contract that is already deployed and has a reference to a `linked library`. Then only do a change to the library code and try to redeploy. We need to identify if the plugin recognizes that there is a change. Also we need to confirm it is properly using the new library and not reusing a previously deployed contract. We may need to do changes to the manifest and store other versions of the `bytecode` including the linked libraries.

7. **Pending Tasks:**
* Refactors
* Optimizations
* Truffle plugin
* Test cases
* Documentation
* Submit PR

---

## Appendix: How libraries are linked

Library linking happens at the **bytecode** level. When a contract thar relies on an external library is compiled, it leaves a **placeholder** for the library address in the form `__$e66798aa9224cd2742272d2e7f8089dbe6$__`. (Note: In previous compiler versions the placeholder can have a more human-readable name, such as `______________$Library$_______`)

A contract with placeholders in its bytecode **cannot be deployed** as it is considered invalid. All occurrences of library placeholders in the contract bytecode **need to be replaced** with the addresses of the pre-deployed libraries. Once the contract is linked to the corresponding libraries, it can be deployed to the blockchain.

The information to replace the placeholders is available in **`solc-output.json`** file. It includes a **`linkReferences`** section with the **`start`** and **`end`** positions to replace each library in the bytecode.

Example:

"contracts/SimpleContractExtLib.sol": {
      "SimpleContractExtLib": {
        "abi": [...],
        "evm": {
          "bytecode": {
            "linkReferences": {
              "contracts/SimpleContractExtLib.sol": {
                "SumLibrary": [
                  {
                    "length": 20,
                    "start": 327
                  }
                ]
              }
            },
            "object": "608060405.....",
            "opcodes": "PUSH1 0x80 PUSH1 0x40 ..."
        }
    ...
   }     

---