# TBD OSP Automated Docs Pipeline Proposal

## Context

TBD is building in the open. We value and we want to grow our community and increase the adoption of Web5 and TBDex.

## Problem

To be inclusive of the public, we should spread our knowledge about the code that we are building in our SDKs the best as we can. The easiest way is to publishing our docs-in-code to our developers portal website: https://developer.tbd.website. But, at the same time we need to move fast, we don't have time to be manually doing this work. Also, we are building things very dynamically with lots of changes in flux and it is very hard to keep track of the API changes, new functions that are being added in our libraries and more.

## Solution

Implement an automated pipeline that can be used in any projects/sdk that we want to publish in our developers portal. Such pipeline should be smart enough to notify developers about missing docs and docs notation mistakes.

## Goal

Help TBD engineers to move faster by:
- Generating docs from code and publishing in our website in an automated pipeline
- Notifying when there are exposed APIs in our SDK that needs to be documented
- Warn about accidental exposed APIs and also missing APIs that should be exposed
- Educating the community with accurate docs in real time, which potentially brings more adoption of our SDKs, more feedback and more contributions!

## Glossary

`source` - refers to the source code of the project/sdk. Eg: `tbdex-js`, `web5-js` etc.
`docs-report` - a report file with details about the documentation generation, also useful for PR branch policy ([api-extractor inspiration](https://api-extractor.com/pages/overview/demo_api_report/#:~:text=This%20design%20enables,change%20has%20occurred.))
`developers-portal` - https://developer.tbd.website
`docs-generator` - a tool that traverse the codebase and generates markdown files. Eg: typedoc-generator for typescript repos.
`next` - term used by docusaurus to represent unstable docs that are merged to main (untagged releases code).

## Automated Docs Pipeline Flow

### Step 1. PR is opened against **main**

The Docs bot simply parse the `docs-report` to create a helpful message to let the developer know if there are docs issues. The developer does not need to worry about it now if they don't want to. But if it's an easy low hanging fruit they could proactively fix it right away!

1. CI executes api-extractor to generate a `docs-report`
2. If `docs-report` detects docs changes, continue:
3. Interpret and extract chores from the `docs-report` such as warnings/errors
4. Add these chores as a new checklist with tasks comment in the PR
- (optional) Add a button to open an issue based with this PR comment, so that we can track the fix.

graph TD
    A[tbdex-js/protocol PR to main] --> B(CI triggers docs-report)
    B --> C{has docs issues?}
    C -- yes --> D[summarizes issues in a PR comment]
    C -- no --> F[End]
    D --> F

### Step 2. New candidate release being created

Here the bot will block the release until all the docs tasks are complete and blessed.

1. Compare `docs-report` with latest tagged release.
2. If `docs-report` has a warnings/errors: execute the steps 3 and 4 from the step 1 above. But now it's a blocking error for the release.
3. After `docs-report` is clean, open PR against the **`developers-portal` repo** with:
- Generated markdown files from `source` code docs annotations using a `docs-generator`
- Add them to the `source` API Reference folder
4. OSP will be able to do a sanity check on the netlify preview link
5. OSP can either accept and/or signalize new changes to OSE
6. OSP & OSE orchestrate the release and developers portal publishing process together

graph TD
    A[tbdex-js/protocol new RC]
    COMMIT[new commited changes: retriggers CI]
    RU[Release unblocked]

    A --> CI
    
    subgraph Stage 1 docs-report
        CI(CI triggers docs-report vs prior release)
        CI --> ERR{has warnings/errors?}
        ERR -- yes --> MSG[Bot summarizes docs-report message in the PR comment]
        MSG --> COMMIT
    end

    subgraph Stage 2 docs-generator
        DOCGEN[docs-generator generates markdown docs]
        DIFF{has docs diff?}

        ERR -- no --> DOCGEN
        DOCGEN --> DIFF
        DIFF -- yes --> PR[New automated PR to developers-portal]
        PR -- changes requested --> COMMIT
    end

    PR -- reviewed & approved --> RU
    DIFF -- no --> RU

# Docs Report

The docs API report file is a Markdown file with the summary of the API signatures, and also includes any Docs warnings and errors.

## Introducing the API report file in the repo

Here's an example of an initial file for the `@tbdex-js/protocol` repo (The current state without any tsdocs fixes): [tbdex-js/packages/protocol/protocol.api.md](https://github.com/leordev/tbdex-js/blob/957dfcf5fa89c7c482e0ba9e015d33856ea74a80/packages/protocol/protocol.api.md)

You can observe that it has a bunch of warnings. This is likely what will happen to any repo that we adopt this pipeline initially.

## Fixing the initial Report

And then, this is what the file looks like after all the warnings were addressed:
[FIXED tbdex-js/packages/protocol/protocol.api.md](https://github.com/leordev/tbdex-js/blob/94aa2244f04747c3bff27a287306452ea3146618/packages/protocol/protocol.api.md)

This is the commit with the changes needed to clean the report file: [94aa224](https://github.com/leordev/tbdex-js/commit/94aa2244f04747c3bff27a287306452ea3146618). The fixes were pretty much adding the release tag (added `@beta` everywhere) and small tsdocs adjusts.

## Frivolous Changes

Now, suppose a developer is working in a fix and actually the changes does not introduce any public exports or change any current API signatures. This is what their code diff will look like:
[Non API changes commit example](https://github.com/leordev/tbdex-js/commit/9c7fc7f221093b52b96fc1acd761f863e1582a69)

You will notice that in the above commit we introduced a new private method `validateRfqQuoteAmountSubunits` and a function `randomNonExportingFunction` that is not being exported and exposed when the package is built.

There are no changes in the API report file, because the changes actually didn't modify any API signature or export anything new to the package consumers.

## Introducing API changes

If the developer is implementing a new feature that introduces new exposed functions or changes the signature of an existing API, the report file will look have the [following diffs in the PR](https://github.com/leordev/tbdex-js/commit/a86f680f68bc97c321f74a90543d80bf8f0e903f#diff-38d187afee3b63a738d3a294ba63bbca7b996bfa8886c6a464f170f098c7e86c):

1. New exposed function

+// Warning: (ae-missing-release-tag) "superRfqUtilityFunction" is part of the package's API, but it is missing a release tag (@alpha, @beta, @public, or @internal)
+//
+// @public (undocumented)
+export function superRfqUtilityFunction(): string;

2. Removed a method from an exposed class and changed a signature of another method

@@ -406,7 +406,6 @@ export class Rfq extends Message<'rfq'> {
     static create(opts: CreateRfqOptions): Rfq;
     get offeringId(): string;
     get payinMethod(): SelectedPaymentMethod_2;
-    get payinSubunits(): string;
     get payoutMethod(): SelectedPaymentMethod_2;
     // (undocumented)
     _private: Record<string, any>;
@@ -414,7 +413,7 @@ export class Rfq extends Message<'rfq'> {
     toJSON(): MessageModel_2<"rfq">;
     readonly validNext: Set<MessageKind>;
     verifyClaims(offering: Offering | ResourceModel<'offering'>): Promise<void>;
-    verifyOfferingRequirements(offering: Offering | ResourceModel<'offering'>): Promise<void>;
+    verifyOfferingRequirements(offering: Offering | ResourceModel<'offering'>, relaxedCheck?: boolean): Promise<void>;
 }

By looking into the changes in the api report caused by the above commit we can parse it and derive the following PR comment through a bot:

>🤖 Hi there sir Dev! I detected a few documentation changes in your PR! Please check below!
>1. New exposed API fn `superRfqUtilityFunction`:
a. requires a release tag `(ae-missing-release-tag)`
b. requires documentation `(undocumented)`
>2. Removed API `Rfq.payinSubunits`:
> - [ ] I understand that we might have users actively using this API and I'm positive on this action.
>3. Changed API Signature `Rfq.verifyOfferingRequirements`:
> - [ ] I understand that we might have users actively using this API and I'm positive it's a benign backwards compatible change or it's not and I want to proceed.
>
> Please address above in this PR or [click on this link](#) to automatically create an issue to track these changes!

PS: The v0 MVP bot can be way simpler than the above parsing, just printing a list of warnings and asking the reviewer to check the report changes.

PS2: **How cool would be to enable a button to generate documentations drafts for all the `undocumented` APIs using ChatGPT**? _Not to mean this AI generated docs will be the final doc, but that button could generate a commit where the developer at least has something to work with (helps to overcome writer's block)._

## Docs Report recap

- The report file is tracked in Git
- The report file is generated on the build command automatically
- The developer must add the report changes otherwise the CI build will fail
- The changes are parsed by a github bot that will add a comment in the PR to facilitate the review (Pipeline Steps a.3, a.4 in the beginning of the page)
- We will have doc maintainers that will be automatically associated to the PR review when having diffs in the report file (Pipeline Steps a.5)
- If there are non api changes (frivolous) the file won't be changed and there is no need to the above PR review steps, no automations will be triggered

In typescript this file is generated by api-extractor and all the information about it can be found here: https://api-extractor.com/pages/overview/demo_api_report/

# Release Tags

The release tags control what we will expose to consumers of the API. It's inspired from [api-extractor](https://api-extractor.com/pages/tsdoc/doc_comment_syntax/#release-tags) as well.

## Typescript Docs

We are going to use three release tags: `@internal`, `@beta`, `@public`. They are applied to API items such as classes, member functions, enums, etc. API Extractor classifies each exported API item individually, according to its intended level of support:

- **internal**: Indicates that an API item is meant only for usage by other NPM packages from the same maintainer. Third parties should never use "internal" APIs. To emphasize this, underscore prefixes should be used for items with an (explicit) `@internal` tag.

- **beta**: Indicates that an API item has been released as a preview or for experimental purposes. Third parties are encouraged to try it and provide feedback. However, a "beta" API should NOT be used in production, because it may be changed or removed in a future version.

- **public**: Indicates that an API item has been officially released, and is now part of the supported contract for a package. If the [SemVer](https://semver.org/) versioning scheme is used, then the API signature cannot be changed without a MAJOR version increment.

> NOTE: TypeScript's `public`/`protected`/`private` keywords are unrelated to the AEDoc release tags. For example, a member function will typically have the `public` TypeScript keyword regardless of whether it is classified as `@public` or `@internal` in the doc comment.

When an API is first introduced, it typically starts as **beta**. As the design matures, it graduates from **beta** --> **public**. The **internal** designation is mostly used to solve plumbing problems, and usually isn't on any road map to becoming public. (There is also a `@deprecated` tag, but it is an option that can be combined with any of the above tags.)

The release tag applies recursively to members of a container (e.g. class or interface). For example, if a class is marked as `@beta`, then all of its members automatically have this status; you DON'T need add the `@beta` tag to each member function. However, you could add `@internal` to a member function to give it a different release status.

> NOTE: If a container (e.g. a class or interface) has an `@internal` tag, then an underscore prefix should be added to its name, but its members do NOT need underscores. Whereas e.g. if `@internal` is applied to a member function of a `@beta` class, then the internal function SHOULD have an underscore prefix.

Lastly, note that certain logical rules apply. For example, a `@public` function should not return a `@beta` type. A `@beta` class should not inherit from an `@internal` base class. etc. API Extractor does not currently validate these rules, but it will soon.

# Other things to Explore and Future Work

- Should `docs-report` errors block a PR?
- Maybe not now because we need to move fast, but definitely a yes when we reach 1.0
- Doc coverage? Apply them to releases?
- Should we split our `developer-portal` website by projects?
- With the new Branding efforts, each project would have it's own branding
- Make it easier for developers to navigate in the website
- And, also from [docusaurus docs](https://docusaurus.io/docs/docs-multi-instance):
- If you build a cross-platform mobile SDK, you may have 2 documentations:
- Android SDK documentation (v1.0, v1.1)
- iOS SDK documentation (v1.0, v2.0)
- WARNING: If each documentation instance is very large, you should rather create 2 distinct Docusaurus sites.
- If someone edits the iOS documentation, is it really useful to rebuild everything, including the whole Android documentation that did not change?
- More topics?...

## Versioning Docs

Docusaurus uses `/next` as the path to the documented code in the main branch, `/` as the latest released version and `/<tagged-version>` for historical versions. So we could have the following structure:

**tbdex-js**
https://developer.tbd.website/api/tbdex-js/ - latest release (eg v0.10.0)
https://developer.tbd.website/api/tbdex-js/next - main branch
https://developer.tbd.website/api/tbdex-js/v0.9 - release v0.9.x

**web5-js**
https://developer.tbd.website/api/tbdex-js/ - latest release (eg v0.8.1)
https://developer.tbd.website/api/tbdex-js/next - main branch
https://developer.tbd.website/api/tbdex-js/v0.7 - release v0.7.x

### Changelogs and Archiving

Docusaurus help managing past versions and archiving. Check the below references:

Jest - https://jestjs.io/versions
React Native - https://reactnative.dev/versions

Jest keeps the main + latest 4 releases, whereas React Native keeps main + latest 3 releases.

### Changes in the Pipeline with Versioning: Developer Portal with N Versions

It keeps the website up-to-date with the main branch:
- The latest tagged release is displayed in the regular website address
- The main branch keeps synced with the website in a `/next` address
- The older N versions are kept under the `/v{semver}` address
- older thna N is archived

#### Step 1. PR is opened against **main**

Exactly like the original proposal, but since we are going to be publishing to the developers website portal when merged to main we should:

1. Consider not skipping the docs chore
2. Add doc maintainers (most likely OSP as the website maintainers) as reviewers

#### Step 2. PR/Commit merged to **main**:

Here we need to merge the docs changes to the website portal.

1. If there are doc diffs, continue:
2. Open PR against the **`developers-portal` repo** with:
- Generated markdown files from `source` code docs annotations using a `docs-generator`
- Replace the `source` API Reference markdown files with them in the `next` folder:
- `developers-portal`/api/`source`/next
3. OSP will be able to do a sanity check on the netlify preview link
4. OSP can either accept and/or signalize new changes to OSE

#### Step 3. New release is published and tagged

Exactly like the original proposal but with a few extra steps for tagging the prior versioned docs in the website.

# References

- [Discord API Reference discussion thread](https://discord.com/channels/937858703112155166/1153511344457392158/1154145390694764657)