# Substrate Docs Forum Post

Over the last few months, I have spoken with some of you about my desire to start a process of revamping the substrate docs. This has been due to a combination of:

1. I felt the lack of quality in it first hand, especially while preparing for PBAs.
2. I felt the importance of it first-hand, especially while talking to students PBAs.
3. And my opinion has always been that the issue can only be fixed ground-up; by core-developers being more involved (at least at the beginning), not the other way around. So it made sense for me to volunteer to do this.

I talked to a bunch of people during the last few months, a few weeks ago distilled it all in a small actionable plan with help from Stefan, and finally presented it to all the engineering team leads last week. In this writing I want to share a summary of what the previous discussions with the rest of the company.

> My initial plan was to collaborate with Lisa about all of this, but this actually coincided with her leaving Parity. So, in some sense, we are starting from scratch here.

## Principles

First, I want to briefly re-iterate a set of principles that I formulated (and have talked about recently) about how we should think about documentation going forward, in a written format.

<details>
<summary>click here to see, to keep the main post shorter</summary>

#### Ground Up 🔺

I strongly think writing documentation is an absolute pain if you don't have a strong ground to build upon, and much easier if you do. This means having strong content to backlink to, instead of needing to re-explain everything, everywhere. This starts with our [rust-docs](http://paritytech.github.io/substrate/).

> *What can be documented in rust-docs, should be documented in rust-docs.*

#### Less is More 🧘

I want to commit to a smaller documentation portal that covers less ground, but in more depth and more quality. This means relying on external content for as much as possible. Things that have been explained elsewhere, by people more suitable than us, should be covered by them. This includes a majority of the fundamentals ("*what is a blockchain*") or specialized technologies that are beyond the scope of me and/or a small documentation team to maintain, such as Ink! and XCM (each of which (will) have their own documentation portal).

> *What can be documented externally, should be documented externally.*

3. Don’t Repeat Yourself – DRY 🪄

This is a mere aggregation of the above two points; let's start small, stand on the shoulder of others/elsewhere for as much as possible, and write as least as possible. The less docs we write, the less we have to maintain it.

> *What is already documented in rust-docs or externally, should simply not be repeated again*.

> This has a trade-off, where, as pointed out by Erin, I can imagine our docs being to littered with links. But, I think it is the lesser of the two evils.

4. By Developers, For Developers 👩‍💻

a good documentation flow cannot start if core-devs don't take the first step. I am not saying everything should be done by them, but the first step should be taken by them, for others to stand upon. This *disconnect* has been, in my opinion, one of the main reasons that our docs have not progressed as much in the past.

This includes a few action items that, given the "ground-up" principle, I will try and tackle as early as possible:

* better PR description and titles.
* requirement for developers to identify if their PR is touching the application programming interface aka. API[^1] of our software, or our documentations (to the best of their knowledge) and if so:
* pay extra attention to the code in that PR being well documented.
* flag the PR (through either a request review, label, or similar) to be audited/reviewed by the those who maintain documentation[^2].

This is a very similar process to what developers are expected to do with regards to audit, now in a new dimension.

> *What can *reasonably* be expected to be documented by core-developers (based on the above criteria), should be documented by core-developers*.

5. Story > Facts 💌

Lastly, we should make sure that our documentation forms a cohesive story, rather than being an unorganized blob of facts. This especially applies to anything that wants to be worthy of the title of a "tutorial". But, even for lower level content like "reference docs", I think the key to forming stories is using backlinks as much as possible, establishing new facts on top of existing ones.

</details>

<br>

To sum up:

> Real knowledge .. it’s built from the *ground up*. To use a math example, you can’t understand trigonometry without understanding arithmetic and geometry... Richard Feynman very famously does this in “Six Easy Pieces,” one of his early physics lectures. He basically explains mathematics in three pages. He starts from the number line— counting—and then he goes all the way up to precalculus. He just builds it up through an unbroken chain of logic. He doesn’t rely on any definitions.
> - [source](https://www.navalmanack.com/almanack-of-naval-ravikant/how-to-think-clearly)

## Action Items

With all of that said, I intend to initiate the following action-items:

1. a new, long-form, multi-step tutorial, focusing on the single most journey a developer can take in the realm of Polkadot, which is going for nothing, to a wasm runtime that can be deployed on a Polkadot core (regardless of whether it is scheduled as a parachain, parathread, etc), dubbed "FRAME Master Tutorial". This should cover the majority of technology pieces (ie. FRAME features) needed to develop a reasonably capable parachain. A lot of my time will go into this, and my goal is to have an initial version of it ready by July for PBA3, and even using it there. Follow that discussion here: https://github.com/Polkadot-Blockchain-Academy/pba-content/issues/592.
2. In collaboration with lots of teams, an audit/review system, as explained above, to hopefully improve the quality of our rust-docs, at least where it matters.
3. In collaboration with lots of teams, a CI system to make sure all code snippets in the new docs repo remain up to date.
4. Whatever is neither document-able in the rust-docs, nor does it fit in the master tutorial will be documented in a "reference docs" section. In the early steps, this should be as little as possible.
5. In collaboration with marketing, a new landing page that showcases all the different paths that a builder can take in our ecosystem. You can see my interpretation of these paths in [this diagram](https://mermaid.live/edit#pako:eNplU22vmjAU_itdE-MXYYDAVT7chHG3hEWmoRJvBn6oUCe50JLaZnOG_74Kd1fUJk045zzPeXk4PcOcFQR6MKOj0bmkpfDAeSwOpCZjb0yJFBxX47YdjTKa0X3FfucHzAVYxBkF6hzl7hfHzQEsqQqUtPdeDlqnX5LwZbEFmvYMUJCi-kIMGFUpc7G9IgecoAOH9O3TgxOxqixKccoGNT6qR_goCAdrKRgvcXVF9J10CeJ046MIxJKKsiZAA99iP_q6ve3gcuIOvvJjP90ApMYlBUAkl1xVV7RA1rKSx8EAhBbDrno-Wi6W6ebz8j7DO09x3sveC7nfd0oOZekHSPw08ZtGtYBqVhVsqGHid5DAX4UPztV3lKoLVOyRgeTudT0odTXvOwx_rNOQKplZQzjelVUvx2sQbT8Amq5rzyi4teNbM_EzCiewJrzGZaFW73yJZ7Bbugx66rPA_C1TK9kqHFb_FJ1oDj3BJZlA2RRYkJcSK7nq_84G05-MDU3oneEf6JmObli2O3Ptmes8TV3LnMAT9KwnW587lmHMTMuZmrbVTuDfLoGhu5ZrO7Zp2nNzOrcNewKJ2jvGo_6hdO-l_QeaquaK).

## Partners in Crime

None of this is really a one-person job.

- The DevRel team has already been very supportive of my through brainstorming, and I am looking forward to further integrate this effort into their vibrant team, if possible. For now, our plan is for me to focus more on the content, and for them to think more about hosting, representation and delivery, but this will surely evolve.
- Two new people have been officially hired into the Substrate's docs team (in the engineering department) who I will be onboarding over the next months, and working together with.
- This is all happening when a lot of forethought about unifying the brand identity of Substrate and Polkadot (and consequently their websites, consequently where all of their docs will be hosted) is happening, and we will have to work together with Marketing and their web team to find both a short term, and a long term solution.

## Where Next?

- First, a hard, but hopefully correct, decision: I don't plan to spend a lot of effort in maintaining the existing set of content. We should put minimal effort in minimizing factual errors and such, but I don't see it within my abilities to implement the above plan there. Instead, we will:
- as noted, try and only fix factual errors, if any.
- mark documents with "last-modified" date, hopefully being a first signal that this webpage *might be old*.
- The new platform will be independent and build from the ground up, either in a new domain, or a new url-prefix -- TBD based on the brand identity matters, discussed above, which is partially beyond my control.
- Gradually, we will move useful content from the old platform into the new one, all the while adhering to the above 5 principles, and peer reviewing them for correctness.
- Eventually, we will mark all the more and more pages of the existing docs as "*deprecated*", and ideally with a link to its new version, if it exists.
- This will also give us a longer time to make sure we don't destroy our SEO by breaking all the links to existing content, which is being actively used in SE, Forums, and so on. This is another effort that needs collaboration with the web team from Marcomms.
- Given the uncertainty about where/how this will be hosted, I will start in a fresh new repo, only containing markdown and rust files that are snippets. This repo's issues will also act as the main task-management platform for this initiative.
- Other than forum and this thread, in terms of communication:
- I will be monitoring the Documentation Team room in Engineering General
- Probably soon, I will start a weekly office-hour call.

[^1]: What is exactly out software API? I have to examine this further, but off the top of my head, I think think of: 1. All FRAME Pallets 2. The entirety of `frame-support` (perhaps can be made more specific), `sp-runtime`, `sp-arithmetic` and a few other handpicked items. All in all, you should be able to open a reasonably complicated substrate node-template, and all of the types, functions, macros that you see and use should be very well documented.
[^2]: Which I would imagine consists of the DevRel team, the documentation team and a few select volunteers from core-development, myself included.