HackMDtitle: Backstage tech docs assessment notes
tags: backstage
Overview
This document contains notes about the CNCF Backstage documentation. These are working notes, taken as I reviewed the existing Backstage documentation and website. They are not intended to be a polished report of any kind.
The ultimate goal of this work is to deliver an assessment of the Backstage documentation as described in the CNCF assessment guidelines. When complete, the assessment will be included in GitHub as a separate document with the other CNCF tech doc assessments (in that same GitHub directory).
The assessed Backstage documentation includes:
- The Backstage website
- The technical documentation
- The GitHub repo
The assessment does not evaluate the Spotify Backstage website, which is a page for commercial Backstage plugins offered by Spotify.
General Comments
This section discusses topics that concern the documentation as a whole. These include documentation quality metrics, information architecture, and analyses of documentation users and objectives.
Documentation Quality
One way of assessig the overall documentation is evaluating documentation quality metrics, including:
- Completeness - what's missing?
- Clarity - is information understandable and interpretable?
- Findability - is anything hidden, mislabeled, or otherwise obscured?
- Effectiveness - does the documentation enable the user to fulfill their goals?
Information Architecture
Backstage is a platform for organizing and managing software projects. This complicates the documentation, because it can be difficult to distinguish among several levels of involvement with Backstage:
- Use Backstage as an organizational contributor (coder, DevOp, tech writer) to catalog , create, or document software.
- Administer a Backstage instance ("app" in Backstage nomenclature), including installation, deployment, basic configuration, and software upgrades.
- Customize a Backstage app, including installing plugins and advanced changes to configuration files.
- Extend a Backstage app, including writing plugins and modifying existing plugins.
- Modify or extend the Backstage platform.
This complexity necessitates clear definitions of roles so that use cases can be documented, indexed, found, and used. The five levels described above suggest that documentation should be organized under at least five roles:
- Using Backstage for software development: end user.
- Installing and maintaining a Backstage app for an organization: IT sysop/admin.
- Modify a Backstage app: Backstage admin.
- Extend a Backstage app: DevOp.
- Extend the Backstage platform: project contributor.
Some of these roles might be combined; for example, the administrator responsible for maintaining the Backstage app (2) could very well be responsible for extending and configuring it for new projects (3). Nonetheless, the roles have distinct tasks that should be documented in separate guides.
Users and Objectives
User roles should be defined in conjunction with objectives. Ideally, techncal objectives (what does the software do?) should be derived from business objectives (what is the software for?). Technical objectives differ depending on role.
Existing Website and Documentation
This is a survey of the existing documentation. It includes the product website, including the technical documentation proper (located under the Docs menu item on the website), and the associated GitHub repos.
Landing Page
(Not the docs landing page, but the project landing page)
- Clean design, consistent color palette and style.
- Briefly sells "Why Backstage?"
- Organized by product/component – Software Catalog, Software Templates, Backstage TechDocs, Backstage Search Platform, Backstage Kubernetes, plugins
Software Catalog
Software Templates
Backstage TechDocs
Backstage Search Platform
Backstage Kubernetes
Link is to a blog post. This is a core feature; it should be documented in the main documentation website. The blog post is from January 2021.
Plugins
Link button says "Contribute". This confuses learning about plugins with jumping in as a code contributor.
Menu
The banner menu. Contents:
GitHub | Docs | Plugins | Blog | Releases | Demos | Community
GitHub
This menu item links to:
https://github.com/backstage/backstage
Issues
Links to the Backstage platform GitHub page. Arguably should lead to an index of all the GitHub repos under https://github.com/backstage. This is a relatively minor issue with pros and cons to every alternative.
Recommendations
Alternatives:
- Create a landing page that is a curated index of Backstage repos. Pros: ease of use; findability. Cons: high maintenance.
- Link to the https://github.com/backstage page. Pros: overview of all Backstage repos; no separate maintenance needed. Cons: clutter. Many of the repos are specialized and irrelvant to a general audience.
- Leave as-is. Pros: direct link to what is probably the target for most users. Cons: leaves no way to know about or access the other repos.
Docs
Links to "What is Backstage?" in the technical documentation overview.
Plugins
Links to the "Plugin Marketplace". These link to their corresponding docs in the technical documentation. The plugin "Explore" buttons link to the plugin's home or code repo pages.
Issues
Not sure the core features need to be on this page; it's not where I'd look for them.
The plugins are listed in tile format. This takes a lot of room; there are well over a hundred plugins.
Recommendations
Consider a list format instead of tiles, or at least make it an option. Tiling is more appropriate for a few featured items, not an entire catalog.
Blog
Links to an active
Issues
Comments are turned off for the blog posts.
Recommendations
Write a short intro to the blog, explaining why comments are disallowed and who the bloggers are. Provide a link to a community where open discussion takes place.
Releases
Release notes. Links directly to the most recent release notes.
Issues
Direct link provides no context: no explanation of release strategy or cadence, no description of release note format.
Recommendations
Insert a Release Notes landing page that address the above issues. Also suggest a link to an explanation of semantic versioning. Also consider prepending a link to this context material to every version's release notes.
Demos
Links to a number (8) of demo videos.
Issues
The first demo's intro blurb also contains an overall intro to the demos, with links to other demos.
Recommendations
Separate the demo introduction and meta-info from the first demo blurb. Put it before the demos.
Community
Lists Backstage community resources by category:
- Official community platforms for the Backstage OSS community.
- "Official Backstage Initiatives".
- "Community Initiatives", two community resources hosted by Roadie.io (that duplicate "Official" resources).
- "Trainings and Certifications", containing one resource, a Linux Foundation course (labeled "Open Mic Meetup"; cut-paste error?).
- "Commercial Partners"
Issues
Not sure I'd consider all of these "community". The "Community Initiatives" section is problematic:
- It's unlabeled commercial partner content
- Its offerings are duplicates of "Official" content from the previous section
Recommendations
Move the "Communities Initiatives" items under Roadie.io in "Commercial Partners".
Footer
The footer content needs to be reorganized. It's divided into unconventional categories (and redundant with the TOC menu) and seems to be based on guesswork about what site visitors might want to see. Look at (almost) any commercial or OSS software footer for better choices.
Docs
"Designing for Backstage" is ambiguous. Use the word "contributor" to avoid this.
Community
More
3 of 4 are Spotify links. Just label it "Spotify" and put "Github" under "Community".
Tag
"Made with :heart: at Spotify". Acknowledgement that the product was developed at Spotify is fine, but this is confusing:
- Is this saying that the page was made at Spotify, or the product? This is often where site generator software or a website development service drops a brand mention (I don't like that practice either, but that's another issue).
- This detracts from the benefit of Backstage being an OSS project.
- There is no mention of CNCF. There should be. (Linux Foundation is acknowledged in the copyright notice.)
Documentation Sub-site
Intro on landing page: What is Backstage? and Benefits (good: benefits listed for user roles).
Overview
Architecture Overview
Issues
Backstage is represented in the documentation as consisting of a few core features (software catalog, tech docs, templates) and a library of plugins (Kubernetes, CI, etc.) Core components are implemented as plugins. The architecture overview says that the UI ties the plugins together.
Recommendation
It might be helpful to describe this architecture even more explicitly. Is there a procedure for promoting a plugin to core component status (as the Kubernetes plugin seems to be in process right now)? If so, document this procedure, both as an explanatory description in the user doc and a how-to in the contributor doc. Add to Release and Versioning Policy.
Roadmap
Might be better moved to the release notes.
Vision
Should be at or near the top of the Overview.
The Spotify Story
This belongs on the website somewhere, but not in the tech docs.
Strategies for Adopting
Probably belongs as part of a larger Best Practices section.
In general it seems like a lot of the material in the Overview mixes together information relevant to users and contributors (Strategies for Adopting, Release and Versioning Policy, Trust Model, Support and Chat Room). Another argument for making to doc user-centric.
Release and Versioning Policy
Trust Model
The Trust Model could be the basis for a contributor user analysis (which should be in the contributor documentation, separate from the user documentation). A separate analysis is also needed for users and properly belongs here in the user documentation.
Getting Started
Add an intro that outlines users and use cases. For an organization, the docs recommend having a group to administer Backstage, so administrator is one distinct user. Setup and config instructions should be geared to that user.
Getting Started
Rename to "Installing"
Begins with a claim that most users will want a standalone installation. Explain first what user types exist and which ones would want a standalone installation.
Getting Started, Configuring Backstage
Three consecutive headings with the same name are two too many. Rename to "Configuring".
Registering an Existing Component: Using the Backstage repo as an example is needlessly confusing. Backstage should develop a dedicated example repo that does nothing but provide a dummy component to add.
Create an App
Again here, user analysis would help. Elsewhere "App" is defined as an instance or installation of Backstage. This should be reiterated here.
Keeping Backstage Updated
Should be part of the admin guide.
Key Concepts
These are component technologies. Should be listed in the admin guide as dependencies.
Contributors
This should be its own guide. Put all the contributor information in one place. (Any information needed by both contributors and users can be referenced from the Contributor documentation. In general, don't duplicate information, especially reference information.)
Getting Involved
Should be part of Contributors Guide.
Backstage Project Structure
Should be part of contributors Guide. See notes on GitHub repos under Menu.
Local Development
This implies remote or distributed development. There needs to be an explanation of the distinction, again with an explanation that explains users and use cases for each. This seems to be contributor documentation, but it's not entirely clear. Might be admin? Get clarification.
This is from the tech docs > Core Features > Software Catalog:
… the Software Catalog enables two main use-cases:
1. Helping teams manage and maintain the software they own. Teams get a uniform view of all their software; services, libraries, websites, ML models — you name it, Backstage knows all about it. 2. Makes all the software in your company, and who owns it, discoverable. No more orphan software hiding in the dark corners of your software ecosystem.
Debugging Backstage
Logging levels. This should be in the Contributor info and in the Troubleshooting section for instances.
Core Features
Software Catalog
A lot of good information here. Most of it belongs in an architecture overview and in reference materials (YAML file format, e.g.).
How It Works
Lists use cases. These should be in overview.
Extending the Model
"The Backstage catalog entity data model is based on the Kubernetes objects format, and borrows a lot of its semantics as well."
You buried the lede! This should be at the front of the model information.
Creating the Catalog Graph
"The Software Catalog in Backstage is intended to capture human mental models using entities and their relationships rather than an exhaustive inventory of all possible things."
More good conceptual information!
Integrations
Plugins
Configuration
Auth and Identity
Permissions
Deployment
Designing for Backstage
Who is "we", the Backstage Design Team? Is this a Spotify unit? CNCF?
Need an explanation: who are the entities described in this section? Again, this is about defining the stakeholders so that the documentation can be targeted. Or in the case of contributors, define roles to allocate responsibilities.
"External" and "internal" refer to Spotify?
Includes "How to Contribute" – again, needs to be reorganized so that the user and contributor material isn't intermixed. A product used to manage software projects is confusing enough without throwing its own development on top without warning!
API Reference
Tutorials
Architecture Decision Records
FAQ
Experimental Backend System
Accessibility
Features
The main website landing page has sections for the core features. Not sure this is the best way to organize the first page potential users will see. Setting that aside for the moment, though, below are the features.
Software Catalog
"Learn More About the Software Catalog > More" button
https://backstage.io/docs/features/software-catalog/
Getting Started (section)
Assumes you've done Getting Started with Backstage and have the server installed already. (OK, to be fair, it says "If you've followed …". But still blindsided.)
Demo Server
It's not technically in the scope of the doc assessment, but the demo server has a serious documentation issue: The demo page does not properly demonstrate the TechDocs feature.
Rather than include integrated project documentation for one or more code projects, the only use of the TechDocs feature on the demo server is a link to the Backstage documentation site. There are numerous issues with this:
- It demos only a small part (basic display) of the Backstage TechDocs core feature.
- It fails to demonstrate integration of Backstage TechDocs with other plugins, especially the Software Catalog.
- It is self-referential and therefore unnecessarily confusing (in the same way using Backstage as an example in Getting Started is self-referential and confusing).
- According to the Backstage TechDocs feature press release, TechDocs is "… the most used plugin at Spotify by far". Its profile on the demo server belies this prominence.
- It speaks negatively about the importance of documentation to the Backstage development team.
Glossary
The glossary is in a separate document here.
Existing Backstage Glossaries
These are the only glossaries I could find on the site. They are inadequate and in any case the project should have an omnibus glossary as part of the documentation.
https://backstage.io/docs/overview/glossary/
Contains only the definitions of the user profiles.
https://backstage.io/docs/auth/glossary/
Auth and identity Glossary. 9 definitions, most to do with OAuth.
https://backstage.io/docs/local-dev/cli-overview/#glossary
CLI glossary. 5 definitions having to do with packaging.
https://backstage.io/docs/overview/architecture-overview/#terminology
Architecture overview terminology
3 definitions: Core, App, Plugins
Key Concepts
https://backstage.io/docs/getting-started/concepts
Not really "concepts", but component technologies: CHANGELOG, Docker, React, YAML, etc.
Website and Doc Infrastructure
The website and documentation source is in the backstage/backstage GitHub repository. The website and documentation are rendered using the Docusaurus static site generator.
The structure of the doc in the source repo is not obvious. Here's a selective listing of the directory hierarchy:
backstage/backstage # main repo
| docs # the documentation site: /docs
| | ... # contains subdirecories for doc sections
| microsite # website directory - contains various Docusaurus files
| | docusaurus.config.js # main Docusaurus config file
| | blog # blog entries: /blog
| | build # the compiled doc site (# cd microsite; yarn build)
| | data
| | | on-demand # tiles for past community session videos
| | | plugins # the plugin tiles on the plugins page: /plugins
| | src # source for the website pages
| | | components # code for website components
| | | pages # the website - most of the pages except for the documentation: /community, /demos, etc.
Next Steps (9/1/2023)
- Complete the Website portion of the Backstage doc assessment.
- Get an appropriate stakeholder coalition together to identify the user roles for Backstage.
- Same coalition can support idenification of use cases (work can probably be done by a tech writer).
- Outline a new documentation structure based on the user roles (tech writer).
