Image Not Showing Possible Reasons

• The image file may be corrupted
• The server hosting the image is unavailable
• The image path is incorrect
• The image format is not supported

Learn More →

Bazel and JavaScript

Bazel community update

August 18 2022

Bazel: most scalable polyglot Build System.

JavaScript & TypeScript: most popular languages.

Note:

• Bazel works with all languages, let's make it good with the most popular ones.
• image from https://octoverse.github.com/#top-languages-over-the-years

Who is Alex Eagle

• Worked at Google on DevInfra 2008-2020
• Bazel most of that time: TL for Google's CI, build/test results UI, Angular CLI
• Started Rules Authors SIG, maintained rules_{nodejs,python,docker}
• twitter.com/jakeherringbone

What is Aspect

I Co-founded Aspect Development to make Bazel the industry-standard full-stack build system

• https://aspect.dev - Support and consulting to help you adopt Bazel
• https://aspect.build - Products making Bazel easier to use
• https://github.com/aspect-build - rules_js is part of our Bazel rules ecosystem

Build systems:

Matrix / Hub-and-Spoke

• Grunt and Gulp fell out of favor
• Instead, each tool became a Build System
• Now each tool needs a plugin for each language

ModernWeb Meetup: Layering in JS tooling
https://www.aspect.dev/resources

Note:

• I drew this live in my talk
  Image Not Showing Possible Reasons

  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported

  Learn More →
• When each column is a tool like webpack or jest
• each row is a language like typescript or sass
• MxN support matrix - so many plugin authors to trust
• A new framework like Qwik has to support every CSS preprocessor??

Like Gulp or Grunt, but way (way) better.

Let's use Bazel!

What is NodeJS

JavaScript engine that runs outside the browser.

Typically used for running dev tools to
build and test JavaScript programs.

Deno is an up-and-coming alternative, see rules_deno

Note:

• just a runtime, requires a package manager

What is rules_nodejs

Bazel rules forked from Google-internal

• toolchain to run hermetic NodeJS interpreter
• shared Bazel interfaces ("Providers") like TypeScript DeclarationInfo

What is rules_js

rules_js is a layer on top of rules_nodejs

About rules_js: My talk from Bazel eXchange 2022

Slides: https://hackmd.io/@aspect/rules_js
Video: https://aspect.dev/resources (first video)

What is pnpm

• "Fast, disk space efficient package manager": https://pnpm.io/
• Works with nearly the whole ecosystem
• Used by the https://rushjs.io/ monorepo JS-only build tool
• Happens to fit perfectly with Bazel semantics!
• Used by rules_js

Note:

• Symlinks to a "virtual store" of fetched packages rather than nested layout
• Their lockfile happens to present 100% of the info needed to fetch AND link the node_modules tree

JS: Client and Server

Client runtime: Browser
Server runtime: Node.js
Both use the same virtual machine (V8), stdlib differs

• rules_nodejs supports both, though confusingly named
• rules_js also supports both

Modules

digraph G {

  rankdir="TB"
  fontname="courier"
  
  subgraph cluster_bazelbuild {
      label = "github.com/bazelbuild"

      subgraph cluster_0 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        nodejs_binary
        nodejs_star [label="nodejs_*"]
        yarn_install
        label = "build_bazel_rules_nodejs";
      }

      subgraph cluster_1 {
        
        toolchain
        label = "rules_nodejs";
        color=blue
      }
  }

  subgraph cluster_npm {
      label = "npm"
      subgraph cluster_2 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        ts [label="ts_project"]

        label = "@bazel/typescript";
      }
    subgraph cluster_6 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        other [label="..."]

        label = "@bazel/*";
      }
      
  }
  
  subgraph cluster_aspect {
      label = "github.com/aspect-build"
      subgraph cluster_3 {
        js_binary
        js_star [label="js_*"]
        npm_star [label="npm_*"]
        label = "rules_js";
        color=blue
      }

      subgraph cluster_4 {
        ts_project
        label = "rules_ts";
        color=blue
     }
      subgraph cluster_7 {
        other_rules [label="..."]
        label = "rules_*"
        color=blue
     }
      subgraph cluster_8 {
        copy_to_bin
        label = "bazel_lib"
        color=blue
     }

  }
  other_rules -> js_binary
  ts -> nodejs_binary
  nodejs_binary -> toolchain
  js_binary -> toolchain
  ts_project -> js_binary
  js_binary -> copy_to_bin
}

NEWS: Shaded boxes are no longer maintained!

Fetch and install npm packages

How npm/yarn solve it

npm install

Install everything needed for the whole
package/workspace

Any build/test script can depend on all npm packages

How Google solves it

Vendor the world: copy npm ecosystem sources into VCS

• Never fetches from the internet
• Never runs any package installation

You could do it this way too.

How rules_nodejs solved it

Just wrap [npm|yarn] install - install the world

Note:

• https://blog.aspect.dev/avoid-eager-fetches

rules_js: ideal solution

• re-use pnpm's resolver (via lockfile)
• fetch with Bazel's downloader
• unpack tarballs with Bazel
• re-use @pnpm/lifecycle to run hooks
  • these are actions - can be remote cached
• link node_modules

Note:

• More later about linking the node_modules

https://blog.aspect.dev/rulesjs-npm-benchmarks

Best case:

• BUILD file declares fine-grained deps
• build only depends on one library
• we only fetch/install one library!

Note:

• This is possible because BUILD files have finer-grained graph than package.json does

Resolving npm dependencies at runtime

How it works in npm

NodeJS programs rely on a node_modules folder

"Was a big mistake" says NodeJS creator, and
Deno fixes it (but here we are

The location of node_modules is expected to be relative to the location of the importing script.

How Google solves it: patch require

Same strategy as "PnP", e.g. Yarn PnP.

How rules_nodejs solves it: runtime "linker"

Similar to npm link: use symlinks to make monorepo libraries appear in the node_modules tree

Note:

• rules_nodejs also has the Google "patch require" strategy but is now discouraged.
• doesn't make deps under the same tree with sources - 'rootDirs' troubles

How rules_js solves it

Node.js tools assume the working dir is a single tree of src/gen/node_modules: we can do that!

• "link" to bazel-bin/node_modules/...
• copy sources to bazel-bin
• actions first cd bazel-out/[arch]/bin

Note:
This requires copy_to_bin and a bit of care in custom rules.

Code Examples

https://github.com/aspect-build/bazel-examples/tree/main/vue

1. WORKSPACE: read 3p npm package lockfile
2. Link npm packages
3. BUILD: declare dependencies on npm packages
4. Run tools from npm
5. Wrap tools in macros
6. Custom rules

Roadmap

rules_js 1.0.0 is available now

Coming soon

• Missing RBE support due to bug with Bazel resolving symlinks
• Gazelle extension to generate BUILD files from srcs
• Bazel 6.0 package manager: bzlmod instead of WORKSPACE
  https://blog.aspect.dev/bzlmod

Thanks!

Q&A time

Or come to our office hours
calendly.com/aspect-build/rules_js-office-hours