Who is Alex Eagle

• Worked at Google on DevInfra 2008-2020
• Bazel most of that time: TL for Google's CI, build/test results UI, Angular CLI
• twitter.com/jakeherringbone

What is Aspect

I Co-founded Aspect Development to make Bazel the industry-standard full-stack build system

• https://aspect.dev - Support and consulting to help you adopt Bazel
• https://aspect.build - Products making Bazel easier to use
• https://github.com/aspect-build - rules_js is part of our Bazel rules ecosystem

What is Bazel

• Build system for "every" langugage
• Incremental: re-build proportional to what you changed
• Cached/parallel: distribute over server farm
• Scalable: works for Google's 2 billion line repo
• Unix Philosophy: just spawns subprocesses, which can be any tool

More: https://www.aspect.dev/resources

When to consider Bazel for frontend

• Large-scale: 1M SLOC / 100 devs
• Monorepo: same use cases as Nx/Rush/Lerna
• Polyglot/full-stack: parachute anywhere
• Integration testing: fast test against backend
• Have a DevInfra team: economy of scale

More: https://www.aspect.dev/resources

None of those apply?

Small, disconnected JS apps shouldn't use Bazel.

The build system recommended by your framework is well supported for small-to-medium scale.

What is NodeJS

JavaScript engine that runs outside the browser.

Typically used for running dev tools to
build and test JavaScript programs.

What is pnpm

• "Fast, disk space efficient package manager": https://pnpm.io/
• Works with nearly the whole ecosystem
• Used by the https://rushjs.io/ monorepo JS-only build tool
• Happens to fit perfectly with Bazel semantics!

What is rules_nodejs

Bazel rules forked from Google-internal

• toolchain to run hermetic NodeJS interpreter
• shared Bazel interfaces ("Providers") like TypeScript DeclarationInfo

rules_js is a layer on rules_nodejs
build_bazel_rules_nodejs is replaced

ModernWeb Meetup: Layering in JS tooling
https://www.aspect.dev/resources last one

Like Gulp or Grunt, but way (way) better.

Let's use Bazel!

Bazel: Loading Phase

Load and evaluate all extensions, BUILD files and macros that are needed for the build.

digraph {
rankdir="LR"
"sources" -> "dependency graph"
"external repos" -> "dependency graph"
}

bazel fetch [targets]

https://blog.aspect.dev/configuring-bazels-downloader

• Bazel is full-featured for fetching external deps
• Can air-gap, security scan, artifactory, etc
• Supply-chain secure, Trust-on-first-use model
• Cache based on integrity hashes

Bazel: Dependency Graph

digraph {
  compound=true
  rankdir="LR"

  graph [ fontname="Source Sans Pro", fontsize=20 ];
  node [ fontname="Source Sans Pro", fontsize=18];
  edge [ fontname="Source Sans Pro", fontsize=12 ];

  subgraph cluster1 {
   label="monorepo"
   
   my_app -> my_lib
   my_app -> my_comp
  }
  subgraph cluster2 {
    label="otherrepo"
    
    other_team_lib
  }
  subgraph cluster3 {
    label="registry.npmjs.com"

    request
    react
    express
  }
  
  my_lib -> request
  my_comp -> react
  my_app -> express
  my_app -> other_team_lib
  
}

bazel query --output=graph [targets]

digraph mygraph {
  rankdir="LR"
  node [shape=box];
  "//examples/macro:node_modules/mocha/dir"
  "//examples/macro:node_modules/mocha/dir" -> "//examples/macro:node_modules/mocha"
  "//examples/macro:node_modules"
  "//examples/macro:node_modules" -> "//examples/macro:node_modules/mocha"
  "//examples/macro:node_modules" -> "//examples/macro:node_modules/mocha-junit-reporter"
  "//examples/macro:node_modules" -> "//examples/macro:node_modules/mocha-multi-reporters"
  "//examples/macro:node_modules/mocha"
  "//examples/macro:node_modules/mocha-junit-reporter/dir"
  "//examples/macro:node_modules/mocha-junit-reporter/dir" -> "//examples/macro:node_modules/mocha-junit-reporter"
  "//examples/macro:test"
  "//examples/macro:test" -> "//examples/macro:_test_srcs\n//examples/macro:test__entry_point"
  "//examples/macro:test" -> "//examples/macro:node_modules/mocha-junit-reporter"
  "//examples/macro:test" -> "//examples/macro:node_modules/mocha-multi-reporters"
  "//examples/macro:node_modules/mocha-junit-reporter"
  "//examples/macro:node_modules/mocha-multi-reporters/dir"
  "//examples/macro:node_modules/mocha-multi-reporters/dir" -> "//examples/macro:node_modules/mocha-multi-reporters"
  "//examples/macro:node_modules/mocha-multi-reporters"
  "//examples/macro:_test_srcs\n//examples/macro:test__entry_point"
}

Bazel: Analysis Phase

digraph {
rankdir="LR"
"dependency graph" -> "action graph" [label = "Bazel rules"]
}

Action: for a requested output, how to generate it from some inputs and tools

e.g. “if you need hello.js, run swc on hello.ts”.

Requires predicting the outputs!

bazel aquery [targets]

digraph {
rankdir="LR"

"index.ts" -> "index.js" [label=swc]
"component.ts" -> "component.js" [label=tsc]
"index.js" -> "bundle.js" [label=esbuild]
"component.js" -> "bundle.js"
"styles.scss" -> "styles.css" [label=sass]
"styles.css" -> "serve" [label=express]
"bundle.js" -> "serve"
}

Bazel: Execution Phase

Execute a subset of the action graph by spawning subprocesses (e.g. node)

digraph {
rankdir="LR"
"changed index.ts" [shape=note]
"changed index.ts" -> "index.js" [label=swc]
"component.ts" [shape=none]
"component.js" [shape=none]
"component.ts" -> "component.js"
"index.js" -> "bundle.js" [label=esbuild]
"component.js" -> "bundle.js"
"styles.scss" [shape=none]
"styles.scss" -> "styles.css"
"styles.css" [shape=none]
"styles.css" -> "serve" [label=express]
"bundle.js" -> "serve"
}

User requested certain targets be built.

Bazel is lazy and will only:

• fetch precise dependencies needed
• run actions required by the transitive dependency closure of those targets
• run actions that are a "cache miss"

How npm/yarn solve it

npm install

Install everything needed for the whole
package/workspace

Any build/test script can depend on all npm packages

How Google solves it

Vendor the world: copy npm ecosystem sources into VCS

• Never fetches from the internet
• Never runs any package installation

You could do it this way too.

How rules_nodejs solved it

Just wrap [npm|yarn] install - install the world

Guaranteed slow when repo rule invalidates
Extra bad when "eager fetching" npm deps

rules_js: ideal solution

Port pnpm to Starlark

• re-use pnpm's resolver (via lockfile)
• fetch with Bazel's downloader
• unpack tarballs with Bazel
• re-use @pnpm/lifecycle to run hooks
  • these are actions - can be remote cached
• link node_modules

https://blog.aspect.dev/rulesjs-npm-benchmarks

Best case:

• BUILD file declares fine-grained deps
• build only depends on one library
• we only fetch/install one library!

Workspaces

Mix of third-party and first-party deps in a tree of package.json files.

Google: single version policy
rules_nodejs: independent top-level dep installs
rules_js: supports pnpm workspaces!

How it works in npm

NodeJS programs rely on a node_modules folder

"Was a big mistake" says NodeJS creator, and
Deno fixes it (but here we are )

The location of node_modules is expected to be relative to the location of the importing script.

How Google solves it: patch require

Same strategy as "PnP", e.g. Yarn PnP.

not compatible. Many npm packages wrote their own require implementation.

How rules_nodejs solves it: runtime "linker"

Similar to npm link: use symlinks to make monorepo libraries appear in the node_modules tree

Slow beginning of every NodeJS spawn
Links appear in source tree w/o sandbox
Bins don't work with genrule/ctx.actions.run
Not compatible with "persistent workers"

How rules_js solves it

Linker is now just a standard Bazel target

Node.js tools assume the working dir is a single tree of src/gen/node_modules: we can do that!

• "link" to bazel-bin/node_modules/...
• copy sources to bazel-bin
• actions first cd bazel-out/[arch]/bin

Install

Copy the WORKSPACE snippet from latest release.

https://github.com/aspect-build/rules_js/releases

Adopt pnpm

Just run pnpm install and check that your workflows work.

A few npm packages still have "hoisting bugs" where they don't declare correct dependencies and accidentally rely on npm or yarn-specific layout.

import pnpm-lock.yaml

npm_translate_lock converts to Bazel's format (Starlark).

WORKSPACE

load("@aspect_rules_js//npm:npm_import.bzl",
     "npm_translate_lock")
npm_translate_lock(
    name = "npm",
    pnpm_lock = "//:pnpm-lock.yaml",
)
 
# Load the starlark version of the lockfile
load("@npm//:repositories.bzl", "npm_repositories")
npm_repositories()

Link the npm packages

BUILD (next to package.json)

load("@npm//:defs.bzl", "js_link_all_packages")
 
js_link_all_packages()

Result of bazel build :all is now

# the virtual store
bazel-bin/node_modules/.aspect_rules_js 
 # symlink into the virtual store
bazel-bin/node_modules/some_pkg
# If you used pnpm-workspace.yaml:
bazel-bin/packages/some_pkg/node_modules/some_dep

bazel build examples/...

Link first-party packages

First declare the package…

my-lib/BUILD

load("@aspect_rules_js//npm:defs.bzl", "npm_package")
 
npm_package(
    name = "lib",
    srcs = [
        "index.js",
        "package.json",
    ],
)

Link first-party packages

… then link to bazel-bin/node_modules tree…

app/BUILD

load("@aspect_rules_js//npm:defs.bzl", "npm_link_package")
 
npm_link_package(
    name = "node_modules/@mycorp/mylib",
    src = "//examples/lib"
)

…then depend on it just like it came from npm!

app/BUILD

js_binary(
    name = "my_app",
    data = [
        "//:node_modules/react-dom",
        "//:node_modules/@mycorp/mylib",
    ],
    entry_point = "index.js",
)

bin entries are provided for all packages

BUILD

load("@npm//typescript:package_json.bzl", typescript_bin = "bin")
 
typescript_bin.tsc(
    name = "compile",
    srcs = [
        "fs.ts",
        "tsconfig.json",
        "//:node_modules/@types/node",
    ],
    outs = ["fs.js"],
    chdir = package_name(),
    args = ["-p", "tsconfig.json"],
)

Each bin exposes three rules:

Wrap existing build system

Use "component libraries" to get coarse granularity

digraph {
rankdir="LR"
vite [label=vite]
vite2 [label=vite]
"lib sources" [shape=none]
"lib sources" -> "vite"
"app sources" [shape=none]
"app sources" -> "vite2"
vite -> vite2
vite2 -> bundle
bundle [shape=none]
}

Pretty fast developer loop in
https://github.com/aspect-build/bazel-examples/tree/main/vue

load("@npm//vite:package_json.bzl", vite_bin = "bin")
load("@npm//vue-tsc:package_json.bzl", vue_tsc_bin = "bin")
 
vite_bin.vite(
    name = "dist",
    args = ["build"],
    out_dirs = ["dist"],
    srcs = [...],
)
 
vue_tsc_bin.vue_tsc_test(
    name = "type-check",
    args = ["--noEmit"],
    data = [...],
)
 
vite_bin.vite_binary(
    name = "vite",
    data = [...]
)

ibazel run :vite

Write a Macro

Bazel macros are like preprocessor definitions.

Good way to give "syntax sugar",
compose a few rules, set defaults.

Indistinguishable from custom rules at use site

Example: mocha test

def mocha_test(name, srcs, args = [], data = [], env = {}, **kwargs):
    bin.mocha_test(
        name = name,
        args = [
            "--reporter",
            "mocha-multi-reporters",
            "--reporter-options",
            "configFile=$(location //examples/macro:mocha_reporters.json)",
            native.package_name() + "/*test.js",
        ] + args,
        data = data + srcs + [
            "//examples/macro:mocha_reporters.json",
            "//examples/macro:node_modules/mocha-multi-reporters",
            "//examples/macro:node_modules/mocha-junit-reporter",
        ],
        env = dict(env, **{
            # Add environment variable so that mocha writes its test xml
            # to the location Bazel expects.
            "MOCHA_FILE": "$$XML_OUTPUT_FILE",
        }),
    )

https://github.com/aspect-build/rules_js/blob/main/examples/macro/mocha.bzl

Write a custom rule

Harder and not recommended for most users.

Start from
https://bazel.build/rules/rules-tutorial
and use
https://github.com/bazel-contrib/rules-template

• rules_webpack - Bazel rules for webpack bundler https://webpack.js.org/
• rules_rollup - Bazel rules for https://rollupjs.org/ - a JavaScript bundler
• rules_jest - Bazel rules to run tests using https://jestjs.io
• rules_deno - Bazel rules for Deno http://deno.land

… and many more by other vendors

http://docs.aspect.build

Catalog coming soon at https://bazel-contrib.github.io/SIG-rules-authors/

Example custom rule: ts_project

No more rootDirs in tsconfig.json

Thank you!

These slides: https://hackmd.io/@aspect/rules_js

Thanks conference organizers and everyone who helped launch rules_js.

Come work with us on OSS! http://aspect.dev/careers
Paid support and consulting: http://aspect.dev
Our projects: github.com/aspect-build