---
title: Bazel and JS
tags: Bazel
type: slide
description: Bazel community update, August 18 2022
slideOptions:
theme: white

---

# :herb: Bazel and JavaScript

## Bazel community update

## August 18 2022

---

Bazel: most scalable polyglot Build System.


JavaScript & TypeScript: most popular languages.

Note:
- Bazel works with all languages, let's make it good with the most popular ones.
- image from https://octoverse.github.com/#top-languages-over-the-years

---

## Who is Alex Eagle

<img src="https://avatars0.githubusercontent.com/u/47395?s=460&v=4" style="border:2px solid grey; border-radius: 200px;" width=100/>

- Worked at Google on DevInfra 2008-2020
- Bazel most of that time: TL for Google's CI, build/test results UI, Angular CLI
- Started Rules Authors SIG, maintained rules_{nodejs,python,docker}
- twitter.com/jakeherringbone

----

## What is Aspect

I Co-founded Aspect Development to make Bazel the industry-standard full-stack build system

- <https://aspect.dev> - Support and consulting to help you adopt Bazel
- <https://aspect.build> - Products making Bazel easier to use
- <https://github.com/aspect-build> - rules_js is part of our Bazel rules ecosystem

----

## Build systems:
### Matrix / Hub-and-Spoke

:construction: The JS ecosystem took a wrong turn

- Grunt and Gulp fell out of favor
- Instead, each tool became a Build System
- Now each tool needs a plugin for each language

----


ModernWeb Meetup: Layering in JS tooling
https://www.aspect.dev/resources :point_right: last one

Note:
- I drew this live in my talk :laughing:
- When each column is a tool like webpack or jest
- each row is a language like typescript or sass
- MxN support matrix - so many plugin authors to trust
- A new framework like Qwik has to support every CSS preprocessor??

----


Like Gulp or Grunt, but way (way) better.

Let's use Bazel!

----

## What is NodeJS

JavaScript engine that runs outside the browser.

Typically used for running dev tools to
build and test JavaScript programs.

Deno is an up-and-coming alternative, see rules_deno

Note:
- just a runtime, requires a package manager

----

## What is `rules_nodejs`

Bazel rules forked from Google-internal

- toolchain to run hermetic NodeJS interpreter
- shared Bazel interfaces ("Providers") like TypeScript `DeclarationInfo`

----

## What is `rules_js`

rules_js is a layer on top of `rules_nodejs`

About rules_js: My talk from Bazel eXchange 2022

Slides: https://hackmd.io/@aspect/rules_js
Video: https://aspect.dev/resources (first video)

----

## What is `pnpm`

- "Fast, disk space efficient package manager": <https://pnpm.io/>
- Works with nearly the whole ecosystem
- Used by the <https://rushjs.io/> monorepo JS-only build tool
- Happens to fit perfectly with Bazel semantics!
- Used by `rules_js`

Note:
- Symlinks to a "virtual store" of fetched packages rather than nested layout
- Their lockfile happens to present 100% of the info needed to fetch AND link the node_modules tree

----

## JS: Client and Server

Client runtime: Browser
Server runtime: Node.js
Both use the same virtual machine (V8), stdlib differs

- `rules_nodejs` supports both, though confusingly named
- `rules_js` also supports both

----

Modules

digraph G {

  rankdir="TB"
  fontname="courier"
  
  subgraph cluster_bazelbuild {
      label = "github.com/bazelbuild"

      subgraph cluster_0 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        nodejs_binary
        nodejs_star [label="nodejs_*"]
        yarn_install
        label = "build_bazel_rules_nodejs";
      }

      subgraph cluster_1 {
        
        toolchain
        label = "rules_nodejs";
        color=blue
      }
  }

  subgraph cluster_npm {
      label = "npm"
      subgraph cluster_2 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        ts [label="ts_project"]

        label = "@bazel/typescript";
      }
    subgraph cluster_6 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        other [label="..."]

        label = "@bazel/*";
      }
      
  }
  
  subgraph cluster_aspect {
      label = "github.com/aspect-build"
      subgraph cluster_3 {
        js_binary
        js_star [label="js_*"]
        npm_star [label="npm_*"]
        label = "rules_js";
        color=blue
      }

      subgraph cluster_4 {
        ts_project
        label = "rules_ts";
        color=blue
     }
      subgraph cluster_7 {
        other_rules [label="..."]
        label = "rules_*"
        color=blue
     }
      subgraph cluster_8 {
        copy_to_bin
        label = "bazel_lib"
        color=blue
     }

  }
  other_rules -> js_binary
  ts -> nodejs_binary
  nodejs_binary -> toolchain
  js_binary -> toolchain
  ts_project -> js_binary
  js_binary -> copy_to_bin
}

NEWS: Shaded boxes are no longer maintained!

---

# Fetch and install npm packages

----

### How npm/yarn solve it

`npm install`

Install everything needed for the whole
package/workspace

Any build/test script can depend on all npm packages :face_with_rolling_eyes:

----

### How Google solves it

Vendor the world: copy npm ecosystem sources into VCS

- Never fetches from the internet
- Never runs any package installation

You *could* do it this way too. :face_with_one_eyebrow_raised:

----

### How rules_nodejs solved it

Just wrap `[npm|yarn] install` - install the world

:thumbsdown: Guaranteed slow when repo rule invalidates
:thumbsdown: Extra bad when "eager fetching" npm deps

Note:
- https://blog.aspect.dev/avoid-eager-fetches

----

### rules_js: ideal solution

:point_right: Port pnpm to Starlark :point_left:

- re-use pnpm's resolver (via lockfile)
- fetch with Bazel's downloader
- unpack tarballs with Bazel
- re-use `@pnpm/lifecycle` to run hooks
- these are actions - can be remote cached
- link `node_modules`

Note:
- More later about linking the node_modules

----

https://blog.aspect.dev/rulesjs-npm-benchmarks

Best case:
- BUILD file declares fine-grained deps
- build only depends on one library
- we only fetch/install one library!

----


Note:
- This is possible because BUILD files have finer-grained graph than package.json does

---

# Resolving npm dependencies at runtime

----

### How it works in npm

NodeJS programs rely on a `node_modules` folder

"Was a big mistake" says NodeJS creator, and
Deno fixes it (but here we are :face_with_rolling_eyes:)

The location of `node_modules` is expected to be relative to the location of the importing script.

----

### How Google solves it: patch `require`

Same strategy as "PnP", e.g. Yarn PnP.

:thumbsdown: not compatible. Many npm packages wrote their own `require` implementation.

----

### How rules_nodejs solves it: runtime "linker"

Similar to `npm link`: use symlinks to make monorepo libraries appear in the node_modules tree

:thumbsdown: Slow beginning of every NodeJS spawn
:thumbsdown: Links appear in source tree w/o sandbox
:thumbsdown: Bins don't work with `genrule`/`ctx.actions.run`
:thumbsdown: Not compatible with "persistent workers"

Note:
- rules_nodejs also has the Google "patch require" strategy but is now discouraged.
- doesn't make deps under the same tree with sources - 'rootDirs' troubles

----

### How rules_js solves it

:point_right: Linker is now just a standard Bazel target :point_left:

Node.js tools assume the working dir is a single tree of src/gen/node_modules: we can do that!

- "link" to `bazel-bin/node_modules/...`
- copy sources to `bazel-bin`
- actions first `cd bazel-out/[arch]/bin`

Note:
This requires `copy_to_bin` and a bit of care in custom rules.

---

# Code Examples

https://github.com/aspect-build/bazel-examples/tree/main/vue

1. `WORKSPACE`: read 3p npm package lockfile
2. Link npm packages
3. `BUILD`: declare dependencies on npm packages
4. Run tools from npm
5. Wrap tools in macros
6. Custom rules

---

# Roadmap

rules_js 1.0.0 is available now

Coming soon :tm:
- Missing RBE support due to bug with Bazel resolving symlinks
- Gazelle extension to generate `BUILD` files from srcs
- Bazel 6.0 package manager: bzlmod instead of `WORKSPACE`
https://blog.aspect.dev/bzlmod

---

### Thanks!

Q&A time

Or come to our office hours
calendly.com/aspect-build/rules_js-office-hours