What is Aspect

I Co-founded Aspect Development to make Bazel the industry-standard full-stack build system

• https://aspect.dev - Support and consulting to help you adopt Bazel
• https://aspect.build - Products making Bazel easier to use
• https://github.com/aspect-build - rules_js is part of our Bazel rules ecosystem

Build systems:

Matrix / Hub-and-Spoke

The JS ecosystem took a wrong turn

• Grunt and Gulp fell out of favor
• Instead, each tool became a Build System
• Now each tool needs a plugin for each language

ModernWeb Meetup: Layering in JS tooling
https://www.aspect.dev/resources last one

Like Gulp or Grunt, but way (way) better.

Let's use Bazel!

What is NodeJS

JavaScript engine that runs outside the browser.

Typically used for running dev tools to
build and test JavaScript programs.

Deno is an up-and-coming alternative, see rules_deno

What is rules_nodejs

Bazel rules forked from Google-internal

• toolchain to run hermetic NodeJS interpreter
• shared Bazel interfaces ("Providers") like TypeScript DeclarationInfo

What is rules_js

rules_js is a layer on top of rules_nodejs

About rules_js: My talk from Bazel eXchange 2022

Slides: https://hackmd.io/@aspect/rules_js
Video: https://aspect.dev/resources (first video)

What is pnpm

• "Fast, disk space efficient package manager": https://pnpm.io/
• Works with nearly the whole ecosystem
• Used by the https://rushjs.io/ monorepo JS-only build tool
• Happens to fit perfectly with Bazel semantics!
• Used by rules_js

JS: Client and Server

Client runtime: Browser
Server runtime: Node.js
Both use the same virtual machine (V8), stdlib differs

• rules_nodejs supports both, though confusingly named
• rules_js also supports both

Modules

digraph G {

  rankdir="TB"
  fontname="courier"
  
  subgraph cluster_bazelbuild {
      label = "github.com/bazelbuild"

      subgraph cluster_0 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        nodejs_binary
        nodejs_star [label="nodejs_*"]
        yarn_install
        label = "build_bazel_rules_nodejs";
      }

      subgraph cluster_1 {
        
        toolchain
        label = "rules_nodejs";
        color=blue
      }
  }

  subgraph cluster_npm {
      label = "npm"
      subgraph cluster_2 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        ts [label="ts_project"]

        label = "@bazel/typescript";
      }
    subgraph cluster_6 {
        style=filled;
        color=lightgrey;
        node [style=filled,color=white];
        other [label="..."]

        label = "@bazel/*";
      }
      
  }
  
  subgraph cluster_aspect {
      label = "github.com/aspect-build"
      subgraph cluster_3 {
        js_binary
        js_star [label="js_*"]
        npm_star [label="npm_*"]
        label = "rules_js";
        color=blue
      }

      subgraph cluster_4 {
        ts_project
        label = "rules_ts";
        color=blue
     }
      subgraph cluster_7 {
        other_rules [label="..."]
        label = "rules_*"
        color=blue
     }
      subgraph cluster_8 {
        copy_to_bin
        label = "bazel_lib"
        color=blue
     }

  }
  other_rules -> js_binary
  ts -> nodejs_binary
  nodejs_binary -> toolchain
  js_binary -> toolchain
  ts_project -> js_binary
  js_binary -> copy_to_bin
}

NEWS: Shaded boxes are no longer maintained!

How npm/yarn solve it

npm install

Install everything needed for the whole
package/workspace

Any build/test script can depend on all npm packages

How Google solves it

Vendor the world: copy npm ecosystem sources into VCS

• Never fetches from the internet
• Never runs any package installation

You could do it this way too.

How rules_nodejs solved it

Just wrap [npm|yarn] install - install the world

Guaranteed slow when repo rule invalidates
Extra bad when "eager fetching" npm deps

rules_js: ideal solution

Port pnpm to Starlark

• re-use pnpm's resolver (via lockfile)
• fetch with Bazel's downloader
• unpack tarballs with Bazel
• re-use @pnpm/lifecycle to run hooks
  • these are actions - can be remote cached
• link node_modules

https://blog.aspect.dev/rulesjs-npm-benchmarks

Best case:

• BUILD file declares fine-grained deps
• build only depends on one library
• we only fetch/install one library!

How it works in npm

NodeJS programs rely on a node_modules folder

"Was a big mistake" says NodeJS creator, and
Deno fixes it (but here we are )

The location of node_modules is expected to be relative to the location of the importing script.

How Google solves it: patch require

Same strategy as "PnP", e.g. Yarn PnP.

not compatible. Many npm packages wrote their own require implementation.

How rules_nodejs solves it: runtime "linker"

Similar to npm link: use symlinks to make monorepo libraries appear in the node_modules tree

Slow beginning of every NodeJS spawn
Links appear in source tree w/o sandbox
Bins don't work with genrule/ctx.actions.run
Not compatible with "persistent workers"

How rules_js solves it

Linker is now just a standard Bazel target

Node.js tools assume the working dir is a single tree of src/gen/node_modules: we can do that!

• "link" to bazel-bin/node_modules/...
• copy sources to bazel-bin
• actions first cd bazel-out/[arch]/bin