# :herb: Bazel and JavaScript
## Bazel community update
## August 18 2022
---
Bazel: most scalable polyglot Build System.
![top-languages-over-the-years](https://hackmd.io/_uploads/H1y-RdzK9.png)
JavaScript & TypeScript: most popular languages.
Note:
- Bazel works with all languages, let's make it good with the most popular ones.
- image from https://octoverse.github.com/#top-languages-over-the-years
---
## Who is Alex Eagle
<img src="https://avatars0.githubusercontent.com/u/47395?s=460&v=4" style="border:2px solid grey; border-radius: 200px;" width=100/>
- Worked at Google on DevInfra 2008-2020
- Bazel most of that time: TL for Google's CI, build/test results UI, Angular CLI
- Started Rules Authors SIG, maintained rules_{nodejs,python,docker}
- twitter.com/jakeherringbone
----
## What is Aspect
I Co-founded Aspect Development to make Bazel the industry-standard full-stack build system
- <https://aspect.dev> - Support and consulting to help you adopt Bazel
- <https://aspect.build> - Products making Bazel easier to use
- <https://github.com/aspect-build> - rules_js is part of our Bazel rules ecosystem
----
## Build systems:
### Matrix / Hub-and-Spoke
:construction: The JS ecosystem took a wrong turn
- Grunt and Gulp fell out of favor
- Instead, each tool became a Build System
- Now each tool needs a plugin for each language
----
![](https://hackmd.io/_uploads/BJWSKWKK5.png)
ModernWeb Meetup: Layering in JS tooling
https://www.aspect.dev/resources :point_right: last one
Note:
- I drew this live in my talk :laughing:
- When each column is a tool like webpack or jest
- each row is a language like typescript or sass
- MxN support matrix - so many plugin authors to trust
- A new framework like Qwik has to support every CSS preprocessor??
----
![](https://hackmd.io/_uploads/SJd_F-ttc.png)
Like Gulp or Grunt, but way (way) better.
Let's use Bazel!
----
## What is NodeJS
JavaScript engine that runs outside the browser.
Typically used for running dev tools to
build and test JavaScript programs.
Deno is an up-and-coming alternative, see rules_deno
Note:
- just a runtime, requires a package manager
----
## What is `rules_nodejs`
Bazel rules forked from Google-internal
- toolchain to run hermetic NodeJS interpreter
- shared Bazel interfaces ("Providers") like TypeScript `DeclarationInfo`
----
## What is `rules_js`
rules_js is a layer on top of `rules_nodejs`
About rules_js: My talk from Bazel eXchange 2022
Slides: https://hackmd.io/@aspect/rules_js
Video: https://aspect.dev/resources (first video)
----
## What is `pnpm`
- "Fast, disk space efficient package manager": <https://pnpm.io/>
- Works with nearly the whole ecosystem
- Used by the <https://rushjs.io/> monorepo JS-only build tool
- Happens to fit perfectly with Bazel semantics!
- Used by `rules_js`
Note:
- Symlinks to a "virtual store" of fetched packages rather than nested layout
- Their lockfile happens to present 100% of the info needed to fetch AND link the node_modules tree
----
## JS: Client and Server
Client runtime: Browser
Server runtime: Node.js
Both use the same virtual machine (V8), stdlib differs
- `rules_nodejs` supports both, though confusingly named
- `rules_js` also supports both
----
Modules
```graphviz=
digraph G {
rankdir="TB"
fontname="courier"
subgraph cluster_bazelbuild {
label = "github.com/bazelbuild"
subgraph cluster_0 {
style=filled;
color=lightgrey;
node [style=filled,color=white];
nodejs_binary
nodejs_star [label="nodejs_*"]
yarn_install
label = "build_bazel_rules_nodejs";
}
subgraph cluster_1 {
toolchain
label = "rules_nodejs";
color=blue
}
}
subgraph cluster_npm {
label = "npm"
subgraph cluster_2 {
style=filled;
color=lightgrey;
node [style=filled,color=white];
ts [label="ts_project"]
label = "@bazel/typescript";
}
subgraph cluster_6 {
style=filled;
color=lightgrey;
node [style=filled,color=white];
other [label="..."]
label = "@bazel/*";
}
}
subgraph cluster_aspect {
label = "github.com/aspect-build"
subgraph cluster_3 {
js_binary
js_star [label="js_*"]
npm_star [label="npm_*"]
label = "rules_js";
color=blue
}
subgraph cluster_4 {
ts_project
label = "rules_ts";
color=blue
}
subgraph cluster_7 {
other_rules [label="..."]
label = "rules_*"
color=blue
}
subgraph cluster_8 {
copy_to_bin
label = "bazel_lib"
color=blue
}
}
other_rules -> js_binary
ts -> nodejs_binary
nodejs_binary -> toolchain
js_binary -> toolchain
ts_project -> js_binary
js_binary -> copy_to_bin
}
```
NEWS: Shaded boxes are no longer maintained!
---
# Fetch and install npm packages
----
### How npm/yarn solve it
`npm install`
Install everything needed for the whole
package/workspace
Any build/test script can depend on all npm packages :face_with_rolling_eyes:
----
### How Google solves it
Vendor the world: copy npm ecosystem sources into VCS
- Never fetches from the internet
- Never runs any package installation
You *could* do it this way too. :face_with_one_eyebrow_raised:
----
### How rules_nodejs solved it
Just wrap `[npm|yarn] install` - install the world
:thumbsdown: Guaranteed slow when repo rule invalidates
:thumbsdown: Extra bad when "eager fetching" npm deps
Note:
- https://blog.aspect.dev/avoid-eager-fetches
----
### rules_js: ideal solution
:point_right: Port pnpm to Starlark :point_left:
- re-use pnpm's resolver (via lockfile)
- fetch with Bazel's downloader
- unpack tarballs with Bazel
- re-use `@pnpm/lifecycle` to run hooks
- these are actions - can be remote cached
- link `node_modules`
Note:
- More later about linking the node_modules
----
https://blog.aspect.dev/rulesjs-npm-benchmarks
Best case:
- BUILD file declares fine-grained deps
- build only depends on one library
- we only fetch/install one library!
----
![](https://hackmd.io/_uploads/HkEK7nCF5.png)
Note:
- This is possible because BUILD files have finer-grained graph than package.json does
---
# Resolving npm dependencies at runtime
----
### How it works in npm
NodeJS programs rely on a `node_modules` folder
"Was a big mistake" says NodeJS creator, and
Deno fixes it (but here we are :face_with_rolling_eyes:)
The location of `node_modules` is expected to be relative to the location of the importing script.
----
### How Google solves it: patch `require`
Same strategy as "PnP", e.g. Yarn PnP.
:thumbsdown: not compatible. Many npm packages wrote their own `require` implementation.
----
### How rules_nodejs solves it: runtime "linker"
Similar to `npm link`: use symlinks to make monorepo libraries appear in the node_modules tree
:thumbsdown: Slow beginning of every NodeJS spawn
:thumbsdown: Links appear in source tree w/o sandbox
:thumbsdown: Bins don't work with `genrule`/`ctx.actions.run`
:thumbsdown: Not compatible with "persistent workers"
Note:
- rules_nodejs also has the Google "patch require" strategy but is now discouraged.
- doesn't make deps under the same tree with sources - 'rootDirs' troubles
----
### How rules_js solves it
:point_right: Linker is now just a standard Bazel target :point_left:
Node.js tools assume the working dir is a single tree of src/gen/node_modules: we can do that!
- "link" to `bazel-bin/node_modules/...`
- copy sources to `bazel-bin`
- actions first `cd bazel-out/[arch]/bin`
Note:
This requires `copy_to_bin` and a bit of care in custom rules.
---
# Code Examples
https://github.com/aspect-build/bazel-examples/tree/main/vue
1. `WORKSPACE`: read 3p npm package lockfile
2. Link npm packages
3. `BUILD`: declare dependencies on npm packages
4. Run tools from npm
5. Wrap tools in macros
6. Custom rules
---
# Roadmap
rules_js 1.0.0 is available now
Coming soon :tm:
- Missing RBE support due to bug with Bazel resolving symlinks
- Gazelle extension to generate `BUILD` files from srcs
- Bazel 6.0 package manager: bzlmod instead of `WORKSPACE`
https://blog.aspect.dev/bzlmod
---
### Thanks!
Q&A time
Or come to our office hours
calendly.com/aspect-build/rules_js-office-hours
{"metaMigratedAt":"2023-06-17T07:04:04.719Z","metaMigratedFrom":"YAML","title":"Bazel and JS","breaks":"true","description":"Bazel community update, August 18 2022","slideOptions":"{\"theme\":\"white\"}","contributors":"[{\"id\":\"20586f0c-3c64-4285-895e-d8d3820edbd0\",\"add\":26057,\"del\":17496}]"}