Slides: https://hackmd.io/@aspect/rules_js
Note:
This talk has a bunch of links which you can come find in the slides.
Bazel: most scalable polyglot Build System.
JavaScript & TypeScript: most popular languages.
Note:
Note:
I Co-founded Aspect Development to make Bazel the industry-standard full-stack build system
More: https://www.aspect.dev/resources
Note:
More: https://www.aspect.dev/resources
Note:
Small, disconnected JS apps shouldn't use Bazel.
The build system recommended by your framework is well supported for small-to-medium scale.
JavaScript engine that runs outside the browser.
Typically used for running dev tools to
build and test JavaScript programs.
Note:
pnpm
Note:
rules_nodejs
Bazel rules forked from Google-internal
DeclarationInfo
rules_js is a layer on rules_nodejs
build_bazel_rules_nodejs
is replaced
ModernWeb Meetup: Layering in JS tooling
https://www.aspect.dev/resources
Note:
Like Gulp or Grunt, but way (way) better.
Let's use Bazel!
In five minutes
Load and evaluate all extensions, BUILD
files and macros that are needed for the build.
digraph {
rankdir="LR"
"sources" -> "dependency graph"
"external repos" -> "dependency graph"
}
Note:
bazel fetch [targets]
https://blog.aspect.dev/configuring-bazels-downloader
digraph {
compound=true
rankdir="LR"
graph [ fontname="Source Sans Pro", fontsize=20 ];
node [ fontname="Source Sans Pro", fontsize=18];
edge [ fontname="Source Sans Pro", fontsize=12 ];
subgraph cluster1 {
label="monorepo"
my_app -> my_lib
my_app -> my_comp
}
subgraph cluster2 {
label="otherrepo"
other_team_lib
}
subgraph cluster3 {
label="registry.npmjs.com"
request
react
express
}
my_lib -> request
my_comp -> react
my_app -> express
my_app -> other_team_lib
}
Note:
bazel query
bazel query --output=graph [targets]
digraph mygraph {
rankdir="LR"
node [shape=box];
"//examples/macro:node_modules/mocha/dir"
"//examples/macro:node_modules/mocha/dir" -> "//examples/macro:node_modules/mocha"
"//examples/macro:node_modules"
"//examples/macro:node_modules" -> "//examples/macro:node_modules/mocha"
"//examples/macro:node_modules" -> "//examples/macro:node_modules/mocha-junit-reporter"
"//examples/macro:node_modules" -> "//examples/macro:node_modules/mocha-multi-reporters"
"//examples/macro:node_modules/mocha"
"//examples/macro:node_modules/mocha-junit-reporter/dir"
"//examples/macro:node_modules/mocha-junit-reporter/dir" -> "//examples/macro:node_modules/mocha-junit-reporter"
"//examples/macro:test"
"//examples/macro:test" -> "//examples/macro:_test_srcs\n//examples/macro:test__entry_point"
"//examples/macro:test" -> "//examples/macro:node_modules/mocha-junit-reporter"
"//examples/macro:test" -> "//examples/macro:node_modules/mocha-multi-reporters"
"//examples/macro:node_modules/mocha-junit-reporter"
"//examples/macro:node_modules/mocha-multi-reporters/dir"
"//examples/macro:node_modules/mocha-multi-reporters/dir" -> "//examples/macro:node_modules/mocha-multi-reporters"
"//examples/macro:node_modules/mocha-multi-reporters"
"//examples/macro:_test_srcs\n//examples/macro:test__entry_point"
}
digraph {
rankdir="LR"
"dependency graph" -> "action graph" [label = "Bazel rules"]
}
Action: for a requested output, how to generate it from some inputs and tools
e.g. “if you need hello.js
, run swc
on hello.ts
”.
Requires predicting the outputs!
bazel aquery [targets]
digraph {
rankdir="LR"
"index.ts" -> "index.js" [label=swc]
"component.ts" -> "component.js" [label=tsc]
"index.js" -> "bundle.js" [label=esbuild]
"component.js" -> "bundle.js"
"styles.scss" -> "styles.css" [label=sass]
"styles.css" -> "serve" [label=express]
"bundle.js" -> "serve"
}
Execute a subset of the action graph by spawning subprocesses (e.g. node
)
digraph {
rankdir="LR"
"changed index.ts" [shape=note]
"changed index.ts" -> "index.js" [label=swc]
"component.ts" [shape=none]
"component.js" [shape=none]
"component.ts" -> "component.js"
"index.js" -> "bundle.js" [label=esbuild]
"component.js" -> "bundle.js"
"styles.scss" [shape=none]
"styles.scss" -> "styles.css"
"styles.css" [shape=none]
"styles.css" -> "serve" [label=express]
"bundle.js" -> "serve"
}
User requested certain targets be built.
Bazel is lazy and will only:
npm install
Install everything needed for the whole
package/workspace
Any build/test script can depend on all npm packages
Vendor the world: copy npm ecosystem sources into VCS
You could do it this way too.
Just wrap [npm|yarn] install
- install the world
Note:
@pnpm/lifecycle
to run hooks
node_modules
Note:
https://blog.aspect.dev/rulesjs-npm-benchmarks
Best case:
Note:
Mix of third-party and first-party deps in a tree of package.json files.
Google: single version policy
rules_nodejs: independent top-level dep installs
rules_js: supports pnpm workspaces!
Note:
NodeJS programs rely on a node_modules
folder
"Was a big mistake" says NodeJS creator, and
Deno fixes it (but here we are
The location of node_modules
is expected to be relative to the location of the importing script.
require
Same strategy as "PnP", e.g. Yarn PnP.
require
implementation.Similar to npm link
: use symlinks to make monorepo libraries appear in the node_modules tree
genrule
/ctx.actions.run
Note:
Node.js tools assume the working dir is a single tree of src/gen/node_modules: we can do that!
bazel-bin/node_modules/...
bazel-bin
cd bazel-out/[arch]/bin
Note:
This requires copy_to_bin
and a bit of care in custom rules.
Documentation and migration guide:
https://docs.aspect.build/rules_js
Copy the WORKSPACE
snippet from latest release.
https://github.com/aspect-build/rules_js/releases
Just run pnpm install
and check that your workflows work.
A few npm packages still have "hoisting bugs" where they don't declare correct dependencies and accidentally rely on npm or yarn-specific layout.
Note:
pnpm-lock.yaml
npm_translate_lock
converts to Bazel's format (Starlark).
WORKSPACE
load("@aspect_rules_js//npm:npm_import.bzl",
"npm_translate_lock")
npm_translate_lock(
name = "npm",
pnpm_lock = "//:pnpm-lock.yaml",
)
# Load the starlark version of the lockfile
load("@npm//:repositories.bzl", "npm_repositories")
npm_repositories()
Note:
$(bazel info output_base)/external/npm
to see what was generatedBUILD
(next to package.json
)
load("@npm//:defs.bzl", "js_link_all_packages")
js_link_all_packages()
Result of bazel build :all
is now
# the virtual store
bazel-bin/node_modules/.aspect_rules_js
# symlink into the virtual store
bazel-bin/node_modules/some_pkg
# If you used pnpm-workspace.yaml:
bazel-bin/packages/some_pkg/node_modules/some_dep
bazel build examples/...
First declare the package…
my-lib/BUILD
load("@aspect_rules_js//npm:defs.bzl", "npm_package")
npm_package(
name = "lib",
srcs = [
"index.js",
"package.json",
],
)
Note:
… then link to bazel-bin/node_modules
tree…
app/BUILD
load("@aspect_rules_js//npm:defs.bzl", "npm_link_package")
npm_link_package(
name = "node_modules/@mycorp/mylib",
src = "//examples/lib"
)
…then depend on it just like it came from npm!
app/BUILD
js_binary(
name = "my_app",
data = [
"//:node_modules/react-dom",
"//:node_modules/@mycorp/mylib",
],
entry_point = "index.js",
)
Note:
bin
entries from package.jsonbin
entrytsc
)There are also more advanced ways, see rules_js/examples
Note:
These options are easy, hard, harder, easy
bin
entries are provided for all packagesBUILD
load("@npm//typescript:package_json.bzl", typescript_bin = "bin")
typescript_bin.tsc(
name = "compile",
srcs = [
"fs.ts",
"tsconfig.json",
"//:node_modules/@types/node",
],
outs = ["fs.js"],
chdir = package_name(),
args = ["-p", "tsconfig.json"],
)
Note:
Each bin exposes three rules:
Use | With | To |
---|---|---|
foo |
bazel build |
produce outputs |
foo_binary |
bazel run |
side-effects |
foo_test |
bazel test |
assert exit 0 |
Use "component libraries" to get coarse granularity
digraph {
rankdir="LR"
vite [label=vite]
vite2 [label=vite]
"lib sources" [shape=none]
"lib sources" -> "vite"
"app sources" [shape=none]
"app sources" -> "vite2"
vite -> vite2
vite2 -> bundle
bundle [shape=none]
}
Pretty fast developer loop in
https://github.com/aspect-build/bazel-examples/tree/main/vue
load("@npm//vite:package_json.bzl", vite_bin = "bin")
load("@npm//vue-tsc:package_json.bzl", vue_tsc_bin = "bin")
vite_bin.vite(
name = "dist",
args = ["build"],
out_dirs = ["dist"],
srcs = [...],
)
vue_tsc_bin.vue_tsc_test(
name = "type-check",
args = ["--noEmit"],
data = [...],
)
vite_bin.vite_binary(
name = "vite",
data = [...]
)
Note:
Bazel macros are like preprocessor definitions.
Good way to give "syntax sugar",
compose a few rules, set defaults.
Indistinguishable from custom rules at use site
Example: mocha
test
def mocha_test(name, srcs, args = [], data = [], env = {}, **kwargs):
bin.mocha_test(
name = name,
args = [
"--reporter",
"mocha-multi-reporters",
"--reporter-options",
"configFile=$(location //examples/macro:mocha_reporters.json)",
native.package_name() + "/*test.js",
] + args,
data = data + srcs + [
"//examples/macro:mocha_reporters.json",
"//examples/macro:node_modules/mocha-multi-reporters",
"//examples/macro:node_modules/mocha-junit-reporter",
],
env = dict(env, **{
# Add environment variable so that mocha writes its test xml
# to the location Bazel expects.
"MOCHA_FILE": "$$XML_OUTPUT_FILE",
}),
)
https://github.com/aspect-build/rules_js/blob/main/examples/macro/mocha.bzl
Note:
bazel query --output=build
Harder and not recommended for most users.
Start from
https://bazel.build/rules/rules-tutorial
and use
https://github.com/bazel-contrib/rules-template
From https://github.com/aspect-build:
… and many more by other vendors
Catalog coming soon at https://bazel-contrib.github.io/SIG-rules-authors/
ts_project
No more rootDirs
in tsconfig.json
BUILD
load("@bazel_skylib//rules:write_file.bzl", "write_file")
# Create a test fixture that is a non-trivial sized TypeScript program
write_file(
name = "gen_ts",
out = "big.ts",
content = [
"export const a{0}: number = {0}".format(x)
for x in range(100000)
],
)
BUILD
load("@aspect_rules_ts//ts:defs.bzl", "ts_project")
ts_project(
name = "tsc",
srcs = ["big.ts"],
declaration = True,
source_map = True,
)
ts_project with custom transpiler
BUILD
load("@aspect_rules_swc//swc:defs.bzl", "swc_transpiler")
ts_project(
name = "swc",
srcs = ["big.ts"],
out_dir = "build-swc",
transpiler = partial.make(
swc_transpiler,
args = ["--env-name=test"],
swcrc = ".swcrc",
),
)
Benchmarks: ts_project w/ SWC
https://blog.aspect.dev/rules-ts-benchmarks
Transpile-only use case on large project
bazel build :devserver
Note:
time for a first, non-incremental build
Sophisticated teams can assemble their own toolchain.
Create an entire JS build system just by composing existing tools in a macro!
Note:
Example: an entire custom build system called
"differential loading":
def differential_loading(name, entry_point, srcs):
"Common workflow to serve TypeScript to modern browsers"
ts_project(
name = name + "_lib",
srcs = srcs,
)
rollup_bundle(
name = name + "_chunks",
deps = [name + "_lib"],
sourcemap = "inline",
config_file = "//:rollup.config.js",
entry_points = {
entry_point: "index",
},
output_dir = True,
)
# For older browsers, we'll transform the output chunks to es5 + systemjs loader
bin.babel(
name = name + "_chunks_es5",
srcs = [
name + "_chunks",
"es5.babelrc",
":node_modules/@babel/preset-env",
],
output_dir = True,
args = [
"../../../$(execpath %s_chunks)" % name,
"--config-file",
"../../../$(execpath es5.babelrc)",
"--out-dir",
"$(@D)",
],
)
terser_minified(
name = name + "_chunks_es5.min",
src = name + "_chunks_es5",
)
rules_js 1.0.0 is available now
Coming soon
BUILD
files from srcsWORKSPACE
These slides: https://hackmd.io/@aspect/rules_js
Thanks conference organizers and everyone who helped launch rules_js.
Come work with us on OSS! http://aspect.dev/careers
Paid support and consulting: http://aspect.dev
Our projects: github.com/aspect-build