# rules_ts benchmarks

###### tags: `blog`

*--DRAFT--*

Aspect's [rules_ts](https://github.com/aspect-build/rules_ts) is a port of [rules_nodejs](https://github.com/bazelbuild/rules_nodejs)'s [@bazel/typescript](https://www.npmjs.com/package/@bazel/typescript) package that provides a `ts_project` rule layered on top of [rules_js](https://github.com/aspect-build/rules_js), Aspect's new high-performance Bazel rule set for Javascript, designed from the group up with performance in mind.

The `ts_project` rule from [rules_ts](https://github.com/aspect-build/rules_ts) has the same API as its predecessor from [@bazel/typescript](https://www.npmjs.com/package/@bazel/typescript), but it leaves the gate with performance improvements that were not possible under [rules_nodejs](https://github.com/bazelbuild/rules_nodejs).

> To learn about how rules_js also makes npm dependencies fast with Bazel check out our [rules_js npm benchmarks](https://blog.aspect.dev/rulesjs-npm-benchmarks)

In this post, we'll compare build times for `ts_project` from [rules_ts](https://github.com/aspect-build/rules_ts) against its predecessor, `ts_project` from [@bazel/typescript](https://www.npmjs.com/package/@bazel/typescript), as well as against `ts_library` from [@bazel/concatjs](https://www.npmjs.com/package/@bazel/concatjs) (the original TypeScript rule from Google), and against the vanilla TypeScript compiler, `tsc`.

## Methology

These benchmarks wre run against a [generated TypeScript code base](https://github.com/aspect-build/bazel-benchmarks/tree/main/rules_ts) of 5 features, 10 modules per feature, 10 components per module and 1001 lines of code per component. This makes for a total of 555 TypeScript files containing 500995 lines of TypeScript code in aggregate.

For the Bazel build, each module maps to one Bazel target for a total of 55 `ts_project/ts_library` targets.

## Configuration

These benchmarks were run on a,

MacBook Pro (16-inch 2019)
2.4 GHz 8-Core Intel Core i9
64 GB 2667 MHz DDR4
Running macOS Monterey 12.3.1

Versions of typescript and rule sets used were,

- [TypeScript](https://www.typescriptlang.org/) 4.6.3
- [build_bazel_rules_nodejs](https://github.com/bazelbuild/rules_nodejs) 5.5.0
- [@bazel/typescript](https://www.npmjs.com/package/@bazel/typescript) 5.5.0
- [@bazel/concatjs](https://www.npmjs.com/package/@bazel/concatjs) 5.5.0
- [aspect_rules_ts](https://github.com/aspect-build/rules_ts) 0.7.0
- [aspect_rules_swc](https://github.com/aspect-build/rules_swc) [PR#57](https://github.com/aspect-build/rules_swc/pull/57) (a soon to be landed performance enhancement which uses a new pure rust CLI for [swc](https://swc.rs/))

## Full builds vs. "devserver" builds

In these benchmarks we measure two different scenarios:

1) A full clean build (`bazel build ...`) followed by an incremental `bazel build ...` after making a change to a leaf TypeScript file.
1) A clean "devserver" build (`bazel build :devserver`), which emulates a typical developer workflow of building while running a tool such as a devserver, followed by an incremental `bazel build :devserver` after making a change to a leaf TypeScript file.

The "devserver" scenario is an important measure that emulates the typical local development workflow of coding while running tools such as a devserver or a test runner such as jest. These tools are often run in watch mode while making changes to source code. The faster build times are on changes the shorter the round-trip-time is to get feedback on those changes.

Ideal build times to maximize developer productivity are less than 1 second on changes to leaf nodes and less than 10 seconds on changes that affect large parts of the graph. Ideally these ideal times are sustained even on large projects.

## ts_project vs. ts_library

`ts_project` was originally developed in [rules_nodejs](https://github.com/bazelbuild/rules_nodejs) as an alternative to `ts_library` to provide a cleaner API better suited for the many ways TypeScript is used outside of Google. While the API was better suited for the wild, it could not compete with `ts_library`, a heavily optimized and deeply integrated wrapper around the TypeScript compiler, on performance.

The new `ts_project` from [rules_ts](https://github.com/aspect-build/rules_ts) has significantly reduced the performance gap with `ts_library` by adding first-class support for Bazel workers.

> [rules_js](https://github.com/aspect-build/rules_js), which [rules_ts](https://github.com/aspect-build/rules_ts) is layered on, has made first-class worker support in [rules_ts](https://github.com/aspect-build/rules_ts) possible by doing away with the dynamic runtime node_modules linking that [rules_nodejs](https://github.com/bazelbuild/rules_nodejs) uses.

While `ts_library` can still slightly outpace `ts_project` with worker mode in full clean build times, in the "devserver" scenario `ts_project` has an significant advantage over `ts_library` by allowing you to configure a separate tool for transpiliation.

In these benchmarks, we'll measure `ts_project` configured with [swc](https://swc.rs/) as the transpiler. [swc](https://swc.rs/) is an order of magnitude faster that TypeScript for pure transpilation but it does not type check, so TypeScript is still used for type checking in this split configuration.

The split configuration also removes type checking from the build graph for devserver and test targets, so only transpiliation is needed to build them, reducing the round-trip-time on changes when running such targets by an order of magnitude. Type-checking is handled in separate targets that can be run explitly or with the catch-all `bazel build ...`.

## Results

Without further ado, here are the results of the benchmarks.

### Full clean builds

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {" ": "ts_project", "Time (seconds)": 53.64},
      {" ": "ts_project (worker mode)", "Time (seconds)": 44.86},
      {" ": "ts_project (swc)", "Time (seconds)": 48.65},
      {" ": "ts_project (worker mode & swc)", "Time (seconds)": 38.67},
      {" ": "rules_nodejs ts_project", "Time (seconds)": 62.37},
      {" ": "rules_nodejs ts_project (swc)", "Time (seconds)": 55.86},
      {" ": "ts_library", "Time (seconds)": 34.59},
      {" ": "tsc", "Time (seconds)": 90.92}
    ]
  },
  "mark": "bar",
  "encoding": {
    "y": {
        "field": " ",
        "type": "ordinal",
        "sort": false
    },
    "x": {
        "field": "Time (seconds)",
        "type": "quantitative"
    }
  }
}

`ts_library` leads the pack for full (transpilation & type checking) clean build times. It has been heavily optimized inside Google and is integrated deeply with TypeScript compiler internals. The `ts_project` API, however, is not well suited for the many ways that TypeScript projects are configured outside of Google and it does not integrate well with many other rules and tools in the frontend ecosystem.

[rules_ts](https://github.com/aspect-build/rules_ts)'s `ts_project` is a competetive runner up. It makes significant performance gains over its predecessor from [@bazel/typescript](https://www.npmjs.com/package/@bazel/typescript) by adding first-class support for worker mode.

> This is only our initial pass at worker mode for `ts_project` and we believe we can optimize it further in the future by taking advantage of Bazel features such as [multiplexed workers](https://docs.bazel.build/versions/main/multiplex-worker.html). Stay tuned for future performance improvements in this rule.

### Incremental full builds

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {" ": "ts_project", "Time (seconds)": 3.90},
      {" ": "ts_project (worker mode)", "Time (seconds)": 3.82},
      {" ": "ts_project (swc)", "Time (seconds)": 3.49},
      {" ": "ts_project (worker mode & swc)", "Time (seconds)": 3.16},
      {" ": "rules_nodejs ts_project", "Time (seconds)": 4.30},
      {" ": "rules_nodejs ts_project (swc)", "Time (seconds)": 3.72},
      {" ": "ts_library", "Time (seconds)": 3.09},
      {" ": "tsc", "Time (seconds)": 7.50}
    ]
  },
  "mark": "bar",
  "encoding": {
    "y": {
        "field": " ",
        "type": "ordinal",
        "sort": false
    },
    "x": {
        "field": "Time (seconds)",
        "type": "quantitative"
    }
  }
}

All the Bazel rules measured are relatively close in incremental full build times with `ts_library` taking the lead and `ts_project` from [rules_ts](https://github.com/aspect-build/rules_ts) with worker-mode and swc for transpilation runner up. In this benchmark, vanilla `tsc` is slowest but in smaller projects it can be quite fast.

### Clean "devserver" builds

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {" ": "ts_project", "Time (seconds)": 70.47},
      {" ": "ts_project (worker mode)", "Time (seconds)": 57.82},
      {" ": "ts_project (swc)", "Time (seconds)": 3.14},
      {" ": "ts_project (worker mode & swc)", "Time (seconds)": 3.11},
      {" ": "rules_nodejs ts_project", "Time (seconds)": 81.23},
      {" ": "rules_nodejs ts_project (swc)", "Time (seconds)": 3.38},
      {" ": "ts_library", "Time (seconds)": 43.62},
      {" ": "tsc", "Time (seconds)": 94.86}
    ]
  },
  "mark": "bar",
  "encoding": {
    "y": {
        "field": " ",
        "type": "ordinal",
        "sort": false
    },
    "x": {
        "field": "Time (seconds)",
        "type": "quantitative"
    }
  }
}

Clean "devserver" builds is where `ts_project` configured with [swc](https://swc.rs/) as the transpiler really stands out and is an order of magnitude faster than the rest. The 500,000+ lines TypeScript code in this benchmark take even the heavily optimized `ts_library` more than 40 seconds to build while [swc](https://swc.rs/) can transpile the same in 3 seconds flat.

[swc](https://swc.rs/) is fast enough to spawn to be configured as one target per TypeScript file, which means that with remote execution the 555 `.ts` file targets in this benchmark could be distributed across 555 remote executors and transpile near instantaneously. `ts_library`, on the other hand, does not split into one target per file so it could parallelize into only 55 targets with remote execution in this benchmark.

### Incremental "devserver" builds

{
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {" ": "ts_project", "Time (seconds)": 5.22},
      {" ": "ts_project (worker mode)", "Time (seconds)": 5.04},
      {" ": "ts_project (swc)", "Time (seconds)": 0.32},
      {" ": "ts_project (worker mode & swc)", "Time (seconds)": 0.34},
      {" ": "rules_nodejs ts_project", "Time (seconds)": 5.58},
      {" ": "rules_nodejs ts_project (swc)", "Time (seconds)": 0.33},
      {" ": "ts_library", "Time (seconds)": 4.44},
      {" ": "tsc", "Time (seconds)": 7.50}
    ]
  },
  "mark": "bar",
  "encoding": {
    "y": {
        "field": " ",
        "type": "ordinal",
        "sort": false
    },
    "x": {
        "field": "Time (seconds)",
        "type": "quantitative"
    }
  }
}

On incremental "devserver" builds, where `ts_library` and even `ts_project` without [swc](https://swc.rs/) are fairly fast, `ts_project` configured with [swc](https://swc.rs/) for transpilation is still an order of magnitude faster.

## The bottom line

With [rules_ts](https://github.com/aspect-build/rules_ts), front-end developers can finally get the near instant round-trip-times they are used to with optimized front-end build systems such as [Vite](https://vitejs.dev/) with Bazel.

We feel this improvement, along with [fast npm dependency management](https://blog.aspect.dev/rulesjs-npm-benchmarks) will get more web developers on board with building with Bazel.

Bazel & JavaScript is about to get a whole lot better!