# Resource batches and the developer -> framework -> bundler -> server pipeline

This document proposes conventions for the use of the [resource batch file format](https://gist.github.com/littledan/e01801001c277b0be03b1ca54788505e#resource-batch-format) `.rba` to package up sites through the pipeline from development to building with frameworks, to bundlers and other optimization tools, to serving in web servers. This pipeline may result in resource batches being ultimately served over HTTP, or other serving strategies (including both current bunder output and simple individual subresources).

## The resource batch file format

Resource batches represent a mapping from paths to metadata + payloads. In the context of web tooling/building/serving, the paths are interpreted as URLs, metadata as HTTP headers, and payloads as response bodies.

Compared to `.tar` or `.zip` files, resource batches omit a lot of legacy cruft, while representng all the information we need for HTTP responses.

Although resource batches can't represent everything that a site will serve (for example, they omit server-side dynamically generated/personalized content), a large component of websites is static, and can be represented by a resource batch.

## Response headers and content negotiation

The `metadata` field of the `resource`s in a resource batch is interpreted to contain HTTP headers. When preloading resource batches over HTTP, only the `Content-Type` header is permitted, but within tools, it can be useful to include more headers:
- There are many more HTTP headers which are important for serving, especially for the main HTML document. Servers can use resource batches to contain all of the headers necessary for a response, even though this means that the response is actually served to the client outside of a resource batch.
- For content negotiation (including language, file format and more), the resource batch on the server side can contain multiple responses for the same URL. It is the server's job to decide which one to serve, based on the headers from the client.

The use of a common resource batch file format throughout the site build stack allows both of these kinds of information to be propagated through the stack, avoiding complicated configuration problems today.

## Resource batch loading API

To preload a resource batch from HTML, use a [`<link type=preloadbatch>`](https://gist.github.com/littledan/e01801001c277b0be03b1ca54788505e#link-relbatchpreload-hreffoorba-resourcesbarjs-bazcss-onloadalertdone) tag. Or, to load resources dynamically, use the [`preloadBatch`](https://gist.github.com/littledan/e01801001c277b0be03b1ca54788505e#windowpreloadbatchnew-urlfoorba-importmetaurl-barjs-bazcss) function.

Both of these APIs require a URL to load the resource batch from, plus `resources` list of URLs. Ideally, this list would include all nested dependencies. However, it can be difficult to calculate this list locally, or even say what the final bundle will be called. This proposal includes conventions for tools to output code which

## Framework/developer code

When developers write source code, they do so in a high level style. For example, JS modules are directly imported with `import` statements and dynamic `import()`. This is later expected to be transformed into particular chunks.

Under the conventions established in this document, frameworks can output the following constructs and expect bundlers to handle the details:
- `<link rel=resourcebatch>`
- `import("foo.js")`
- `preloadBatch(null, [new URL("resource.xyz", import.meta.url)])`

Frameworks are encouraged to output their static contents in resource batches, for later processing by bundlers and serving.

There would be a simple tool for web developers to assembly a resource batch out of a directory of files, as well. This could be used with the same conventions to be passed to a bundler.

## Bundler transformations

In a resource batch world, a bundler converts one resource batch (from the developer/framework) into another (to be used directly by the server). This document establishes the following conventions for bundlers:
- When the bundler sees `<link rel=resourcebatch>`, its job is to fill in the `href=` and `resources=` list. It does this based on its analysis of which resources are used by the enclosing HTML file.
- When a bundler sees `import("foo.js")`, it replaces it with `preloadBatch("import.rba", [new URL("foo.js", import.meta.url), "dependency.js", ...]).then(() => import("foo.js"))` with all of the dependencies of `foo.js` statically detected and inserted.
- When a bundler sees `preloadBatch(null, [new URL("resource.xyz", import.meta.url)])`, it detects all of the dependencies of `resource.xyz` statically and adds them to the resource list, as well as filling in the location of the resource batch.

Bundlers are encouraged (paradoxically) add resources to the resource batch to contain the output of existing bundling techniques. These will never be served *within* a resource batch, since they will never be in the `resources=` list. They are placed in the resource patch that the bundler outputs in order to communicate with the server.

## Server expectations

A server, given a resource batch `index.rba`, is expected to serve it in the following way:
- Each URL within the resource batch can be fetched from the server, with the relative URLs in the index being resolved relative to the enclosing directory where `index.rba` is present.
- If a URL is present multiple times, then the server can choose what to respond with based on its own content negotiation algorithm.
- If `index.rba` itself is fetched, then it is served as follows:
- The request must come with a header indicating the subset of resources to be served. If that header is missing, respond with an error.
- Omit resources which are not included in the subset requested in the header.
- Based on the other headers, perform content negotiation to choose appropriate responses for each resource with duplicate URLs (in terms of Content-Type, language, etc).
- The Content-Type of a resource batch is `application/resourcebatch`.
- Serve the result compressed if possible.