# `npm query` Command & Dependency Selector Syntax

## Summary

Introduce a new `npm query` commmand which exposes a new dependency selector syntax (informed by & respecting many aspects of the [CSS Selectors 4 Spec](https://dev.w3.org/csswg/selectors4/#relational)).

## Motivation

- Standardize the shape of, & querying of, dependency graphs with a robust object model, metadata & selector syntax
- Leverage existing, known language syntax & operators from CSS to make disperate package information broadly accessible
- Unlock the ability to anwser complex, multi-faceted questions about dependencies, their relationships & associative metadata
- Consolidate redundant logic of similar query commands in `npm` (ex. `npm fund`, `npm ls`, `npm outdated`, `npm audit` ...)

## Detailed Explanation

This RFC's spec & implementation should closely mimic the capabilities of existing CSS Selector specifications. Notably, we'll introduce limited net-new classes, states & syntax to ensure the widest adoption & understanding of paradigms. When deviating, we'll explicitely state why & how.

## Implementation

- `Arborist`'s `Node` Class will have a new `.querySelectorAll()` method
- this method will return a filtered, flattened dependency Arborist `Node` list based on a valid query selector
- Introduce a new command, `npm query`, which will take a dependency selector & output a flattened dependency Node list (output is in `json` by default, but configurable)

### Dependency Selector Syntax

#### Overview:
- there is no "type" or "tag" selectors (ex. `div, h1, a`) as a dependency is the only type of `Node` to query
- `dependency` & `link` types
- the term "dependencies" is in reference to any `Node` found in the `idealTree` returned by `Arborist`

#### Selectors

- `*` universal selector
- `#<name>` dependency selector (equivalent to `[name="..."]`)
- `#<name>@<version>` (equivalent to `[name=<name>]:semver(<version>)`)
- `,` selector list delimiter
- `.` class selector
- `:` pseudo class selector
- `>` direct decendent/child selector
- `~` sibling selector

#### [Attribute Selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/Attribute_selectors)
- `[]` attribute selector (ie. existence of attribute)
- `[attribute=value]` attribute value is equivalant...
- `[attribute~=value]` attribute value contains word...
- `[attribute*=value]` attribute value contains string...
- `[attribute|=value]` attribute value is equal to or starts with...
- `[attribute^=value]` attribute value begins with...
- `[attribute$=value]` attribute value ends with...

#### Pseudo Selectors
- [`:not(<selector>)`](https://developer.mozilla.org/en-US/docs/Web/CSS/:not)
- [`:has(<selector>)`](https://developer.mozilla.org/en-US/docs/Web/CSS/:has)
- [`:where(<selector list>)`](https://developer.mozilla.org/en-US/docs/Web/CSS/:where)
- [`:is(<selector list>)`](https://developer.mozilla.org/en-US/docs/Web/CSS/:is)
- [`:root`](https://developer.mozilla.org/en-US/docs/Web/CSS/:root) matches the root node/dependency
- [`:scope`](https://developer.mozilla.org/en-US/docs/Web/CSS/:scope) matches node/dependency it was queried against
- [`:empty`](https://developer.mozilla.org/en-US/docs/Web/CSS/:empty) when a dependency has no dependencies
- [`:private`](https://docs.npmjs.com/cli/v8/configuring-npm/package-json#private) when a dependency is private
- `:link` when a dependency is linked
- `:deduped` when a dependency has been deduped
- `:override` when a dependency is an override
- `:extraneous` when a dependency could be but is not deduped
- `:outdated` when a dependency is not `latest`
- `:vulnerable` when a dependency has a `CVE`
- `:semver(<spec>)` matching a valid [`node-semver`](https://github.com/npm/node-semver) spec
- `:path(<path>)` [glob](https://www.npmjs.com/package/glob) matching based on dependencies path relative to the project
- `:realpath(<path>)` [glob](https://www.npmjs.com/package/glob) matching based on dependencies realpath
- `:type(<type>)` [based on currently recognized types](https://github.com/npm/npm-package-arg#result-object)

#### Generic Pseudo Selectors:

A standardized pseudo selector pattern will be used for `Object`s, `Array`s or `Arrays` of `Object`s accessible via `Arborist`'s `Node.package` metadata. Pseudo classes generated in this way will allow for iterative attribute selection.

`Array`s specifically use a special `value` keyword in place of a typical attribute name. `Arrays` also support exact `valu`e matching when a `String` is passed to the selector. See examples below:

#### Example of an `Object`:

/* return dependencies that have a `scripts.test` containing `"tap"` */
*:scripts([test~=tap])

#### Example of an `Array` Attribute Selection:

/* return dependencies that have a keyword that begins with "react" */
*:keywords([value^="react"])

#### Example of an `Array` matching directly to a value:

/* return dependencies that have the exact keyword "react" */
/* this is equivalent to `*:keywords([value="react"])` */
*:keywords("react")

#### Example of an `Array` of `Object`s:

/* returns */
*:contributors([email="[e-mail]"])`

### Classes

Given that `Arborist` will control our understanding of the DOM (Dependency Object Model), claseses are predefined by their relationships to their ancestors & are specific to their dependency type. Dependencies may have, & are allowed to have, multiple classifications (ex. a `workspace` may also be a `dev` dependency).

- `.prod`
- `.dev`
- `.optional`
- `.peer`
- `.bundled`
- `.workspace`

### Attributes

There are several known attributes that are normalized & queryable living in `Node.package` (aka. `package.json`).

- `[name]`
- `[version]`
- `[description]`
- `[homepage]`
- `[bugs]`
- `[author]`
- `[license]`
- `[funding]`
- `[files]`
- `[main]`
- `[browser]`
- `[bin]`
- `[man]`
- `[directories]`
- `[repository]`
- `[scripts]`
- `[config]`
- `[workspaces]`
- `[dependencies]`
- `[devDependencies]`
- `[optionalDependencies]`
- `[bundledDependencies]`
- `[peerDependencies]`
- `[peerDependenciesMeta]`
- `[engines]`
- `[os]`
- `[cpu]`
- `[keywords]`

### Command

#### `npm query "<selector>"` (alias `q`)

#### Options:

- `query-output`
- Default: `json`
- Type: `json`, `list`, `explain`, `outdated`, `funding`, `audit`, `duplicates`, `file`

#### Response Output

- an array of dependency objects is returned which can contain multiple copies of the same package which may or may not have been linked or deduped

[
  {
    "name": "",
    "version": "",
    "description": "",
    "homepage": "",
    "bugs": {},
    "author": {},
    "license": {},
    "funding": {},
    "files": [],
    "main": "",
    "browser": "",
    "bin": {},
    "man": [],
    "directories": {},
    "repository": {},
    "scripts": {},
    "config": {},
    "dependencies": {},
    "devDependencies": {},
    "optionalDependencies": {},
    "bundledDependencies": {},
    "peerDependencies": {},
    "peerDependenciesMeta": {},
    "engines": {},
    "os": [],
    "cpu": [],
    "workspaces": {},
    "keywords": [],
    "ancenstry": "",
    "path": "",
    "realpath": "",
    "parent": "",
    "vulnerabilities": [],
    "cwe": []
  },
  ...

#### Usage

npm query ":root > .workspace > *"  # get all workspace direct deps

### Extended Example Queries & Use Cases

// all deps
*

// all direct deps
:root > *

// direct production deps
:root > .prod

// direct development deps
:root > .dev 

// any peer dep of a direct deps
:root > * > .peer  
  
// any workspace dep
.workspace

// all workspaces that depend on another workspace
.workspace > .workspace
    
// all workspaces that have peer deps
.workspace:has(*.peer)

// any dep named "lodash"
// equivalent to [name="lodash"]
#lodash 
  
// any dep named "lodash" & version "1.2.3"
// equivalent to [name="lodash"][version="1.2.3"]
#lodash@1.2.3 
[name="lodash"]:semver(1.2.3)

// querying by semver
#lodash@^2
[name="lodash"][version="^2"]
  
// all deps living alongside vulnerable deps
*:vulnerable ~ *:not(:vulnerable)

// has any deps
*:has(*)

// has any vulnerable deps
*:has(*:vulnerable)

// deps with no other deps (ie. "leaf" nodes)
*:empty

// all vulnerable deps that aren't dev deps & that aren't vulnerable to CWE-1333
*:vulnerable:not(.dev:cwe(1333))
  
// manually querying git dependencies
*[repository^="github:"],
*[repository^="git:"],
*[repository^="https://github.com"],
*[repository^="http://github.com"],
*[repository^="https://github.com"],
*[repository^="+git:..."]
  
// querying git dependencies
*:type(git)

// find all references to "install" scripts
*[scripts=install], 
*[scripts=postinstall],
*[scripts=preinstall]

.prod:not(.dev)
.prod:not([name="lodash"])
    
// license querying (RFC)
*[license="MIT"], *[license="ISC"]

// find all packages that have @ruyadorno as a contributor
*:contributors([email=[e-mail]])

## Future

### Command Mapping to `query-output`

Previous commands with similar behaivours will now be able to utilize `Aborist` `Node.querySelectorAll()` under-the-hood & will fast-follow it's implementation.

#### `npm list`

npm list # equivalent to...
npm query ":root > *"

npm list --all # equivalent to...
npm query "*" --query-output list

npm list <pkg> # equivalent to...
npm query "#<pkg>" --query-output list

#### `npm explain`

npm explain <pkg> # equivalent to...
npm query "#<pkg>" --query-output explain

#### `npm outdated`

npm outdated # equivalent to...
npm query ":root > *:outdated" --query-output outdated

npm outdated --all # equivalent to...
npm query "*:outdated" --query-output outdated

#### `npm fund`

npm fund # equivalent to...
npm query ":root > *[funding]" --query-output funding

npm fund --all # equivalent to...
npm query "*[funding]" --query-output funding

npm fund <pkg> # equivalent to...
npm query "#<pkg>" --query-output funding

#### `npm audit`

npm audit # equivalent to...
npm query ":root > *:vulnerable" --query-output audit

npm audit --all # equivalent to...
npm query "*:vulnerable" --query-output audit

npm audit <pkg> # equivalent to...
npm query "#<pkg>" --query-output audit

#### `npm find-dupes`

npm find-dupes # equivalent to...
npm query "*:deduped" --query-output duplicates

#### `npm view`

npm view # equivalent to...
npm query ":root" --query-output view

npm view <pkg> # equivalent to...
npm query "#<pkg>" --query-output view

## Commands _could_ read from `stdin`

In a future RFC & major version, `npm` could begin reading from `stdin` to chain commands together with a common understand of a dependency object. All of the below commands would add the able to execute as if they were passed package specs.

audit, bugs, ci, config, deprecate, diff, dist-tag, docs, edit, exec, 
explain, explore, find-dupes, fund, get, install, install-ci-test, 
install-test, link, list, outdated, pack, pkg, prune, publish, rebuild, 
repo, restart, run-script, set, set-script, shrinkwrap, star, stars,
start, stop, team, test, token, uninstall, unpublish, unstar, update,
version, view

### Example of piping from `npm query` to other commands

# list workspaces w/ peer deps
npm query ".workspace:has(*.peer)" | npm ls

# install the same dev deps across all workspaces
npm query ":root > *.dev" | npm install --workspaces

# find and enforce a single version of dependencies
npm query ""

# find cve
npm query "*:cve(<id>)"

# show audit details for vulnerable deps that aren't ReDoS dev deps
npm query "*:vulnerable:not(.dev[cwe~=1333]:not(cwe=))" | npm audit explain

## Prior Art

- [`HTML`](https://html.spec.whatwg.org/) & [DOM](https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model)
- [`CSS`](https://www.w3.org/Style/CSS/specs.en.html), [Selectors](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) & [Pseudo Classes](https://developer.mozilla.org/en-US/docs/Web/CSS/Pseudo-classes)
- [`estree`](https://github.com/estree/estree)
- [`abstract-syntax-tree`](https://www.npmjs.com/package/abstract-syntax-tree)
- [`postcss-selector-parser`](https://www.npmjs.com/package/postcss-selector-parser) / [API](https://github.com/postcss/postcss-selector-parser/blob/master/API.md)
- [`css-selector-parser`](https://www.npmjs.com/package/css-selector-parser)
- [`@npmcli/arborist`'s `Node` Class](https://github.com/npm/cli/tree/latest/workspaces/arborist)
- [pnpm's --filter](https://pnpm.io/filtering)
- [Gzemnid](https://github.com/nodejs/Gzemnid)

## Unresolved Questions and Bikeshedding

- Q. Is there any such thing as a bare specifier?
- A. No. Unlike CSS Selectors, there's no `"element"`, or `"package"` in this context, equivalent. The selector syntax presuposes all entities are packages.
- Q. Should this syntax cover **all** possible queries of a dependency graph?
- A. No. This spec is meant to provide a sufficently mature mechanism/building block for (re. 80/20 rule applies here)
- Q. These are conflicting statements:
- there is no "type" or "tag" selectors (ex. `div, h1, a`) as a dependency is the only type of `Node` to query
- `dependency` & `link` types