# parser issue prioritization

I have grouped the parser-related issues (current as of 2024–07–08) into six categories, highest priority first:

1. **round-trip failures** – these are the most important, IMO, because they are effectively deferred parse failures. Something is broken, but you don’t learn about it until you try to edit the definition later
2. **parser failures** – this is anything that prevents the code from making it to the typechecker. Code is the most important part of a PL, so we shouldn’t fail to parse good code
3. **transcript parser failures** – These don’t affect code in the codebase, but they can affect testing & error reporting, and again, we shouldn’t reject valid input
4. **incorrect `Doc` parsing** – this at least gets the definition into the code base, but not always in the way we want (and this doesn’t include round-trip failures, those are in category 1)
5. **incorrect transcript parsing** – again, it works, but not always quite the way we want
6. **poor messaging** – confusing error messages, but at least something is _actually_ wrong with the code
7. **feature requests** – new features, not incorrect behavior

Within each of these categories, I’ve _mostly_ ordered the tickets newest-first, just assuming they’ve bothered someone _recently_, so that bumps them up. Also, issues with duplicates are ordered by the _newest_ duplicate (again, it bothered someone recently) and perhaps should get a boost for being repeatedly requested.

## recommendation: never fail on `Doc` parsing

I think the bulk of the `Doc` parsing already approximates this – like with Markdown, we just fall back to dumping things into `Word`. One place we don’t, however, is with code that’s intended to be evaluated in some way – `Transclude`, `Example`, etc. Failing to parse or run that code causes the Unison parser to fail.

But I propose that having the Unison parser succeed is more important than capturing the `Doc` correctly. We should emit warnings for `Doc` failures, but leave it up to the user to choose whether they’re happy to `add` or `update` with the incorrect `Doc` or to fix them first.

I’ve split some categories into “downgradable” and “non-downgradable” sections, the “downgradable” issues can be deprioritized if we make this one change. I think that alone would make users a lot happier with the parser.

A second aspect of this I’ve put into a section called “`Doc` parser overrun”. These are cases where simply having the `Doc` parser warn instead of fail isn’t sufficient – it’s where the `Doc` parser continues past what should be its closing `}}`, so even if the `Doc` “succeeds” somehow, it leaves the Unison parser in an unparsable state.

## rough prioritization

### round-trip failures

downgradable to “incorrect `Doc` parsing”

- [#4727](https://github.com/unisonweb/unison/issues/4727) – `Example` prints byte literal incorrectly
- [#4726](https://github.com/unisonweb/unison/issues/4726) – `Example` either requires extra parens or prints without enough parens
- [#4725](https://github.com/unisonweb/unison/issues/4725) – `Example` prints with invalid syntax
- [#2860](https://github.com/unisonweb/unison/issues/2860) ([#4660](https://github.com/unisonweb/unison/issues/4660)) – `Example` prints as invalid `Transclude`
- [#2884](https://github.com/unisonweb/unison/issues/2884) – `@typecheck` adds extra `let`

non-downgradable

- [#4736](https://github.com/unisonweb/unison/issues/4736) – pretty-printer w/symbolic names
- [#4733](https://github.com/unisonweb/unison/issues/4733) – discards empty assignment
- [#2889](https://github.com/unisonweb/unison/issues/2889) ([#3757](https://github.com/unisonweb/unison/issues/3757), [#4728](https://github.com/unisonweb/unison/issues/4728)) – omit `let` in multi-line case
- [#4711](https://github.com/unisonweb/unison/issues/4711) – delayed Int literal doesn’t round-trip (maybe omitting parens?)
- ~~[#2822](https://github.com/unisonweb/unison/issues/2822) ([#4681](https://github.com/unisonweb/unison/issues/4681)) – can’t start NameSegment with underscore~~
- [#4474](https://github.com/unisonweb/unison/issues/4474) – ability with long constructor line-wraps
- [#4348](https://github.com/unisonweb/unison/issues/4348) – `use` printed on wrong line (**MAYBE DUPLICATE** [#4344](https://github.com/unisonweb/unison/issues/4344), fixed)
- [#3730](https://github.com/unisonweb/unison/issues/3730) – pure computation rendered without `{}` (maybe intentional (see [#738](https://github.com/unisonweb/unison/issues/738)), but inconsistent)

### parser failures

downgradable to “poor messaging”

- [#5178](https://github.com/unisonweb/unison/issues/5178) – `@source{Stream.emit}` gives the wrong error message

`Doc` parser overrun

- ~~[#5076](https://github.com/unisonweb/unison/issues/5076) – cascading doc parser failure~~
- [#5073](https://github.com/unisonweb/unison/issues/5073) – doc parser overruns \`}}\` and fails

non-downgradable

- [#5179](https://github.com/unisonweb/unison/issues/5179) – `error "impossible"` in lexer
- ~~[#4731](https://github.com/unisonweb/unison/issues/4731) – `match` with no cases~~
- [#2885](https://github.com/unisonweb/unison/issues/2885) – parser fails to resolve hashes (parser shouldn’t be resolving hashes!)
- [#1537](https://github.com/unisonweb/unison/issues/1537) – can’t use one or more pipes as operator
- [#1157](https://github.com/unisonweb/unison/issues/1157) – can’t use emoji identifiers (also, confusing error)
- [#733](https://github.com/unisonweb/unison/issues/733) – symboly prefix patterns
- [#686](https://github.com/unisonweb/unison/issues/686) – last line is `use`

<details>
<summary>transcript parser failures</summary>

- ~~[#5168](https://github.com/unisonweb/unison/issues/5168) – parser blows up on more backticks~~
- ~~[#2826](https://github.com/unisonweb/unison/issues/2826) ([#2863](https://github.com/unisonweb/unison/issues/2863)) – transcripts break on fenced blocks (support longer fences, restrict to start of line)~~
- ~~[#1421](https://github.com/unisonweb/unison/issues/1421) – a bug regardless, but related to significant whitespace~~
</details>

### incorrect `Doc` parsing

- [#2483](https://github.com/unisonweb/unison/issues/2483) ([#4633](https://github.com/unisonweb/unison/issues/4633), [#4777](https://github.com/unisonweb/unison/issues/4777), [#4993](https://github.com/unisonweb/unison/issues/4993)) – incorrect parsing when against `(` `)`
- [#4651](https://github.com/unisonweb/unison/issues/4651) – aliased `Eval` blocks
- [#2890](https://github.com/unisonweb/unison/issues/2890) ([#3895](https://github.com/unisonweb/unison/issues/3895)) – delimiter requires whitespace afterward
- [#2440](https://github.com/unisonweb/unison/issues/2440) – Unit type fails to became type link

<details>
<summary>incorrect transcript parsing</summary>

- ~~[#5141](https://github.com/unisonweb/unison/issues/5141) – parser ignores comment delimiters~~
- ~~[#1809](https://github.com/unisonweb/unison/issues/1809) – omits backtick in output~~
</details>

### poor messaging

- [#5017](https://github.com/unisonweb/unison/issues/5017) – confusing message / bypasses fold
- [#813](https://github.com/unisonweb/unison/issues/813) ([#3990](https://github.com/unisonweb/unison/issues/3990), [#4755](https://github.com/unisonweb/unison/issues/4755)) – incorrect parse overruns the fold
- [#4751](https://github.com/unisonweb/unison/issues/4751) – confusing message when reserved word in wrong place
- [#4580](https://github.com/unisonweb/unison/issues/4580) – definitions that don’t start in column 0
- [#4050](https://github.com/unisonweb/unison/issues/4050) – misleading error with pattern match & typo
- [#3743](https://github.com/unisonweb/unison/issues/3743) – `TrivialError` printed when symbolic name not parened
- [#1683](https://github.com/unisonweb/unison/issues/1683) – confusing error message when pulling non-extant file
- [#793](https://github.com/unisonweb/unison/issues/793) – confusing error message with invalid syntax

### feature requests

- [#2228](https://github.com/unisonweb/unison/issues/2228) – underscores in numeric literals
- [#499](https://github.com/unisonweb/unison/issues/499) – pluggable syntax
- [#1842](https://github.com/unisonweb/unison/issues/1842) – pluggable syntax (Lisp)
- [#449](https://github.com/unisonweb/unison/issues/449) – add namespace blocks