# Chasm Meetings

## May 7th 2022 8AM (UTC)

### Agenda

### What happened since the last meeting?

### What's the current status?

- Chasm exists and is functional
- Missing Issue: Can't add classes
- Can't slice into root node because of LazyClassNode ugliness
- Suggestion: Abstract lazy nodes (maybe reuse for methods) and clean up CoW
- Issue: Local Variable Representation
- Local variable indices exist, but are not useful for Chasm
- There are multiple ways to represent locals
- Parameters are part of locals, but: Parameters are also part of the descriptor
- We have to ensure no redundant data in the tree
- What about signatures?
- Ignored by JVM, used for decompilation and reflection
- Ideally keep them valid
- But they contain redundant information
- Missing issue: Records?
- Chasm-Lang exsits and is functional
- Issue: Intrinsics
- Issue: Specification
- Issue: Better Errors
- Chasm-Gradle-Plugin exists and is functional (kinda)
- Simple implementation, but it works
- IDE integration is meh
- Attached as files rather than maven dependencies (maybe ok?)
- Attached once per source set (e.g. once for main and once for test)
- Doesn't run on sync (maybe ok?)
- Slow!
- Mostly because of deflate and inflate
- Consider "caching" of unzipped input files
- Consider writing only files changed since last transformation
- Speed is not super important in prod (because caching)
- Speed is important in dev (because no caching)
- Chasm-Access-Widener exists locally and is partly functional
- Doesn't have descriptor matching because of the above issues
- Quilt loader integration is missing
- Waiting on loader merging upstream
- Might require significant changes to KnotClassLoader
- Chasm-Mixin is missing

### What needs to be done next?

- Focus on Chasm class issue
- Focus on Lang Specification
- Focus on Loader
- Focus on Accesswidener
- After that:
- Development branch
- Repository versioning

### Additional Notes

- Meetings every 2 weeks (in the weeks without dev meeting)
- Somewhat fixed time: Monday 5pm UTC before dev-meeting
- This is just a test

## May 16th 2022 5PM (UTC)

### Agenda

- Remapping

### What happened since the last meeting?

Nothing

### What's the current status?

Same as last meeting

### What needs to be done next?

Same as last meeting

### Remapping

#### Mixin

- Mixin is based of Strings
- Mixin annotations don't get remapped.
- Refmap is gnererated containing mappings from dev names to runtime names
- Annotations are remapped at runtime

#### Chasm

- We need to remap transformers
- Preferably at compile time, but needs to be supported at runtime
- Different ways of doing it:
- Regex replace in the actual transformer
- Make "special strings" that get remapped
- Pro: Doesn't require much work work in chasm
- Con: Only works for fully qualified names
- Make a special node in the chasm structure
- E.g. "remap" object in the root node
- Pro: No extra logic in chasm
- Con: Remapper needs to understand chasm file format
- Intrinsic function for remapping
- E.g. `remap("net/minecraft/World")`
- Con: Remapping only at runtime
- Con: Frontend needs to be mapping aware
- Pro: No extra work in chasm required (intrinsics provided by runtime)
- Pro: Remapper doesn't need to understand chasm file format
- Extract special node to a separate File
- E.g. "remap" object in separate file
- Pro: Remapper only needs to remap json
- Con: Special logic to import external files BUT wanted anyway (for config or similar)
- Put references into special class containting references
- E.g. `transformerId.Remap`
- Pro: Remapper needs no extra work
- Pro: No extra work in chasm
- Con: Cursed af
- Con: Bigger files
- Solution number 4 seems preferred

The transformer (doesn't need to be remapped):

// transformer.json
{
  id: "ExampleTransformer",
  remap: file("refmap.json"),
  trasformations: [
    {
      target_class: classes[c.name = $.remap.classes.world]
    }
  ]
}

Refmap in dev, generated by frontend:

// refmap.json
{
  mappings: "org.quiltmc:quilt-mappings",
  classes: {
    class_0: {
      name: "net/minecraft/World"
    }
  },
  methods: {
    method_0: {
      class: "net/minecraft/World",
      name: "methodName",
      argTypes: [
        "Lnet/minecraft/entities//Entity;"
      ],
      returnType: "Ljaval/lang/String;"
      descriptor: "net/minecraft/World/methodName(Z)V"
    }
  }
}

Refmap in mod, remapped by build tool:

// refmap.json
{
  mappings: "org.quiltmc:hashed",
  classes: {
    class_0: {
      name: "net/minecraft/unmapped/C_25nhd73"
    }
  },
  methods: {
    method_0: {
      class: "net/minecraft/unmapped/C_25nhd73",
      name: "m_fhjd73n",
      argTypes: [
        "c_hgd9dud"
      ],
      returnType: "Ljaval/lang/String;"
      descriptor: "net/minecraft/unmapped/C_25nhd73/m_fhjd73n(Z)V"
    }
  }
}

## May 30th 2022 5PM (UTC)

### Agenda

- Additional files/includes
- Versioning
- Blanketcon

### What happened since the last meeting?

- https://github.com/QuiltMC/chasm/pull/47

### What's the current status?

- Chasm exists and is functional
- Issue: Local Variable Representation
- See above
- Missing issue: Records?
- Chasm-Lang exsits and is functional
- Issue: Intrinsics
- Issue: Specification
- Issue: Better Errors
- Issue: Treat `classes` as a fucntion input
- Chasm-Gradle-Plugin exists and is functional (kinda)
- Simple implementation, but it works
- IDE integration is meh
- see above
- Slow!
- see above
- Doesn't work with loom
- Chasm-Access-Widener exists locally and is partly functional
- Doesn't have descriptor matching because of the above issues
- Quilt loader integration is missing
- ~~Waiting on loader merging upstream~~
- Loader is diverging from upstream
- Might require significant changes to KnotClassLoader
- Probably replace the entirety of Knot
- Wait for loader-plugins since that also messes with Knot
- Chasm-Mixin is missing

### Additional Files/Includes

We want to include additional files in transformers, e.g.

{
  id: "example_transformer",
  refmap: include("refmap.json"),
}

Problem: In order to allow caching, we need to be aware of all used input files.

Solution:
- Keep track of files used during last execution, rerun if any changed
- Requires some additional logic in chasm to emit file list
- Pessimistically assume all files in a certain path to be required
- No extra logic in chasm, but might hash unnecessary files
- Disadvantage: "magic direcory" makes it hard to differnetiate between transformers and libs

We choose option one.

### Versioning

Currently: Chasm uses 0.0.1-SNAPSHOT for everything.

Once chasm becomes "somewhat stable", we need better versions.
Suggest a git layout for the "far" future:

- Main branch
- Always the lastest release version (e.g. 1.2.3)
- Bump major opr minor version when merging development branch
- Development branch
- Always has the next minor release snapshot version (e.g. 1.3.0-SNAPSHOT)
- Probably just released on the snapshot maven
- Aways based on the main branch
- Rebase when a patch gets applied
- All development happens in PRs
- Feature PRs
- Target development branch
- Merging doesn't bump the version
- Patch PRs
- Target main branch
- Bump patch version

### Blanketcon

Blanketcon is happening and we have a keynote June 11th, 18:00 UTC, directly after quilt dev meeting.
We need to prepare a presentation!

## ~~June 20th 2022 5PM (UTC)~~
Cancelled

## June 17th 2022 5PM (UTC)

### Agenda

- Module Restructure
- JavaCC21
- Lang CodeGen
- Lang API

### What happened since the last meeting?

- Blanketcon happened!
- Keynote: https://www.youtube.com/watch?v=_Q4QH0PW6dE
- Some issues have been opened
- JavaCC21: https://github.com/QuiltMC/chasm/issues/48
- Restructure: https://github.com/QuiltMC/chasm/issues/49
- Code Gen: https://github.com/QuiltMC/chasm/issues/51
- Lang API: https://github.com/QuiltMC/chasm/issues/52

### Current status

Same as last time, except one PR for restructuring:
https://github.com/QuiltMC/chasm/pull/53

### Restructure Modules
Chasm and Chasm-Lang interop will be removed from Chasm-Lang to make the module dependency free.
Two options:
- Move the code into the chasm module
- Move it into a new module
For now, we choose the first option.

### Switch to JavaCC21
Antlr problems:
- Needs dependency that is 3 times the size of chasm
- Shadowing is awkward with licensing

Switching to JavaCC:
Pros:
+ Chasm-Lang is dependency free
+ Chasm-Lang is much smaller (including dependencies)
+ No licensing concerns whatsoever
Cons:
- Requires a somewhat sizable rewrite
- JavaCC is ooooold and outdated -> JavaCC21 instead
- JavaCC21 has no versioning
- JavaCC21 has no docs, but it's mostly the same as JavaCC

Conclusion: Leave issue open until it bothers me.

### Lang Code Generation
People have to generate Chasm-Lang.
For both frontends (Mixin, AW) and the remapper.
It would be nice to have a library for this.
Basically:
- Construct/Modify an `Expression`
- Write an `Expression` to file
- Or similar?

### Lang API
Currently there's no api/internal split.
Additionally, the public API isn't "nice".
Needs work.

## July 18th 2022 5PM (UTC)

### Agenda
- Go over PR#54
- Discuss further steps

### What happened since the last meeting?
- Chasm-Lang rework 2
- Draft PR: https://github.com/QuiltMC/chasm/pull/54

#### PR 54
- Switch to JavaCC21
- Concerns: Bad dependency
- AST is currently free of implementation detail
- Bad OOP, likely to change it back
- Currying is broken
- Probably best fixed with closures
- Consider replacing the Chasm specific tree with the new Chasm-Lang AST
- Saves on conversions
- Reduces code base size
- Possibly a lot of work

## August 15th 2022 ~~5pm~~ 6:30pm

Present:
- CheaterCodes
- Kropp

### Agenda

- Review current stauts
- Contribution Guideline
- Chasm Roadmap
- Innclude directives
- Json parity
- Parser Talk

### What happened since the last meeting?

PRs:
- #54 Lang rework (by CheaterCodes)
- #55 Chasm Renderer (by RTTV)
- #57 Map & Flatten (by will)
- #58 Fix Annotations (by FoundationGames)
- #64 Add Metadata (by CheaterCodes)
- #65 Fix Slice Offset (by CheaterCodes)
- #66 Fix Trailing Commas (by CheaterCodes)
- #67 Source Spans (by CheaterCodes)

Lots of testing by FoundationGames:
- Add chasm to nil-loader (kind of)
- Start on a mixin fork (Chasmix)

### Currently in the works

PRs:
- #61 Fix operator precedence (and more) (by Kropp)
- #60 Fix error in writing annotations (by Kropp)
- #59 Map operators (by FoundationGames

### Contribution Guideline #50

- CONTRIBUTING.MD
- General stuff, PR procedure etc.
- Every bugfix PR should included tests

Branch protection is enabled:
- Contribution only via PR
- Requires one Review (from the team)
- Must pass checks (build + tests + checkstyle, aka ./gradlew build)
- Must be up to date (via merge)

### Chasm Roadmap
TLDR: I want to write stuff down without creating issues.

GitHub Projects?
+ Would allow for tracking milestones
+ Allows short entries without issues
+ Also tracks issues
- Extra effort for no gain?

### Include directives

We want the option to include other Chassembly files in a Chassembly file.
E.g.

{
  lib: include('other/lib.chasm'),
  call: lib.helper(5),
  ...
}

Problem: Who's managing the filesystem?
- Easy solution: Don't provide this functionality
- Execution environment can provide an intrinsic (e.g. quilt-loader)
- Problem: Not portable
- Problem: Makes Chasm harder to use
- Better Solution: Require an implementation
- Execution environment *must* provide an intrinsic (e.g. quilt-loader)
- Problem: Makes Chasm harder to use
- Big change: Make Chasm filesystem aware
- Move all jar unpacking logic etc. into chasm
- Problem: More work for Chasm, possibly less flexibility
- Problem: Even less portable?
- Best Solution?: Require an implementation, provide a default in ChasmUtils
- Chasm provides an optional default implementation, but allows replacing it.
- Problem: Possibly complex/out of scope

### Json Parity

Include is intended for Chassembly files.
Main usecase: Reference (remapped) class names in separate file.

Question: Should those files be Json? Should Chassembly be a superset of Json?

Reasoning:
Remappers might want to use a Json library and not Chassembly.
Specifically, "new quilt-mappings" might be Json based, so Json library is already bundled.

What's needed to read Json?
- Strings as keys
- Number formatting?
- Parsing details? (e.g. String escapes)
- Do we want to test this? (test suite)
- How much is missing for json5?
- ~~How much is missing for js?~~

### Parser Talk

Previously: Antlr4
- Adaptive LL(\*)
- Visitor Based
- Priblem: Big runtime dependency (>300kB)

Currently: JavaCC21
- LL(k)
- Embedded Java Code based
- Problem: Lots of code repetition

Alternatively: JavaCC21 with tree building #63
- LL(k)
- Autogenerate Parse Tree
- Problem: Generated AST is almost useless
- Nodes are mostly just empty marker classes
- Functionality needs to be injected
- 50kB of empty classes
- Not really suitable as API

Ideal: Parser generator XX
- LL or LALR or whatever, powerful enough for Chassembly
- Visitor Based (No parse tree required)
- No runtime dependency, small footprint
- Problem: Don't know any

Good news: Parsing is not an implementation detail
- Public AST should remain stable
- Changing the parser shouldn't require a full rewrite

## September 14th 2022 5:30pm

Present:
- Kropp
- Earthcomputer
- CheaterCodes

### Agenda

- Review current stauts
- Include Directives
- Release Roadmap
- Labels
- Chassembly Optimization
- Chassembly IntelliJ Plugin
- ASM Chasm Frontend?

### What happened since the last meeting?

- Multiple bugfix PRs #70, #71, #72, #74
- Include Intrinsics #73

### Include Intrinsics

- New intrinsics to include files:
- file
- Implementer must provide a `readFile` function:
- Must be pure
- Path format (currently) implementation defined
- Must be OS independent
- Possibly also allows configs e.g. `include("config:modid/config/name")`

### Release Roadmap

Github project now available under https://github.com/orgs/QuiltMC/projects/12/views/1
- Every team member can (and should!) add items to the list
- Every team member can (and should!) convert drafts to issues
- Please use liberally!

### Labels

Problem: Copying instructions from one method to another poroduces invalid bytecode.
How should this be handled?

- Transformers SHOULD prefix labels they produce with their ID
- Method labels are *not* globally unique
- Method labels *could* be unique, but this doesn't actually solve problems
- Two transformers copying from the same method still need to prefix
- Prefixing isn't hard
- Not enforced in code?
- Not sure if we could
- General assumption:
- Compiler developers know what they're doing
- So that users don't have to
- Needs documentation!!
- Transformer ID is provided by execution environment
- We need to pass the ID to the transformer (just like `classes`)

### Optimization

Why is it needed?
- Recursion is essential to transformers
- E.g. Looping through a list
- Intrinsics only help for (common) special cases
- Default performance is suboptimal

Possible optimization:
- Common subexpression extraction and reordering
- Must maintain original behaviour (especially errors)
- Tail recursion
- Reorder for more tail recursion
- Perform function substitution late to keep stack space small
- Mutability
- Lists are immutable
- Modifying a list requires copying it
- If the original isn't used afterwards, the list can be mutated instead
- Similar to Rust move vs. copy

### Chassembly IntelliJ Plugin

Earthcomputer has been working on an IntelliJ plugin for Chassembly language support.
- "Not meant to help writing Chassembly by hand!!1!11!" (exceptions apply)
- Make frontend development easier
- Mostly for writing libraries and for manual experimentation
- Not yet published

### ASM Chasm Frontend

Earthcomputer has been working on a Chassembly JVM bytecode interpreter
- Support legacy ASM trasformers
- Partially suppoirt mixin plugins?
- Limitations apply
- Still pure (e.g. no internet access)

### Kotlin Frontend?

Kotlin team has created a channel for discussing frontend ideas.
No specifics yet.

## January 11th 2023 20:00pm UTC

Present:
- Kropp
- Pyrofab
- Will
- Earthcomputer
- CheaterCodes

### Agenda

- Chasm pre-release
- Transformer Format
- Lock Lifetimes
- Transformer Sorting

### What happened since the last meeting?

- Lots of Chassembly things (syntax, intrinsics)
- "Virtual" Transformers

### Chasm pre-release