# Tooling Team Meetings

2023-01-02: Moved to https://hackmd.io/@tmattio/Sy5CU8x5i

## 2022-12-12: Team meeting

- Paul-Elliot: odoc source code gen. Fixing knowne bugs
- Also worked a bit on ocaml-platform and reviewed PRs
- In holidays next week. Continue work on odoc code gen.
- Release ocaml-platform-installer
- Jules: Mostly odoc. Spent time on a tricky bug. Release of odoc, did general maintenance. Jon found a critical bug. Merged PRs on plaform-installer, especially pure binary and cache migration. Released this morning. Off Wed and Friday.
- Guillaume: Worked on ocamlformat support for JS. Also pushed a PR with the diff of AST/CST. Looked into table parsing for odoc.
- Emile: odoc w/ Paul-Elliot to take line directives into account. Also worked on ocamlformat to check AST/CST diff issue
- Thibaut: Platform roadmap and quarter review.

## 2022-12-06: Team meeting

Attendees: Emile, Paul-Elliot, Guillaume, Jules

Agenda:

- Planning
- dune fmt
- odoc table
- odoc sources
- ocamlformat
- odoc release
- platform installer

Past:

- Paul-Elliot
+ odoc sources
reference driver, module alias bugs
+ platform installer
cache migration

- Guillaume
off

- Emile
+ odoc table
+ dune fmt
Discussion, blocked
+ ocamlformat
JS, ast mapper

- Jules
+ odoc sources
struggle with module aliases
+ odoc
general maintenance

Priorities for the quarter:

- platform installer release
+ merge what's still open
+ eventual polish
+ test within the team before any release

- odoc release
+ managed Jon
call with Jon to plan together

- odoc sources
+ Demo is ready!
+ Polish

- ocamlformat
+ Urgent bug from JS

Other priorities:

- dune fmt: Format without having a config
+ Requires `diff-friendly` profile

- odoc tables
+ Polish needed
Remove extra node from AST
Alignment in heavy syntax
Cleanup parser

Assignments:

- Paul-Elliot
+ HTML in odoc-sources
+ platform installer release
+ background: ppxlib manual

- Jules
+ maintain the odoc-sources PR
+ platform installer release
+ odoc release

- Emile
+ HTML in odoc-sources
+ Look at diff-friendly
+ Test the platform installer
+ dune fmt discussions

- Guillaume
+ fix bug reported by JS
+ odoc tables

## 2022-11-28: Bi-weekly

Attendées: Emile, Jules, Thibaut

Agenda:

- Priorities
- Carl request to have someone come at JS

Action points:

- Source code in odoc
- Jules and Paul-Elliot to continue this
- Platform Installer
- Dedicate 1~2 days to complete WIP PR
- Dune fmt
- Won't be done this quarter, depends on diff-friendly profile.
- Emile to land a PR for the ocamlformat conf stanza in Dune.
- Tables
- Jules and surtout Emile to take over after dune fmt PR
- Commercial support
- 2210 -> to merge after final review
- https://github.com/ocaml-ppx/ocamlformat/issues/2201
- Jules and Emile to have a look to find a solution
- Thibaut will continue to discuss with JS to see if we want to send someone to their offices to complete their migration to ocamlformat with Carl. Would solve a lot of collaboration issues, since after the migration they would be able to plan their migrations of ocamlformat.

## 2022-11-22: Dev

Attendées: Paul-Elliot, Jules

Odoc TODO list:

- Wrapped libraries: Carry source code when expanding module aliases.
When hidden modules are expanded inside the entry module, source code information is lost.
- Test other things that have their own page
That the links are correct, for example functor results, `module M = Map.Make (String)`
- Update the reference driver to render source code

Odoc important follow-up:

- Update Dune
Trivial: Register the new dependencies on the `.cmt` and pass the new flags to `odoc compile`.
- Link back from source to doc
Big usability improvement.
The information is already in Lang, it just needs to be processed and presented.

## 2022-11-21: OCaml Platform Roadmap Workshop

Attendees: Thibaut, Paul-Elliot, Jules, Emile

### Agenda

- Q4 roadmap, now with priorities
- Review high-level Platform spec (principles and workflows)
- Discuss 2023 roadmap for Tooling team
- For selected workflows, discuss metrics and users to involve

### Notes

- Workflows
- Editors
- Would be nice to have support for OCaml on GitHub code explorer
- Might be difficult to ship vim extension with LSP because it adds a lot of constraints (deps, version of Vim, etc.)
- Everything is done in the editor
- How to catch up with progress on the OCaml Platform?
- Formatting
- Paul-Elliot: Should be able to format by structured item
- Jules: It's a goal of ocamlformat
- Worried about dune <> opam
- How to discover publication workflow and exploration of libraries if we hide the opam mentions (but the repo is still opam-repository)
- Rename opam-repository to ocaml/packages?
- Debugging
- OCamlPro might have good feedback on this!
- Litterate programming
- Name isn't relevant anymore, it's not only markdown anymore
- Google search will never give mdx as a first result
- Documentation
- It's actually the workflow of odig
- Installer generation
- Need to update dune file to make static build. Should be able to do this from the CLI.

- Roadmap
- Dune <> opam is the most impactful
- Risk and Cost
- Difficulty: Planned
- Impact: 5/5
- We must contribute to this. Especially integration with tools.
Init, fmt, doc.
- Documentation (tooling and packages)
- maintenance of odoc
- Search documentation
- Documenting the OCaml Platform (and its tools). Especially Dune.
- Editor support for odoc (syntax and tooling)
- Have a CI on opam-repository
- Gamify the doc on OCaml.org?
- Button to open an issue directly on OCaml.org package doc
- Guidelines on documentation on OCaml.org
- Bounty for the documentation
- Review documentation of projects
Projects: ocamlformat, odoc, etc...
Is it visible on `docs.ocaml.org` ? Is the `README` clear ?
- Documentation (workflows)
- Add docs on the workflows to OCaml.org
- Tutorial on how to document a package
How to write an `index.mld` page, how to look good on `docs.ocaml.org`.
Style guide and conventions.
- Debugging
- Risk and Cost
- Difficulty: 5/5
- Impact: 4/5
- It's a huge gap in development workflow
- It's the foundation of other languages
- People use Printf
- Have the integration in the editor (DAP)
- Risk: might be too big to chew, do we need someone with exp on the compiler?
- Team is highly motivated to work on this
- Linting
- Risk and Cost
- Difficulty: 3/5 (MVP is easy, but difficult to go beyond)
- Impact: 3/5
- A lot of design to do, how do we define the rules?
- Might not be high priority: users learn to code reading code also. Not a blocker if it doesn't exist.
Jules: Disagree.
- We can keep a list of rules we want to have in a linter, then revisite later
- R2C
Interesting rules:
* Migration between OCaml versions. eg. deprecated functions alternatives, stricter type checking
* Functions not compatible with Windows (might be attributes added to the Stdlib)
* Make use of thread-safety attributes added to the stdlib
- Project generation
- Risk and Cost
- Difficulty: ?
- Impact: More than linting
- Useful for beginners and advanced users
- Should also be able to update an existing project (e.g. add documentation?)
- See good ideas on Spin, Drum, Oskel but the goal is to improve `dune init`.
- Host templates in a central repository
- Host templates in third party repositories
eg. a library like Dream hosting its own templates
- Open a REPL
- A lot of people use it, but as soon as you're on a large project, it's limited
- Distributing applications
- Not sure how many people build applications though? Although that's the ultimate goal of a programming language: building applications.
- How to distribute to cloud providers
- Editor support
- Odoc markup support in editors (syntax highlighting)
- General maintenance

Notes:
- Paul-Elliot: focus on the most impactful in terms of number of people
- Emile: web frameworks are critical for the adoption of a language
- Jules: Linter seems to be commonly requested. Even from the OCaml core team (eg. migrate from deprecated functions)

- The cream
- Package publication
- Not a priority: target is experienced developers, not beginners

## 2022-11-14: Planning

Agenda:

- Catch up for Thibaut
- Planning

Work last weeks:

- Jules
- Ocaml-platform: Removed the "duplicated" cache.
- odoc: Link to source code, using shapes. Still need to work out the edge cases. (e.g. wrapped libraries)
- Thibaut:
- Update on the Platform roadmap. Shared with Jane Street and need to talk to users
- Paul-Elliot:
- Continued to work on odoc. Syntax highlighting is working, now working on jump to definition. Later also will work on types
- OCaml-platform: small issues and implement "binary only" PR
- Ppxlib documentation: made good progress
- Emile
- Dune: Specify in Dune files which files are to be formatted
- ocamlformat: Investigate the JS issue.

OCamlformat:

- Can we have an explanation of the pain point they have?
- This would allow us to suggest alternatives
- ETA: a week, but let's try to understand what the problem is anyways.

- Sharing code:
- Can they share code they would like to work on from their open source code?

odoc:

- Before we work on the index file, we want to finish the source code in doc project

## 2022-11-09: Dev

Présents : Jules, Paul-Elliot, Emile

Done: Adding syntax highlighting on top of Jules+Guillaume branch.

Work this week:

- Jules:
+ ocaml-platform: Versionned data and some refactoring
Persistent data might get out of date as new version of the tools are
installed. We need a way to migrate them.
First a refactor to clarify the way the cache is used and remove edge cases.
Then a refactor to remove uneeded persistent data (overriden system-compiler packages).
+ Investigate the canonical/hidden case in odoc

- Paul-Elliot:
+ Write syntax highlighting css
+ Prototype extracting interesting locations and associated data from cmts
+ Continue Ppxlib manual

- Emile:
+ Integration between Dune and OCamlformat

Links documentation -> source code:

- Actually implement links documentation -> source code
Source code is now generated, with coloration. Locations informations are in Lang.

- Find test case about canonical and hidden paths
For example, "wrapped" toplevel modules generated by Dune.
Module "A" of a library "Foo" have its source code inside the compilation unit
"Foo__A". It is possible that this "hidden" module is never passed to
html-generate. We must make sure that the source code is generated.

- Write css of syntax highlighting.

Counting occurences project:

- Output usage statistics of identifiers to enable ranked search
We discussed that the informations needed for this project are also the
information needed for "source -> documentation" links. These informations
are: a list of interesting locations in the implementation and their
associated identifier.
It should be done first.

Future work:

- Prototype of extracting interesting locations and associated data from cmts.
These interesting locations are:
+ use of a value, and the identifier associated
+ use of a value, and the type associated
+ use of a type, and associated identifier

## 2022-11-07: Planning

Présents : Jules, Paul-Elliot, Emile

- Previous:
- Paul-Elliot: Generation of a document from a set of position. Used for syntax highlighting.
- Jules & Guillaume:
+ odoc:
* Load and resolve locations of declarations and store them in Lang.
Locations are looked up from the Shape of the module, which
contain a description of the whole signature, including locations.
Shapes are stored in `.cmt` files but Odoc is often given a
`.cmti`. The corresponding `.cmt` is now also loaded to retrieve
the Shape. This operation might fail and the locations will simply
not be available in the generated doc.
- Next:
- Emile : Work on occurance counting.
- Paul-Elliot: Integrate in the current branch. (pair program with Jules on Tuesday)
+ Generate the html source in the same folder as the interface html
+ Discussion on the shape location vs odoc location (for directory structure): the shape reduction is not allowed to use information from outside (not compiled) modules.
- Paul-Elliot: working on https://github.com/tarides/ocaml-platform-installer/issues/132
- Jules: Work on ocaml-platform-installer
+ Pure binary: naming our cached packages "ocamlformat+bin+platform"
won't work anymore with Opam's `with-dev-setup`, we need to override
the original package in every case.
https://github.com/tarides/ocaml-platform-installer/issues/128
+ Versionned cache: We need to change the layout of the cache and need to migrate automatically
+ Investigate faster resolution of versions. Currently several seconds.

## 2022-11-02: Planning

Présents : Jules, Guillaume, Paul-Elliot, Emile

- Yesterday:
- Paul-Elliot: Generating html:
- Interfacing with highlight-js or hilite is not possible
- look at what Haskell is doing: syntax highlighting with info from lexer
- Jules et Guillaume:
- Add locations to the document Lang types (alternative to using the shapes)
- Next:
- Paul-Elliot: Try to implement ternary red-black tree and html generation
- Jules: Continue on loading cmts and looking up locations. Planned PP with Guillaume.

## 2022-10-26: Dev

Présents : Jules, Guillaume, Paul-Elliot, Emile

- Yesterday:
- Paul-Elliot: experimented on Typedtree traversal and ocaml env lookup
- Jules et Guillaume: Expérimente avec les shapes pour lookup les definitions. WIP
- Next:
- Jules + Guillaume continue pairing on same subject
- Paul-Elliot + Emile : use typed-tree traversal and env info to generate html augmented source code.

Prochaine réunion: Lundi?

## 2022-10-25: Dev

Présents : Jules, Guillaume, Paul-Elliot

Discussion:

- Quand stocker les shapes, source code et autres informations sur l'implémentation
- Au moment de la phase "compile" d'odoc a les avantages suivants:
- `odoc compile` prendrait le code source en argument et les `cmt` dans les chemins de recherche (`-I`)
- le .cmt et le code source pourrait ne plus être présent aux autres phases
- On peut générer un document (au sens de odoc) représentant l'implémentation qui pourra être transformé en html (ou autre) par différents renderer et non seulement celui officiel d'odoc.
- Il faudra modifier le driver d'odoc (dune) dans tous les cas

- Guillaume a commencé le backend: https://github.com/ocaml/odoc/pull/904

Travail aujourd'hui :

- Jules et Guillaume pair-programment sur les shapes et la pipeline
- Paul-Elliot expérimente avec le backend html (récupèrer des infos dans le typedtree et les utiliser pour qqchose de simple)

## 2022-10-24: Planning