Octez Merge Team synchronisation

# Octez Merge Team synchronisation ## Resources - [Google Meet](https://meet.google.com/nic-eftz-jui) • [Marge Bot logs](https://grafana.nomadic-labs.cloud/d/margebot/margebot?orgId=1) • [Merge team on/deboarding process](https://hackmd.io/l6Oqd_3RT_OV7A1k2c9J9w) - [Merge dispatcher](https://gitlab.com/tezos/tezos/-/issues/1062): - Dec 2021 -> May 2022: @Sventimir is the merge coordinator - May 2022 -> Now: @arvidj is the merge coordinator - Future Vacations / OOO <details> <summary>Current merge team members</summary> - **DaiLambda**: Jun - **Functori**: Alain, Mohamed - **Marigold/Ligo**: Tom Jack, Lin, Pierre-Emmanuel C. ... - **Nomadic Labs**: Albin, Antonio, Arvid, Boubacar, Diane, Eugen, François, Ilias, Jérémie, Julien, Killian, Klakplok, Mathias, Nicolas A., Pierrick, Pirbo, Raphaël C., Raphaël P., Romain, Sylvain R., Thomas, Valentin, Victor A., Vincent, Yann - **Oxhead**: Seb - **TriliTech**: Andrea, Joel, Ole, Emma Turner </details> <details><summary>Merge dispatcher role</summary> - Self-assign to the [Meta issue - Dispatcher](https://gitlab.com/tezos/tezos/-/issues/1062) - Assign [draft MRs with no Assignee](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=opened&draft=yes&assignee_id=None) back to their author - Assign people to [MRs that don't have an Assignee](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=opened&assignee_id=None). - This is not necessarily Merge team members - Label MRs: - There is no policy on mandatory labels - A good rule of thumb is to start with [MRs with no label](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=opened&label_name[]=None) - [MRs with no update in the last month](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&sort=updated_asc&state=opened) - the first time, ping the author/assignee to ask for the status - the second time (if no response the first time), close the MR - Ping people who have something to do (use “MR status tracker” or [A2K](http://a2k.nomadic-labs.com/)). - Assign easy/reviewed ones to Marge Bot. - You can start with [already approved MRs](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=opened&approved_by_usernames[]=Any&draft=no&not[assignee_username]=nomadic-margebot) - Prepare the meeting - For each agenda point, start on previous Friday a Slack thread in `#merge-team` channel, to let MT members discuss asynchronously each point - Fill-up this document with new agenda </details> ## 2025-02-10 - L1: Propose Gabriel M. and Adam AG to the merge team. - Tobi ## 2025-02-03 - Add Felix Puscasu to merge team? - He has been an integral part of the RISC-V team from almost the beginning. Most of the RISC-V instructions supported by the interpreter have been implemented by Felix. He has and continues to ensure the RISC-V machinery is specification compliant. Felix is capable of tackling complex tasks end-to-end, which he has demonstrated by implementing the memory translation unit of the RISC-V VM, for example. - Beyond his technical skills, Felix is a valuable author and reviewer of merge requests. His desire to understand things end-to-end underscores this. He is a reviewer I can trust. ## 2025-01-13 - Git subtrees for vendored libraries (Alain) - [Manual](https://manpages.debian.org/testing/git-man/git-subtree.1.en.html) - [Tutorial](https://www.atlassian.com/git/tutorials/git-subtree) - Alternatives: - [git-subrepo](https://github.com/ingydotnet/git-subrepo) - [git-vendor]() (wrapper subtree, not very flexible) - Pros: - Transparent for users/devs (unlike submodules) - Easy to pull changes from upstream, even with local patches (just `git subtree pull` and fix conflicts) - Possible to contribute back to upstream - Distributed with git standard in distros - Cons: - Slightly larger history (though, only one extra commit per sync with `--squash`) - Need to clean up after `subtree add` - Build system ## 2024-10-21 - Migration to OCaml 5 (Mathias) ## 2024-09-25 - Arvid will be stepping down as merge team coordinator as of next week. The future of the merge team and the new coordinator, if any, will be announced shortly. - What about adding **Marina Polubelova** in the merge team? - (François) (if I can be available) Regarding https://gitlab.com/tezos/tezos/-/merge_requests/15030, Romain questions the integration of Opentelemtry has an observability framework into the codebase. I have written a document about a proof of concept which can be accessed [here](https://docs.google.com/document/d/1il1E5DmCcr7WQzi_Xn3_73hVX8RauWtTXZ0Vm9H9INs). While for the DAL node, this POC is convincing enough to move forwards on this, it may be interesting to synchronise on this topic with other units. I suggest to make a recap of my current understanding of the framework, what it implies and why it may be a good idea to be used in general. ## 2024-09-09 - What about adding **Mattias Roux** in the merge team, at least to focus on **context/irmin/brassaia** code where we need his approval in any case ( @MathiasBourgoin ) - What about adding **Julien Sagot** in the merge team, at least to focus on **teztale** code where we need his approval in any case ( @MathiasBourgoin ) - François: (can't attend the meeting): He did good reviews on my few MRs about teztale and seems able to understand its limits. ## 2024-09-02 - What about adding Paul Laforgue in the merge team (coopters: @picdc and @saroupille) ## 2024-06-17 - RPC deprecation policy ([name=Victor A.]) - [!13723](https://gitlab.com/tezos/tezos/-/merge_requests/13723) - Simple deprecation policy allowing to deprecate and then deactivate (thanks to a sunset date) some RPCs ## 2024-06-10 - let's define rules for when to revert - suggestion paraphrased from our infrastructure engineering manager: - if something is broken - and it looks like there will be no satisfying solution in the next 4 hours - then revert and take the time to discuss a satisfying solution - define broken: has to be a "vote"? - reverting is done in an emergency, cannot wait to have a quorum - a vote shows that people care / are affected - people may not want to get involved and vote - but if we reverted more often people would be used to the process - not symmetric: but have a threshold of how many people agree that it is breaking - easy to re-revert, harder to revert later - may break something for bakers but not for devs so maybe devs would not notice and thus not care and thus not vote for reverting - devs can fix things so if it only affects devs it's less critical - why do users build from source? - there are reasons - but that's not important because the fact is that right now there are users who do build from source - reverting is "temporary" nothing is more permanent than "temporary": how do we move forward after? - being able to revert often also allows to move fast; if we can't revert we have to be extra careful before merging anything - just revert quickly, post a message, have the discussion - conclusion (?): - just follow the usual 2-approval process - but do post a message on slack to start the discussion - the sooner the revert, the easier - should have some guidelines for 90% of cases - who are users - how to move forward with Rust libs - the problem: - `amerge`, `ranlib` etc. may allow to avoid future problems when importing new Rust libs - but if two different toolchains are visible from the PATH, they brick the build - solution: configure script? - other solutions? - already have a script that checks that clang is installed - installing clang installs the LLVM toolchain, usually - the `clang-14` package depends on `llvm-14-linker-tools` - It also recommends `llvm-14-dev` - so you'd end up with that if you compile the rollup kernels - LLVM package is different because it includes utilities beyond clang but also utilities - if you need clang you shouldn't have to install more - in most cases you only need clang, not LLVM - `OBJCOPY` can be used to configure location of this executable but probably not the case for the other 2 - can we configure dune or cargo to not link a given C library? - maybe dune can be linked with the Rust lib instead to see the e.g. `blst` symbols - what about providing a shared object (DLL) for blst? - could work but probably similar issues? - compile the `.so` first, then compile Rust and OCaml with this `.so` for the dependency - requires identifying `blst` and other such libraries to extract them as shared objects - requires vendoring those C libraries too? - many build systems advocate to vendor everything but is that doable? seems hard to make sure we found all of them - similar problems with Alpine and we package some libraries ourselves because of that (to change the flags) - C libraries we use are small and self-contained? easy to vendor? ## 2024-06-03 - Remove requirement to sign commits - let's remove it - because merbocop is no more doc MR needs two approvals, Nic asks if he can self-approve them in that case] - he can, and perhaps we can implement a technical solution later - Discuss the dissolution of the merge team - Put your ideas [here](https://hackmd.io/Yfu-PHYrSam3zW3EUnTuZg). ## 2024-05-28 - Reminder about ideas for the merge team dissolution - Rediscuss the following MT proposals - Pietro Abate - Has lots of skills, some lacunes - Reviews and writes a lot of stuff in the CI/build/release system - Approved - Paul Laforgue - Long experience (although busy with thesis) - Post-pone - Julien Coolen - Lots of reviews, appropriate comments - Sometimes a bit slim reviews, approves without comments etc. - Reminder to Arvid: write the reminder on how to approve MRs. - Improved on approves recently - But lacks a bit of global picture - Post-pone - Gabriel Moise - The previous decision was taken when several pepole having worked with him were not present. Worked on critical parts, but doesn't see the big picture yet. - Currently workin on the RPC on the node. Maybe his inclusion could be revised if Victor (A) think he approved. His MR are a bit light in term of scope (misses global image). - Scope: if we were in a codeowners context, were would he be? A: that's hard to answer, no special area, woudl have to be on a project basis. ## 2024-05-20 @Raphaël P suggest adding the following people to the merge team - Pietro Abate - Julien Coolen - Paul Laforgue Reasoning: These devs have not joined the merge team in the past but it was based on the previous criterion for joining the merge team. Now that the requirements for entry have changed, they definitely fit. They have specific knowledge for specific areas of the codebase and have had bumped approvals for a while now. ## 2024-05-13 - [name=Ryan Tan] Proposal of Gabriel Moise to the merge team. - Mainly worked on the baker and a wide range of L1 backlog tasks - Thorough reviews with good feeback - Consistently prepares good quality MRs that are easy to review and validate - Meticulous work [replacing the legacy bindings](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=merged&author_username=gabriel.moise&search=legacy+bindings) - Example reviews: [!10796](https://gitlab.com/tezos/tezos/-/merge_requests/10795), [!10554](https://gitlab.com/tezos/tezos/-/merge_requests/10554) - Example MRs: [!12103](https://gitlab.com/tezos/tezos/-/merge_requests/12103) - Anecdote: I've been working with Gabriel for the past 9 months. His reviews and MR preparation have consistently been improving. Additionally, I often observe MT members having no issue bumping his approval. ## 2024-04-22 - [name=andrea] > Would it be possible to give approval rights for a specific folder? For context we do not have enough MT members for the Threshold Encryption sequencer, and asking for MT members to blindly bump all the time is going to slow down development. I'd like to give @Michael (and possibly @Julien Coolen) approval rights only for changes that affect the directory etherlink/bin_dsn_node/ (and possibly the tezt folder since we'll need to test the integration extensively). What was decided: - This is no the first time this issue comes up - With monorepoisation it makes sense to have a more formal solution - However, this will take some time. Arvid will come with a proposition in two weeks using either CODEOWNERS or a bot (merbocop, margebot or other). - Quick fix: add Michael Zaikin to the merge team - Arvid will do the technical stuff - Andrea will give him the onboarding information - With the caveat that: - Andrea is responsible for what Michael is doing - Michael should only modify things in the sequencer folder. Etc. ## 2024-04-15 - [name=ole] Ole want's to merge a job to the CI that breaks whenever the nix shell no longer builds. - https://gitlab.com/tezos/tezos/-/merge_requests/12806 (merged), then - https://gitlab.com/tezos/tezos/-/merge_requests/12843 ## 2024-01-15 - [name=arvid] Merbocop -- what to do with it? - [name=lthms] I thought we agreed we would unplug it? - What it does: some static checks, adds doc-only tags, (proposes/updates) reviewers - proust and romain uses the codeowner -- reviewer thing - noisy, mostly ignored - [name=romain] (and Valentin) Registering Tezt tests with `let () =`, and detecting dead code - using mli files - removing register file would be better, rely on side effects instead - [name=arvid] Flake test pipeline - https://gitlab.com/tezos/tezos/-/merge_requests/8355 - [name=arvid] Rust versions - there are two sources of rust - dependencies of octez - kerneles - 3 categories - look at rust-toolchain files - (root is for cotez deps) - kernel has their version - riscV has a slighly newer version - downside: - if we take that max - we enforce everyone to use a late version of rust - would be great to have only one version for kernels - how difficult is it ot have require users ot have the riscV version of rust - using rustup -- basically free - using 2 rust versions ? one bleeding edge and one "stable" - [name=arvid] CI-in-OCaml ## 2023-12-11 - [name=Seb] _needs_ to leave merge-team and Gitlab "owners" group (not TF-funded any more) - Also need to pass on ownership of merbocop (repository + needs to be redeployed somewhere else than Oxhead Alpha's AWS). - Other things to do? ## 2023-11-06 - [name=Killian], [name=Raphaël] - Communication about opam packages: - When to create new packages in the repo? - How to create multi-libraries packages? - Etc. - This is a merge team issue. Current status of opam packages, how to deal with the packages. Recently, small project on reducing the number of packages (~200 -> ~80). too many packages: issues on ocaml/opam-repo repo, (CI etc). the method for reduction was creating sub-libraries and transforming old packages into sub-libraries. now we have a few "container" packages, including e.g. one container per protocol. we now have ~80 pkgs. Changes in manifest. we want to make sure developers understand the guidelines / how to (when to) add new packages etc. - when revieiwng mrs: be mindful about adding opam pakcages. would be shame if the issue reappeared. - ideas: write guidelines, killian as codeowner, doing regular reviews (e.g. at releases), making an issue on merbocop, making a check in the CI? - https://gitlab.com/tezos/tezos/-/merge_requests/10602 ## 2023-10-23 - [name=Andrea], [name=Ryan]. - Can we add custom CI behaviours and pipelines for the `intrusive-profiling` branch? - Prevent merging into master - manual fast-forward merge into branch - Build custom docker images on demand - Other... - Main branch for the reduce block time. Can't merge to master. But they need their own merge flow. Custom pipelines (custom docker images..). - IIUC, the `intrusive-profiling` branch is meant to add project-specific configuration, e.g. pipelines, patches. - Who can merge to non-master branches? - Reminds me of the: Pietro's yes node for tztraffic, François' I-don't-remember-what - [name=Raphaël P] - Proposal to allow unassigned MRs as WIP MRs (implicitly assigned to the author) - Gitlab has this specific semantic (in milestone pages, unassigned = Draft) - WIP MRs are useful - We use "Draft" but it means either - MRs that are not yet ready to review, or - MRs that are being reviewed and have a fixup commit pushed in ## 2023-10-09 - [name=Raphaël C.] [name=Raphaël P.] - Proposal to allow TODO comments to point to an MR (not just an issue) - Useful for stacked MRs where the lower MRs introduce temporary hacks / assumptions and where higher MRs refine the code. - [name=François T.] Can we see this proposition as a shortcut for creating fir the issue, link it into the first MR and then reference it to the second MR? - [name=François T.] To be usable in practice: If you stack two MRs A and B, with B on top of A. The dev must do: 1. open MR B 2. open MR A referencing B 3. modify on Gitlab MR B to reference MR A It does not sound much easier than just opening an issue, no? - [name=François T.] Could we generalize this to other Gitlab elements such as milestones (if it happens to have an interest)? - `(* TODO: generalise generator to other types (done in stacked MR!12121) *)` - [name=François T.] To ease external tools (for the future), maybe it would be better to enforce that the MR number or the issue number appears on the first line. - https://gitlab.com/tezos/tezos/-/merge_requests/9530#note_1477201637 ## 2023-10-02 - [name=Arvid] - Switching to [merged result pipelines](https://docs.gitlab.com/ee/ci/pipelines/merged_results_pipelines.html) to enable [GitLab merge trains](https://docs.gitlab.com/ee/ci/pipelines/merge_trains.html) - Merged result pipelines are necessary for merge trains - We want merge trains to merge faster - We do not necessarily want merge result pipelines though - I suggest we try it out for a week or so and gauge developer confusion - Suggestion from the crowd: make some change of UI so that the switch is noticeable (maybe change title of manual trigger or something more noticeable?) ## 2023-09-25 - [name=Pierrick] - Rody Bozman (@rodi.bozman) as MT - Worked on the EVM Rollup for 9 months (and SCORU before as far as I am aware) - Has an extensive knowledge on the full EVM stack (kernel and proxy, testing) - Reviews thoroughly and owns responsibility on parts of the projects he is confident and knowledgeable on - On the other hand, doesn't approve MRs on subject outside of his area of expertise, and asks a lot of questions to understand them and assess what he understood and what he didn't (and drive on adding more comments) ## 2023-09-18 - [name=Raphaël P] monorepo generalisation latest news: - We didn't make a design document, instead we made [a plan of action (draft)](https://docs.google.com/document/d/1Hlzcw7YeJJjGYw4Zu-dBi4R0ErupDtmk4NWq8hEkgH8/edit?usp=sharing) ## 2023-09-11 - [name=Ole] - Ilya (ilya.peresadin) for MT - Worked on [plenty of MRs](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=merged&author_username=ilya.peresadin) - Dealt with lots of complex work - Also navigated contentious MRs - Reviewed [even more](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=merged&reviewer_username=ilya.peresadin) - Very thorough reviewer - I have very high confidence in his approvals - Areas touched since joining - Protocol (+ env) - Smart Rollups in general - Rollup node - EVM kernel/Etherlink - Sequencing ## 2023-08-28 - [name=Victor D] [name=Alain M] - Where to put injector binary (not merge? src? devtools) - Used for testing scenarios about the injector component of rollup node - Could be released (not ready rn, in the future) for users in their applications - Conclusion: add the injector binary it to the `contrib` directory - [name=Raphaël P] - Quick reminder on the new exception management features of Lwt ## 2023-08-21 Leftover topics that we couldn't decide on because of "quorum" - [name=Thomas L] [name=Raphaël P] - Organising the sources to support "monorepo" - projects that need it: octogram, client-libs (but also existing components such as snoop? tps-evaliation? open-api?) - other directories than `src/`: what do we want? - `protocols/` for protocols - `octez/` to clearly identify what is part of the Octez toolsuite - `devtools/` for binaries that octez developers use during development (benchmarking tools, automatic refactoring tools, etc.) - `contrib/` for non-octez code that we still want to release but possibly not through the same channel/at-the-same-time - `demos/`? - others? - build rules: what `make` makes, what the CI builds, what the CI tests, etc. ## 2023-08-07 - [name=Raphaël P] - Adding kaitai-lib and kaitai_of_data_encoding code into contrib. - Non critical code: will be used to generate de/serialisation primitives in other programming languages - Adding client-libs into tezos/tezos - These are libraries for de/serialising binary-encoded chain data in other languages as well as signing/hashing that data. - Where to add it? How? - [name=Thomas L] - Can we move forward with merging Octogram? - [opam-repository MR is ready](https://gitlab.com/tezos/opam-repository/-/merge_requests/436) - [tezos MR has been updated to address the MT’s concerns](https://gitlab.com/tezos/tezos/-/merge_requests/9575) - Same treatment as the TPS tool (not build by `make`, dedicated Makefile rule, build by the CI though) - where should we put it? - currently in `src/` - could be moved in `tezt/` or `contrib/` ## 2023-07-24 - [name=Valentin] - Pierre-Emmanuel ([@pe.cornilleau](https://gitlab.com/pe.cornilleau)) for MT member. - He has been working on EVM for 7 months, he is been there for a while now (worked a _bit_ on SCORU before). - He's well aware of what he feels confident to approve, and what he doesn't. So I trust him to not approve everything - Put as reviewer for [120 merge requests](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=opened&reviewer_username=pe.cornilleau) on tezos/tezos, I know it's not a perfect metrics but I rather see 120 than 20. - [=>] no objections, will add him? - [name=François] - How to integrate `Octogram` into master? - `octogram` was developed by Thomas L. - `octogram` uses several external dependencies (`dmap`, `jingoo`, `yaml`) - tool to run large-scale experiments à la 1Mtps demo. - ansible for tezt. - scenarios are scripted in yaml, ansible-inspired syntax - general puropose tool not only useful for the demo - should it go into the devtools folder? - do we want it to be installable through opam? - we could add a new docker image for dev dependencies - would be good to see how it affects caching - [=>] - thomas will break apart the MR and merge non-controversial parts. - thomas will investigate the impact on end-users, docker images, CI - - [name=Ole] - Stop merges to master for the Protocol Snapshot - How best to do that? - How long would the freeze last? - Once a decade : > - [=>] Will attempt to do the merge embargo tomorrow, create snapshot review and merge it. We'll do it without any merge embargo unless devops gives us a very convenient way of doing so. ## 2023-07-17 - [name=Andrea] - Gauthier (@gsebil08) for MT member - Gauthier contributed extensively to DAC. - While he lacks context on other areas of the codebase, one of his strengths is that he is always keen to deep-dive and understand new components. - He has extensive experience as a developer, and he contributed several ideas that will help DACs to run smoothly in production (for example, the original suggestion around API versioning was his). - He recently took the lead in driving the setup of a community DAC. - For this reasons, I think he will be a valuable addition to the MT. - Example MR 1: https://gitlab.com/tezos/tezos/-/merge_requests/8644 - Example MR 2: https://gitlab.com/tezos/tezos/-/merge_requests/8653 - Example review: https://gitlab.com/tezos/tezos/-/merge_requests/8032 - Should Marge only add an MR to the processing queue when the CI is green on the MR's current state? - When the CI has not been run manually, Margebot should trigger it and wait for it to succeed before adding the MR to the queue. --> This would be harder to implement, but without this it's too cumbersome - Pro: avoid making every other MR in the queue wait for a CI that's going to fail, when the failure could have been detected by running the CI in parallel without blocking the queue. - Con: run some unneeded CIs, e.g. when the only changes since the last green CI are squashes or typos. - Con: need to manually check when last pipeline is green in order to assign Marge - Con: human race conditions trying to assign Marge (why re-run pipeline yourself when Marge can do it?) - Con: when the queue is empty, causes the CI to run twice before an MR can be merged? - Maybe reminding everyone to run a pipeline before assigning Marge / after each review is enough? ## 2023-06-19 ... ## 2023-06-12 - Proposing Sota ([satos](https://gitlab.com/satos---jp)) from DaiLambda to the merge team. (Nicolas A.) - He's been working around Snoop and gas for 9 months. - He is *extremely* rigourous when analysing bugs, reviewing and commenting. - He is the go-to-guy for analysing bugs in Snoop. - He takes relevant initiatives. - He breaks down work in several merge requests and commits that make sense and are easy to follow. - Example issue analysis: [#5063](https://gitlab.com/tezos/tezos/-/issues/5063). - Example merge request: [!8799](https://gitlab.com/tezos/tezos/-/merge_requests/8799). - Example review: [!9046](https://gitlab.com/tezos/tezos/-/merge_requests/9046) (shows how he explains everything he did to check a request, even small ones). - It's a GO, arvid'll deal with the practicalities and check with RC and Jun - Marge and priorities : should we have a policy as to when some MRs should have higher priorities than other ones ? - Should we have more than 2 levels of priority? - Should be used to fix urgent issues in the CI / packaging (production things) - Should not be used to bump MRs of "normal development" MRs - If we add more levels, when do we stop? - Note: this is the first time we have this issues, it was a perfect storm of deadline + production issues - Would've been better to push back the stabilisation deadline? - Just put a message in the slack channel - Adding priority:low? - The base issue is that the CI is slow - Intra-team stacked MRs could be merged internally and then merged to master. - It's not about quality, it's about urgency - Soft conclusion: add another label `marge-priority:critical`, follow the situation. Also have the discussion tomorrow in the protocall. Would be nice to be able to push stabilisation period to avoid such rushes. ## 2023-06-05 - Adding Ryan Tan (Gitlab: @ryan.tan3) to the merge team For the last 6 months, Ryan worked extensively on the DAC project. He has gradually taken more and more responsibility, and is currently driving the first release of DAC. He writes very good code, and has good knowledge of the main libraries that we use. He also has knowledge of how the protocol works (all trili's engineers undergo a challenge where they implement a global counter and associated manager operations, when they join). For these reasons, I recommend him as a member of the MT. - Example MR: https://gitlab.com/tezos/tezos/-/merge_requests/8747 - Example review: https://gitlab.com/tezos/tezos/-/merge_requests/8883 - All merged MRs: https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=merged&author_username=ryan.tan3 - Conclusion: Will invite - A step of the CI now depends on a private docker image - A tool needed to build kernels - Pushed into our private AWS docker repo - Valentin: no specific reason to make private - Boubacar: will talk to devops. ## 2023-05-22 - Yann: Adding Victor Dumitrescu in the merge team - Worked on the separate crypto libraries before their inclusions - Good reviews, good work - Will make our lives easier now that crypto libraries are merged - May not have a good global view of the code base, may not be aware of best practices, etc - But is that even a relevant critera in these days, given the size of the code base - It's a yes - Yann: Is it good practice to use a fixed RANDOM_SEED for PBT in the CI? - On the one hand, I understand the need for predictability and working around flakiness... - But on the other hand, amortizing random generation over CI runs seems a better practice to find potentially tricky cornercases. - Couldn't we have a systematic process for unpredictable/flaky tests as in: - Deactivate the test (or provide a temporary RANDOM_SEED that makes it work fine) - Create a following issue assigned to the owner of the tests - As long as the variability in execution is low and the tests is not known to be flaky, I think it's fine - Another option: We could have a separate CI that runs random seeds + longer runs. We could do some automatization do automatically swithc up seeds and create MRs and the like. Potentially we could use Long test here. - We need more process to handle the discovery of failing tests. - Does putting the PBT in a separate CI make it easier to ignore the tests? - Adding a "margeblock" when the scheduled pipelines are failing? - There is a plan betwen Romain & Boubacar to deal with flaky (long?) tests. They have a process set up: - Detect - Assign a team per test - Configure LTF to put a webhook per team - Idea: "responsabilize" teams - Could also - be applied to regular tests. - What to do now / conclusion: - PBT, random or not: - In the CI: deterministic - In a separate CI / scheduled pipeline: non-deterministic - How the seed is set: - Currently ad hoc, ICs put hard coded seeds in the code - We should go through the environment variable instead - For non-QCheck tests: consider Qcheck, or just read the same environment variable as QCheck - For developers: - The social process for how to reduce flakiness - Follow the work of Romain & Boubacar, extend to other tests. - Mathias B. in the merge team? - Is more and more involved in incident management - Being MT helps doing that - However, hasn't been (AFAIK) a very active IC nor reviewer lately - Has good global understanding - Has been demonstrating rigour in incident mgmt. - It's a yes. ## 2023-05-15 - Adding Marc Beunardeau in the merge team - useful for the crypto part of the `tezos/tezos` repository (which was outside before), in addition to Marco, Antonio (and Victor D. later?) - testimonies: - "Marc is clearly a senior engineer, he knows the protocol code quite well, he's an expert on the crypto part of the codebase" - "I trust Marc to not approve outside of his range of expertise, he knows what he knows, he's a good candidate for the merge team" - "I agree. We had discussed him years ago, conclusion was that he was very good at crypto but not necessarily a very good OCaml engineer. But I'm not against" - "You're right, he's not an OCaml expert although he writes pretty decent OCaml code. And he was able to find security issues, both in implementation and usage: good value to ensure the quality of the code, with focus on security. And he would not push for weird OCaml solutions." - => YES - TODO: who will add him / tell him / give him the rules? - Yann is in charge (to either do it or to delegate to e.g. Raphaël Proust) - Note: Marco technically left the merge team. - said he was going to focus on the crypto part - was after the release of sapling, so a while ago - Marco was focusing on the crypto directory. Now everything moved back to `tezos/tezos` so would make sense for him to come back to the MT. Lots of progress on the CI in particular since then. - Allowing only 1 MT approval on some parts of the code? - Or allowing self-bumping if we know an MR touches a part of the code that is not critical? - What would be a good definition of "not critical"? Something like "cannot impact Mainnet, and is not used to test things that can impact Mainnet"? - Good examples of use cases? (ask Raphaël C.) - This is already a rule we have for the documentation. This point was raised a few months ago. We don't need to have a long discussion about what is critical or not (too many cases). But we can at least find some places that we consider not critical. - the `devtools` directory is a good example - Context: Polyrun: we are asked to move the code generator into `tezos/tezos`. But this is typically non-critical code. Goal is not to review the generator but the generated code. - close to what snoop is doing - we commit the generated code - what is Polyrun? - code generator: takes a description of the Michelson language (which is part of the Michelson documentation) and converts it to OCaml code that we include in the protocol, and also into Rust code. The clearly critical part is the generated OCaml code. This is a piece of code that has contained many bugs / provoked many incidents in the past. The generator itself is not critical in the sense that we didn't use it until it was developed. We've been using for a few months, it helped refactoring. Plan to continue using it to do large refactorings. - It's not even on `tezos/tezos` yet so it's obvious it's not on the critical path. - In the future, we can expect both the generator and the generated code to be reviewed, but with different levels of attention. - The criticality of the generated part is clear and will probably remain clear. See e.g. snoop. - Should we actually merge it into `tezos/tezos`? - it's not released - moving it to `tezos/tezos` will take some time - not sure of the benefits - benefits: - avoid any friction for components that are strongly coupled (here, the code generated by Polyrun) - better ownership of the code (otherwise, as a group we're not necessarily aware that Polyrun even exists; and, as a group, we want to maintain its quality) - maintain transparency - unified release process (N/A for Polyrun) - sharing the infrastructure (CI) (for Polyrun: we do formatting checks, and the CI checks that the generated files are actually the generated ones) - TeCI? TeCI is outside of `tezos/tezos`'s CI. - inconvenients: - repo gets bigger - CI runs for everything even if we only modify Polyrun (and vice-versa) (but this could be improved) - for Polyrun, current CI takes <1 min, on `tezos/tezos` would take much longer - in general we need a better monorepo architecture - Opinion: tests have the same criticality as the piece of code they test. - For doc, we have a deterministic way to figure out (done by Merbocop, who automatically approves). - If we add documents listing which components are critical and which ones are not critical, it would be great to conduct audits of the source code of the Octez distribution. (External security audits.) Having a least of what is critical or not would be useful. - Conclusion for the initial question: yes. We already do that for the doc. Decision will be on case-by-case basis to decide which parts are critical. - Conclusion for the whether Polyrun is critical: it's not critical. (Generated code is, though.) - Author more happy to move it if it can be considered non-critical. - Author will move Polyrun into `tezos/tezos`. - Patch Merbocop to declare the Polyrun directory as non-critical? - Or `devtools`? - `benchmarks-tools`: tools used to run the benchmarks twice a week automatically. - If we make a mistake we could fail to see gas regressions. - Conclusion: this is critical. Hence `devtools` is critical. Move it away? - `gas_parameter_diff`: same. - Conclusion: it is critical. - `non-critical` directory? nobody is against - Conclusion: - Raphaël will create `non-critical` directory - Maybe Merbocop will be patched to auto-approve MRs that only modify this directory (and/or other non-critical parts like `doc`) - Raphaël will move Polyrun there. ## 2023-04-17 - Wishing Arvid a speedy recovery ## 2023-03-20 - Antonio L. in the MT - works on rollups - anonymous: worked with Antonio multiple times, made good reviews, makes perfect sense for Antonio to join the merge team. Was one of the first engineers at NL to work on rollups (TORU). Worked on epoxy etc. Integrated in the protocol. Currently working on smart rollup node. Has good overview of the codebase. - other worked with him but not here today => ask them offline (Alain and Ole in particular) - TODO: Valentin will ask in the merge team channel ## 2023-03-06 - `kernel SDK` release pipeline in `tezos/tezos` (@emturner) - https://docs.google.com/document/d/1kR_YVfslo4qHi961uURuOCIgS8_Wng1rqNULZ3UX-jU/edit?usp=sharing - release crates - lightweight release process decouples from octez release process - should be able to trigger patches and to the crates - technically, all you do is run a command with a private token - sync with DevOps/Killian - do some experimentation and come back with a proposition, coordinate with release manager. - automatic tagging of Mondaynet commits? (@lthms) - what is a mondaynet commit ? - some kind of autotagging? (a scheduled job?) - should be easy to do in a scheduled job (ref. doc job that does automatic git stuff in the CI) - some kind of tagging scheme: `MONDAYNET-YYYY-WXX` or sth - - CI time is increasing again (@lthms) - Some guerilla work in progress: - https://gitlab.com/tezos/tezos/-/merge_requests/7883 - https://gitlab.com/tezos/tezos/-/merge_requests/7404 - Ask boubacar to set up pipeline monitoringuser - new members - no proposals - meeting time? (@arvid) - ask tokyo people what they think ## 2023-01-30 - `tezos/tezos` as a monorepo welcoming the EVM Rollup and the Crypto codebase (@lthms and @yrg) - CI: will it be slower? - This needs work, we can't afford to make it slower - tests can run in parallel so it shouldn't increase too much - Maybe time to consider dune cache? - Current status: tezos/tezos already _is_ a monorepo to some extent (e.g., error-monad is a library hosted in the same repo as the binaries) - Scope: - want to add: crypto libraries, EVM rollup, possibly more later - not everything (e.g., data-encoding?) (some project may not benefit from the added paperwork of dealing with the tezos/tezos process (CI, reviewing, margebot, sequential histtory of commits, etc.) or may have some CI tests that are not portable into tezos/tezos (e.g., testing against multiple versions of ocaml)) - Goal: to avoid friction during development of rollups - Releases: - Need to be able to release libraries separately from the release cycle of octez? - maybe difference to be made between stable releases and making usable tools available to users - Rust: - need more realistic integration tests - essentially need to have more control over the software installed in docker images and such, and more recent versions of other toolchain - need to be done with care for CI time (work is already ongoing) (can things be pre-installed in docker images?) ## 2023-01-23 - Enforce names of private branches on tezos/tezos? - Pro: clean up - Cons: more effort and restrictions - List stale branches removed, remove stale branches regularly. - Point to `contributing.rst` - Single approval on non-critical parts (point added by RC) - Currently documentation changes require a single approval but other changes to some non-critical parts still require two of them. - IMO, we should: - identify these non-critical parts of the code base, - lower the approval requirement for them too, and - ensure that these non-critical parts cannot affect the quality of the critical parts without us noticing. - Examples of non-critical stuff: - `src/proto_demo*` - Configuration of the formatters - GitLab (merge description, issue?) templates - `devtools/`? - `emacs/`? - `src/tooling/`? - Yes - 1. Define and agree on list of folders - 2. Manually auto-bump - 3. Ask merbocop developers to implement feature - 4. Communicate and document the change - 5. Look into formalising this using a GitLab native approach - Moving more stuff to devtools? - Just be careful about what is critical and not - Suggestion to add Lucas Randazzo as a MT member (NA) * Lucas has produced high-quality MRs and reviews for gas-related stuff. * He is rigorous, I trust his approval. He likes to question processes. * [Example of a designed MR (!6892)](https://gitlab.com/tezos/tezos/-/merge_requests/6892) * [Example MR for review (!7052)](https://gitlab.com/tezos/tezos/-/merge_requests/7052#note_1196524811) * [Authored MRs](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=merged&author_username=lrand) (32 merged) * [Reviewed MRs](https://gitlab.com/tezos/tezos/-/merge_requests?scope=all&state=merged&reviewer_username=lrand) * -> Accepted - proposal for OMT changes (related to previous discussion): - comments: - this is a first-step proposal aiming to make an incremental change towards more specialisation-based review/approval; further changes would be made later - this first-step aims to gather more concrete data re: expertise coverage in the merge-team, possible lacks in the coverage - once we have information about coverage we can decide whether to and how to split the OMT into smaller more specialised parts - proposal: - make a three-way mapping of - OMT members - source-tree hierarchy - projects - (e.g., make sentences like “Dev McDevelopper has expertise on `src/lib_foo/*`”, “`src/lib_foo/*` falls under the responsibility of the project BAR”, “project BAR is managed by Manager McSomething”, “project BAR also has so-and-so working on it”) - result on merbocop vote: remote threads ## 2023-01-16 - Future of the OMT (with guest Fedor) (see subsection Future of OMT below) - Do we have OMT-consensus on <https://gitlab.com/oxheadalpha/merbocop/-/merge_requests/98> (@smondet) - Enforce names of private branches on `tezos/tezos`? <details><summary>Future of the OMT</summary> ### Future of OMT: #### A short presentation for context #### Going forward - What does the review and merging process become? - How do we keep all the dev actors (form different companies) involved in this? - What does the cross-company communication become? - Who is responsible for maintenance of this process? (Including merbocop, margebot, the CI.) - Who is responsible for the evolution of this process? (Including changing the CODEOWNERS file, changing the rules for approval, changing the tools, etc.) - If we change the merging process, how do we do the transition? - Do we organise training for the review process? ##### A few proposals </details> ## 2023-01-09 - Happy new year! - Merge request templates (@vch9): - https://docs.gitlab.com/ee/user/project/description_templates.html - https://medium.com/@alex.roh/gitlab-how-to-automatically-add-code-reviewers-to-your-merge-requests-bcdefc2a8599 - One use-case I see: I set-up a template for the project I'm assigned to, and I can easily ask reviews from my coworkers on the project. For example [here](https://gitlab.com/vch9/tezos/-/merge_requests/9), I put `/assign_reviewer @yrg @lthms @picdc @sribaroud ...` which will assign reviewers at merge request creation time. The only drawback is if I put `vch9` in the `assign_reviewer` list, it will put me as a reviewer of my own merge request. - It could also be nice for new-comers to easily have reviewers based on the feature: node, proto, build..etc. - Membership discussion (@pirbo) - The externalization of Tezt ## Older meetings - [2022-01-31 to 2022-12-12](https://hackmd.io/q3LP40leSv-hfiFqmkqj1Q) - [2021-03-08 to 2022-01-31](https://hackmd.io/2EMl8hpXQ2m6HpqaeAEU2Q) - [2020-07-13 to 2021-03-08](https://hackmd.io/-KERABhGQLOgHi1LPeq5Ng) - [Big Bang to 2020-07-06](https://hackmd.io/MofecNp_TZyexeWJy4NfTA)

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.