Members:
* Jun Furuse [@camlspotter](https://gitlab.com/camlspotter) (DaiLambda, project coordinator)
* Kazuhiro Kobayashi [@kazkob](https://gitlab.com/kazkob) (DaiLambda)
* Hiromasa Saito [@taiseiKMC](https://gitlab.com/taiseiKMC) (DaiLambda)
* Nicolas Ayache [@naih](https://gitlab.com/naih) (Nomadic Labs)
* Pavlo Khrystenko [@Zett98](https://gitlab.com/Zett98) (Marigold)
* Sota Sato [@satos---jp](https://gitlab.com/satos---jp) (DaiLambda)
* Zaynah Dargaye (Nomadic Labs, unit manager)

[Meeting](https://meet.google.com/uqe-ndea-tgm): Wednesday 10:00 CET

## Resources

* [How to trigger a full benchmark run](https://hackmd.io/i9dzz4ZmSxSn5z8WFlKFKA?view)
* [How to read benchmark results](https://hackmd.io/B87Y_Dy0Rs2fXk05vB-2Og?view)
* [Gas parameter updates (old)](https://hackmd.io/fH2qY7kVTfGPUyQ8a2N8_g?view), [Gas parameter updates (new)](https://hackmd.io/wW3tAToiQuaJx8A7gZb0Og). **The two documents need to be merged.**
* [Documentation about benchmark scripts (benchmark run, regression detection)](https://gitlab.com/tezos/tezos/-/tree/master/devtools/benchmarks-tools)
* [Snoop: user tutorial](http://tezos.gitlab.io/developer/snoop_tutorial.html)
* [Snoop: developer's portal](http://tezos.gitlab.io/developer/snoop.html)

## Active milestones

### [Gas effort meta-milestone](https://gitlab.com/tezos/tezos/-/milestones/236)

### [Add Snoop generated files to lib_protocol](https://gitlab.com/tezos/tezos/-/milestones/237) (Jun, Pavlo, Hiromasa)
* **On-going.**
* Goals:
* **Adapt break down items following the document below.**
* **Tech based goals+plans: https://hackmd.io/5WRLMWxoQWGX_cGd5drrjg**
* [x] Snoop: Removing local model names from codegen https://gitlab.com/tezos/tezos/-/issues/6043
* [x] Snoop: Split benchmarks with more than 1 models https://gitlab.com/tezos/tezos/-/issues/6042
* [x] Snoop: Codegen using the destinations https://gitlab.com/tezos/tezos/-/issues/6044
* [x] Snoop: use auto build in tezt-suite https://gitlab.com/tezos/tezos/-/issues/6048
* [x] Snoop: Move the cost defs in Proto to xxx_costs.ml https://gitlab.com/tezos/tezos/-/issues/6045
* [x] Snoop: Fix the codegen bugs https://gitlab.com/tezos/tezos/-/issues/6049
* Pavlo and Hiromasa will mostly work on this milestone this week (08-16)
* *Nicolas will mostly work on this milestone this week (08-30) to review [!9847](https://gitlab.com/tezos/tezos/-/merge_requests/9847).*

### [Memory allocation benchmarks](https://gitlab.com/tezos/tezos/-/milestones/238) (Jun, Kazuhiro)

https://docs.google.com/spreadsheets/d/1J2SSYw8Q1Mmb_mRt9MQMX-JRcTEZPoK7QvVTOrBwQCk/edit#gid=0

100% of michelson opcodes are now with memory allocation models, and being reviewed!

* **On-going.**
* Goals
* [x] Allow Snoop to measure memory allocation in a benchmark.
* [x] Adapt code generation to account for memory allocation, merging it with runtime.
* [x] Parameter inference for memory allocation models
* [x] New regression LASSO + shift up
* [x] Implement models for the memory allocation benchmarks.
* All models are done and being reviewed
* [x] Detect regressions of memory
* *Kazuhiro and Sota will mostly work on this milestone this week (08-16).*

### [Document gas-related processes](https://gitlab.com/tezos/tezos/-/milestones/239) (Nicolas)
* **Finished :tada:**
* Goals:
* [x] Document how to update gas values.
* **Accepted**: https://hackmd.io/fH2qY7kVTfGPUyQ8a2N8_g?view.
* How to update the parameters and cost functions have changed.
* Jun will update this document in another MR or as a new version.

### [Fix imprecise benchmarks](https://gitlab.com/tezos/tezos/-/milestones/240) (Hiromasa, Sota)
* **On-going.**
* Goals:
* [x] Finish the items in the spreadsheet.
* [x] Define a subset of benchmarks to fix.
* [x] Analyse and propose a fix.
* [x] Implement fixes.

### [Fix IO benchmarks](https://gitlab.com/tezos/tezos/-/milestones/241) (Hiromasa, Jun)
* Started
* Goals:
* Consideration updated 2023-08-23: https://hackmd.io/evWPScUHRvCIVfCIRPDV7g?view
* [x] Reinstate or remove unused IO benchmarks.
* [x] analyzing the situation: https://gitlab.com/tezos/tezos/-/issues/6069
* [x] Make the benchmarks work again
* [x] Imporove the IO benchmarks considering the OS disk cache
* **Centralized list of all benchmark issues: [#5640](https://gitlab.com/tezos/tezos/-/issues/5640).**
* Jun is going to work on this 2023-08-23

### [Gas parameter updates](https://gitlab.com/tezos/tezos/-/milestones/242) (everybody)
* **Recurring activity.**
* Goals
* Analyze benchmark results.
* 2023-08-19 result: no big issue found
* 2023-08-26: failed somehow!
* Update gas parameters when their value changes.

## [Minutes from 2022](https://hackmd.io/kaD-o_RPS7OSxfOA_Ch86Q)

## 2023-09-27

- Periodical benchmarks
- Lots of alerts of alloc parameters
- Do not worry. Their increase x4 is expected.
- IO benchmark (Low priority)
- [!9991](https://gitlab.com/tezos/tezos/-/merge_requests/9991) for the new IO benchmarks.
- Should be merged, but with a lower priority: it does not change the gas parameters.
- The storage cost parameters must be increased but not in Q3.
- [Slides](https://tezos-dev.slack.com/files/UCQP7FLD7/F05TP0XLGLX/slides.pdf)
- Memory allocation benchmark (High priority)
- gas_parameter.json [!10157](https://gitlab.com/tezos/tezos/-/merge_requests/10157)
- `Ediv` and `And_int_nat` have strange intercept values
- We may be able to fix them (Sota) [!10259](https://gitlab.com/tezos/tezos/-/merge_requests/10259)
- charge at least 10 milli gas [!10206](https://gitlab.com/tezos/tezos/-/merge_requests/10206)
- Use `cost_*_synthesized` in the protocol (Jun)
- Add aliases `let cost_* = cost_*_syntheized` to `Michelson_v1_gas_costs`
- Random seeds (High priority)
- Seed must be recorded [!10160](https://gitlab.com/tezos/tezos/-/merge_requests/101606) (Hiromasa)
- Seed is FIXED in `tezt/test/main.ml`. We should use random seeds
- Easy to do, but with a risk of finding unstabilities. Not the work for Q3.
- Documents: will be finished in the 1st week of October
- How to update gas parameters [!10143](https://gitlab.com/tezos/tezos/-/merge_requests/10143)
- How to run the new IO benchmarks (Jun)
- Memory allocation benchmarks (Jun)

## 2023-09-13

* Benchmark warnings
* Some mitigations are considered by Nicolas and Pavlo? Details?
* Not really warnigs but caused by local model renames
* Codegen
* gas_parameters.json is now in the code!
* CI test for codegen is on review
* More functions will be auto-gen'ed but will change gas slightly
* Memory allocation
* All the models are now in the code!
* The results seem to be almost ok. Only 3 anomalies, and their fixes are being considered.
* Many alloc costs < their runtime counterparts
* Ast optimizer
* On review
* Should simplify the syntheized gas cost definitions of runtime and allocation
* IO benches
* The latest result suggests x2 for reading and x80 for writing.
* x80 for writing is apparently overestimating but no good way to ease it for now. Any ideas?
* Afraid that there is no enough time to update the storage costs in this Q.
* More discussions and observations will be required in Q4.

## 2023-09-06

* [x] We need to add the right to access the reference machine to Pavlo.
* 1 machine is prepared
* [x] Nicolas needs to configure the other.
* [x] Need to investigate the last benchmark failure
* Not real alerts but due to the local model name change
* Codegen: will make an MR for more auto-generated cost funcitons. side-effect: change regression data
* Memory allocation benchmarks: all the michelson opcodes are now with memory allocaiton modles. Only 1 MR left for it.
* Kazuhiro and Sota will check the output of the memroy allocation cost funcitons and report if there are something really strange.
* Soon we will switch to the synthesized gas cost functions. Kazuhiro has an MR for it reviewed.
* IO benchmarks: the rewrite is done. Got some numbers scary.
* Need slides to be publicly availalbe

- Jun : go back to the codegen
- Pavlo : Codegen. Review. MR for model changes.
- Hiromasa : Codegen too. Reviewing IO MRs
- Sota : fix imprecise benchmarks
- Kazuhiro : Memory allocation benches

## 2023-08-30

I MUST FILL IT

## 2023-08-23

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-08-16, Jun): Jun has analyzed. No issue.
* 2023-08-22 benchmark has failed! => a small bug was found. Rerunning now. => Benchmark done!!
* Jun will check the results.
* Q3 projects
* IO benchmark: Jun has started investigating it.
* Codegen:
* Most of the parts are done and many MRs are being reviewed.
* Investigated the runtime+memory syntehsized code. We need further optimizations of the code. Jun has written a prototype: https://gitlab.com/dailambda/tezos/-/tree/jun@max10?ref_type=heads
* Memory allocation: many models are added and being reviewed

## 2023-08-16

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-08-19, Jun): Jun has analyzed. No issue.
* Q3 projects
* Jun has started investigating the IO benchmark milestone
* Sub-milestones
* Codegen:
* Most of the parts are done and many MRs are being reviewed.
* Investigated the runtime+memory synthesized code. We need further optimizations of the code. Jun has written a prototype: https://gitlab.com/dailambda/tezos/-/tree/jun@max10?ref_type=heads
* Memory allocation: many models are added and being reviewed

## 2023-07-19

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-07-15, Jun): Jun has analyzed. https://tezos-dev.slack.com/archives/C04HZHR11DW/p1689472829062199
* Next results (2023-07-19): Jun is analyzing
* Q3 projects
* **Milestones created.**
* **Agile development.**
We have changed our priorities and focuses 2 milestones in the first half of the quarter:
* Generate cost functions
* Pavlo, Hiromasa, Jun, hope to finish 2023-08-10
* Memory allocation benchmarks:
* Kazuhiro, Sota, Jun, hope to finish 2023-08-10
* Fix imprecise benchmarks:
* Reinstate IO benchmarks
* Remember to keep milestones/issues/requests small and up-to-date.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-07-12

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-07-04, Jun): ?
* Next results (2023-07-11): **:arrow_right: Kazuhiro** to analyze.
* Timelock issue: [!9302](https://gitlab.com/tezos/tezos/-/merge_requests/9302) (merged).
* Some timelock gas values might be impacted, let's keep that in mind during the next run analysis.
* Q3 projects
* **Milestones created.**
* **Agile development.**
* Memory allocation benchmarks:
* Parameter inference for memory allocation models (Kazuhiro?)
* Adapt code generation to account for memory allocation (Jun?)
* 15 days
* Fix imprecise benchmarks:
* Pick 2 benchmarks to fix
* Analyze
* Fix
* 15 days (one each for Pavlo and Sota?)
* Reinstate IO benchmarks (Hiromasa?)
* Remember to keep milestones/issues/requests small and up-to-date.
* **This plan does not fit the current status of projects.**
* Jun and the team will establish new goals.
* Each goal should be short-term goals (deliverables every 2 weeks for instance).
* Each goal should involve 2/3 people, so that knowledge is spread and people don't feel alone on their project.
* A goal can simply be an item of the current milestone/work break down.
* **Jun will coordinate the gas effort from now on.**
* Everyone from DaiLambda is welcome to participate to the protocol unit meetings, but considering the time is not ideal, there's no obligation of course!

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-07-03

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-06-20, Sota): :heavy_check_mark:
* Next results (2023-07-04): **:arrow_right: Jun** to analyze.
* Timelock issue: [~~!8845~~](https://gitlab.com/tezos/tezos/-/merge_requests/8845) replaced by [!9302](https://gitlab.com/tezos/tezos/-/merge_requests/9302).
* **Merged.**
* **Some timelock gas values might be impacted, let's keep that in mind during the next run analysis.**
* Q3 projects
* Create milestones. **:arrow_right: Nicolas**
* Remember to keep milestones/issues/requests small and up-to-date.
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-06-26

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-06-20, Pavlo): :heavy_check_mark:
* Next results (2023-06-27): **:arrow_right: Sota** to analyze.
* Timelock issue: [!8845](https://gitlab.com/tezos/tezos/-/merge_requests/8845).
* Reviewed.
* **There might be an issue for some Michelson op-codes not being correctly handled: [#5943](https://gitlab.com/tezos/tezos/-/issues/5943).**
* **Jun is experimenting gas consumption on Nairobi. So far, nothing too bad to report.**
* **Q3 projects**
* **Discussed with managers, here are the conclusions.**
* Add Snoop generated files to lib_protocol.
* The same way `michelson_v1_gas_costs_generated.ml` was introduced, with a manually edited file next to it if needed.
* Add a display info for cost function command.
* **Confirmed.**
* *Jun and Pavlo?*
* Memory allocation.
* **Confirmed.**
* *Jun and Kazuhiro?*
* Internal documentation.
* Write a note that explains how we handle regressions after a full benchmarks run, and what to check before injection.
* **Confirmed.**
* *Nicolas?*
* Improve benchmarks.
* Fix a defined subset of imprecise benchmarks (using scores). **Confirmed.**
* Fix IO benchmarks. **Confirmed.**
* Check that the verifications performed by benchmarks not generating code still hold. **Postponed.**
* *Hiromasa and Sota?*
* Snoop technical debt.
* Remove tags and local model names dependency in Tezt/Snoop.
* **Postponed.**
* Benchmark interface simplifications.
* Finish migrating benchmarks to simpler signatures.
* **Postponed.**
* Verify gas compositionality.
* **Postponed.**
* Machine independence.
* **Postponed.**
* **Create milestones. :arrow_right: Nicolas**
* Remember to keep milestones/issues/requests small and up-to-date.
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-06-19

**Welcome to the new member of the team: Hiromasa :tada::wave:**

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-06-13, Nicolas): **run was not good, analyzing Friday's (2023-06-16)**: TODO!
* Next results (2023-06-20): **:arrow_right: Pavlo** to analyze.
* Timelock issue: [!8845](https://gitlab.com/tezos/tezos/-/merge_requests/8845).
* Reviewed.
* Brainstorming projects for Q3 and what we'd like to work on.
* Synthesize propositions, propose to the team, and then the final list to the managers. **:arrow_right: Nicolas**
* **Here is the list discussed on Slack.**
* Add Snoop generated files to lib_protocol. **Confirmed.** *Jun and Pavlo?*
* The same way `michelson_v1_gas_costs_generated.ml` was introduced, with a manually edited file next to it if needed.
* Add a display info for cost function command.
* Memory allocation. **Confirmed.** *Jun and Kazuhiro?*
* Snoop technical debt. **Postponed.**
* Remove tags and local model names dependency in Tezt/Snoop.
* Internal documentation. **Confirmed.** *Nicolas?*
* Write a note that explains how we handle regressions after a full benchmarks run.
* Benchmark interface simplifications. **Postponed.**
* Finish migrating benchmarks to simpler signatures.
* Improve benchmarks. *Hiromasa and Sota?*
* Fix a defined subset of imprecise benchmarks (using scores). **Confirmed.**
* Fix IO benchmarks. **Confirmed.**
* Check that the verifications performed by benchmarks not generating code still hold. **Postponed.**
* Verify gas compositionality. **Postponed.**
* Machine independence. **Postponed.**
* Remember to keep milestones/issues/requests small and up-to-date.
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-06-12

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-05-30, Kazuhiro): ?
* Next results (2023-06-13): **:arrow_right: Nicolas** to analyze.
* Open chest issues became a timelock issue.
* **Reviewed.**
* **Brainstorming projects for Q3 and what we'd like to work on.**
* **We can each write down what projects we think could be created for Q3, in priority order.**
* Jun:
* https://hackmd.io/ylHqnTSqTFKrBCP91a2tIQ
* Kazuhiro:
* TODO
* Nicolas:
* Working towards machine independence.
* Verify gas compositionality (macro benchmarks).
* Memory allocation benchmarks.
* Benchmark interface and UX simplifications.
* Improving gas-related processes.
* Full runs update the cost functions of benchmarks with regressions (i.e. directly in the protocol).
* Document internal processes (notably gas reviews after a run and gas updates).
* All the other issues that are tagged as technical debt or in the backlog.
* Fix imprecise benchmarks.
* Gas parameter updates.
* I'd like to work on the documentation in priority so that we know anyone can pick up the projects if we leave.
* Pavlo:
* Simplify writing benchmarks for developers
- simpler generator definition
- simpler benchmarks definitions
* Snoop technical debt:
- Make model names independent
- introduce execution tracing and make library FS independent
- Introduce free monad to record execution traces and store them in order to save them in case of error and use Storage abstraction for read write operations.
- Benchmark_meta DSL.
- introduce shared workload generation
- allow codegen in nested files.
* Benchmarks parallelisation:
- important for quickly testing assumptions or experimental changes. total run would take 6 hours instead of 14 hours(as of now)
* Sota:
* Fix unstable benchmarks.
* Split non-interpreter gas-cost functions to separated files.
* Synthesize propositions, propose to the team, and then the final list to the managers. **:arrow_right: Nicolas**
* Remember to keep milestones/issues/requests small and up-to-date.
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-06-05

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-05-30, Jun): :ok:
* Next results (2023-06-06): **:arrow_right: Kazuhiro** to analyze.
* Open chest issues: unexpected cost variation.
* **After a few changes, we found that the intercept benchmark was not correctly designed and messed inference. We'll simply drop it for now.**
* A potential gas issue has been flagged. Nicolas will share the link.
* **We had feedback from experimented people in this area: it happens every now and then :shrug:**
* **Homework for next week:**
* let's each have a look at all the opened issues and requests, and try to fit them into categories/topics;
* let's also have a first thought on their respective priority;
* we'll discuss the lists during next week's meeting;
* this will give us a first draft of potential milestones for Q3.
* (We can each put our list here.)
* **:arrow_right: All**
* Remember to keep milestones/issues/requests small and up-to-date.
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-05-30

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-05-23, Nicolas): :heavy_check_mark:
* Next results (2023-05-30): **:arrow_right: Jun** to analyze.
* **Open chest issues: unexpected cost variation. Analysis in progress with the crypto team.**
* **A potential gas issue has been flagged. Nicolas will share the link.**
* **Remember to keep milestones/issues/requests small and up-to-date.**
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-05-22

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-05-16, Sota): :heavy_check_mark:
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* Next results (2023-05-23): **:arrow_right: Nicolas** to analyze.
* **Timelock issues [!8494](https://gitlab.com/tezos/tezos/-/merge_requests/8494): merged, but we now have an issue with `Open_chest`.**
* **Remember to keep milestones/issues/requests small and up-to-date.**
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-05-15

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-05-09, Pavlo): :heavy_check_mark:
* **(Fix landed: [!8662](https://gitlab.com/tezos/tezos/-/merge_requests/8662).)**
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* Next results (2023-05-16): **:arrow_right: Sota** to analyze.
* **Timelock issues [!8494](https://gitlab.com/tezos/tezos/-/merge_requests/8494): reviewed and approved, waiting for action from Raphaël T.**
* **Remember to keep milestones/issues/requests small and up-to-date.**
* Stakeholders want more transparency on our progresses, so let's focus on finishing things.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-05-09

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* **Friday's run failed (encoding issue with `IUnit`).**
* **Fix in progress: [!8662](https://gitlab.com/tezos/tezos/-/merge_requests/8662).**
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* Next results (2023-05-10): **:arrow_right: Pavlo** to analyze.
* **Timelock issues: Jun reviewed.**
* **Code generation is being added to the periodic run.**
* **In the future, it could be good to have all generated files placed in a specific sub-directory of `lib_protocol`.**
* Remember to keep issues/milestones up-to-date.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-04-24
* Recurring benchmark:
* last Friday went well, Jun found no prob,
* Kazuhiro found something strange but ok because due from an update
* Other alarms seems ok. Let's see tomorrow
* HaCl* implementation version change, we may have significant changes if it is merged
* ~~Pavlo~~ will handle (Tuesday's run 2023-04-25) handled by Kazuhiro
* Sota wanted to run his own branch on the reference machine, but he is not sudoer (should we a special access for Gas team member?)

## 2023-04-17

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* **Tuesday's run failed (chest key issue).** Wednesday's run (2023-03-12, Sota): :heavy_check_mark:
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* Next week's results (2023-04-18): **:arrow_right: Jun** to analyze.
* **When Jun or Nicolas are away and we're missing a merge team member's approval, let's advertise our requests on [#mr-advertizing](https://tezos-dev.slack.com/archives/C02G6333NEN).**
* Remember to unassign yourself from issues you're not actively working on.
* And keep issues/milestones up-to-date.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

- Timelock gas benchmarking, Timelock expert Raphael Toledo, Jun will ask to Marco or Marc Bernardeau

## 2023-04-11

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates.
* Tuesday's run (2023-03-04, Pavlo): ?
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* Next week's results (2023-04-11): **:arrow_right: Sota** to analyze.
* **Next injection around April 13th.**
* Let's make sure runs will be OK.
* **Milestone migration: [%204](https://gitlab.com/tezos/tezos/-/milestones/204).**
* Let's unassign ourselves from issues we're not actively working on.
* The issues reported in the milestones are the priority, let's try to keep them up-to-date.
* Don't hesitate to move issues around if you think they're not properly tagged.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-04-03

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Tuesday's run (2023-03-28): :heavy_check_mark:
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* Next week's results (2023-04-04): **:arrow_right: Pavlo** to analyze.
* **Next injection in April.**
* Let's make sure runs will be OK.
* Still working on Q1 milestones because the managers are still discussing OKRs.
* Current Q2 milestones: https://hackmd.io/u11EvMGESgieGq2fMNiwcg?view.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-03-27

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* ~~Tuesday's run (2023-03-21)~~ **Failed.** Friday's run (2023-03-24): :heavy_check_mark:
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* Next week's results (2023-03-28): **:arrow_right: Nicolas** to analyze.
* Then Pavlo, Sota, Jun, Kazuhiro and back to Nicolas.
* **Next injection in April.**
* Let's make sure runs will be OK.
* Almost at the end of Q1, **the priority** is to **finish the active merge requests**.
* **:arrow_right: Nicolas** will propose a first version of Q2's project descriptions this week.
* **Here is the draft:** https://hackmd.io/u11EvMGESgieGq2fMNiwcg?view.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-03-20

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-03-17): :heavy_multiplication_x:
* **Several issues, runs didn't complete. :arrow_right: Nicolas**
* Add a configuration check script.
* Pavlo is adding checks that Tezt will correctly handle all benchmarks.
* **Once this is fixed, let's review results only once a week: Tuesday's runs.**
* We can take turns:
* open the latest `selected.csv` report from the [#gas-benchmarcks-reports](https://app.slack.com/client/T59LZHQ11/C04HZHR11DW?cdn_fallback=2) channel.
* `alerts` also lists added/removed/renamed benchmarks;
* report anything suspicious in the `Parameter under review` sheet of the [shared document](https://docs.google.com/spreadsheets/d/1dOpmB3FOAtdJF6gOa1NTtINA5j5iBouaPdgk4m0Hc1k/edit#gid=80403341) and on the Slack channel;
* Nicolas will do next week (03-28), and then in alphabetical order:
* Pavlo (04-04);
* Sota (04-11);
* Jun (04-18);
* Kazuhiro (04-25);
* Nicolas (05-02);
* etc.
* Next injection in April.
* Become Snoop experts: Jun and Nicolas during Q2.
* So that the knowledge is shared among several people.
* Almost at the end of Q1, **the priority** is to give a boost to the active merge requests and **put more efforts in reviews**.
* **Let's keep the effort going!**
* **:arrow_right: Nicolas will propose a first version of Q2's project descriptions this week.**

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-03-13

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-03-11): :heavy_check_mark:
* Next injection in April.
* Become Snoop experts: Jun and Nicolas during Q2.
* So that the knowledge is shared among several people.
* Less than a month left, **the priority** is to give a boost to the active merge requests and **put more efforts in reviews**.
* **Very good effort last week, good job everyone :thumbsup:**

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-03-06

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-03-03): :heavy_check_mark:
* Next injection in April.
* Become Snoop experts: Jun and Nicolas during Q2.
* So that the knowledge is shared among several people.
* Less than a month left, **the priority** is to give a boost to the active merge requests and **put more efforts in reviews**.
* **Work directions for Q2:**
* Complete unfinished work from Q1.
* Gas updates as a recurring activity.
* Includes fixing benchmarks when the values are not good enough.
* Implement the models for the memory allocation benchmarks.
* Brainstorm over macro benchmarks = start the [verify gas compositionality](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng?both#Verify-gas-compositionality) milestone.
* Snoop technical debt.
* Documenting (renaming?) non-intuitive parts of Snoop: local models, grouping, dependencies.
* Reinforce some concepts:
* local models not by a string;
* grouping benchmarks in Tezt not by tags.
* This is probably too much, we'll ask managers where they want to place the priorities.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-02-27

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Parameter update alert was detected for Sapling, due to the update of a Rust library.
The values are being updated: [7814](https://gitlab.com/tezos/tezos/-/merge_requests/7814).
* Latest benchmarks run (2023-02-24): :heavy_check_mark:
* Need to document the process of updating the values of the reference run.
See this [Slack message](https://tezos-dev.slack.com/archives/C04HZHR11DW/p1673691872794659). **:arrow_right: Nicolas**
* **Done in point 8 of [this section](https://gitlab.com/tezos/tezos/-/milestones/165#workflow-for-adjusting-gas-parameters) of the main milestone.**
* The idea is to use the merge request that updates the cost function to declare that the values in the reference run are updated too, and have someone from the team to approve by adding a comment like "*I approve the update of gas parameter values for the reference run*".
* **Is that OK for everyone or is there a better way to do this?**
* Next injection in April.
* Become Snoop experts: Jun and Nicolas during Q2.
* So that the knowledge is shared among several people.

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-02-20

**Current effort**

* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-02-17): :heavy_check_mark:
* Need to document the process of updating the values of the reference run.
See this [Slack message](https://tezos-dev.slack.com/archives/C04HZHR11DW/p1673691872794659). **:arrow_right: Nicolas**
* Next injection in April.
* Become Snoop experts: Jun and Nicolas during Q2.
* So that the knowledge is shared among several people.
* **We're understaffed. This means that some of us might have to review more than they're coding or the other way around. If this creates some frustration, let's talk about it and adjust :slightly_smiling_face:**

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-02-13

**Let's welcome Pavlo in the team :tada:**

**Current effort**

* Q1 milestones.
* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-02-07) :heavy_check_mark:
* Need to document the process of updating the values of the reference run.
See this [Slack message](https://tezos-dev.slack.com/archives/C04HZHR11DW/p1673691872794659). **:arrow_right: Nicolas**
* Next injection in April.
* Become Snoop experts: Jun and Nicolas during Q2.
* So that the knowledge is shared among several people.
* A recent incident emphasizes the need for other teams to be able to run the benchmarks and interpret the results.
* ~~Write a process. **:arrow_right: Nicolas**~~ **[Done](https://gitlab.com/tezos/tezos/-/milestones/165#manually-triggering-a-benchmark-run-emergency-process) for the technical part, but we'll need to identify who can take the responsibility of running the benchmarks in the gas team as well as in other teams.**
* **Could be interesting to be able to run a subset of the benchmarks.**

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

**Discussion about a new interface and benchmark registration process based on this [proposition](https://hackmd.io/a0_3Riu5RhKs2FhQUbfg2w).**

## 2023-02-06

**Current effort**

* Q1 milestones.
* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-02-03) :heavy_check_mark:
* Need to document the process of updating the values of the reference run.
See this [Slack message](https://tezos-dev.slack.com/archives/C04HZHR11DW/p1673691872794659). **:arrow_right: Nicolas**
* Next injection in April.
* Create Q1 milestones: see [above](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng?both#Active_milestones).
* *Who populates each one of them?*
* ~~Write the status and context. **:arrow_right: Nicolas**~~ **Done.**
* ~~Write the work break down.~~ **Done.**
* Migrate old milestones: see [above](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng?both#Old-milestones-2022-Q4).
* ~~[Consolidate Snoop's models and inference strategy](https://gitlab.com/tezos/tezos/-/milestones/145) left. **:arrow_right: Nicolas**~~ **Done.**
* **Become Snoop experts: Jun and Nicolas during Q2.**
* So that the knowledge is shared among several people.
* Pavlo from Marigold will reinforce the team.
* A recent incident emphasizes the need for other teams to be able to run the benchmarks and interpret the results.
* **Write a process. :arrow_right: Nicolas**

**Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status, and report what milestone we will each mostly work on during the week. :arrow_right: All**

## 2023-01-30

### Agenda

#### Current effort

* Q1 milestones.
* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-01-27) :heavy_check_mark:
* Need to document the process of updating the values of the reference run.
See this [Slack message](https://tezos-dev.slack.com/archives/C04HZHR11DW/p1673691872794659). **:arrow_right: Nicolas**
* Next injection in April.
* Create Q1 milestones: see [above](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng?both#Active_milestones).
* ~~Create the milestones. **:arrow_right: Nicolas**~~ **Done.**
* *Who populates each one of them?*
* The goal is to describe the context and make it easy to understand and follow for people not working on it.
* Write the status and context. **:arrow_right: Nicolas**
* Write the work break down:
* ~~Gas parameter updates: Nicolas.~~ **Done.**
* Snoop usages: Jun.
* Memory allocation benchmarks: Kazuhiro.
* ~~Verify gas compositionality: Nicolas.~~ **Done.**
* *Who's working on what? So that we don't step on each other's feet.*
* We all have rather separate activities.
* Let's simply get in touch on specific points whenever there is a need for more interaction.
* Migrate old milestones: see [above](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng?both#Old-milestones-2022-Q4).
* [Consolidate Snoop's models and inference strategy](https://gitlab.com/tezos/tezos/-/milestones/145) left. **:arrow_right: Nicolas**
* Kyoto University: wonders what costs the most when making a transaction for example.
* Could be interested when studying the gas compositionality.
* Report what milestone we will each mostly work on during the week. **:arrow_right: All**

#### Update [active milestones](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-milestones) status

## 2023-01-23

### Agenda

#### Current effort

* Defining work directions for Q1 and later (OKRs).
* ~~A first draft of propositions have been discussed between Zaynah and Fedor.~~
* ~~The draft needs to be reviewed by Lucas, and better include DaiLambda's work (described in [this document](https://hackmd.io/RP6CCk64Q0ivpAI73yEXJg)). **:arrow_right: Nicolas**~~
* In facts, OKRs need to be simpler, shorter, and broader than exepcted. They're done.
* Q1 milestones will be discussed today.
* Continue unfinished work from Q4 in the meantime.
* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-01-20) :heavy_check_mark:
* Need to document the process of updating the values of the reference run.
See this [Slack message](https://tezos-dev.slack.com/archives/C04HZHR11DW/p1673691872794659). **:arrow_right: Nicolas**
* Next injection in April.
* Create Q1 milestones: see [above](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng?both#Milestones-for-2023-Q1).
* Create the milestones. **:arrow_right: Nicolas**
* *Who populates each one of them?*
* The goal is to describe the context and make it easy to understand and follow for people not working on it.
* Gas parameter updates: Nicolas.
* Snoop usages: Jun.
* Memory allocation benchmarks: Kazuhiro.
* Verify gas compositionality: Nicolas.
* *Who's working on what? So that we don't step on each other's feet.*
* We all have rather separate activities.
* Let's simply get in touch on specific points whenever there is a need for more interaction.
* Report what milestone we will each mostly work on during the week. **:arrow_right: All**

#### Update [active projects](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-projects) status

##### DaiLambda updates

###### Code gen (Jun)

Checking issue https://gitlab.com/tezos/tezos/-/issues/4018 . We should make the file 100% auto-generated. Anything hand-edited must be in a separate file.

###### Allocation benchmark (Kazuhiro)
* Measured the allocated words for all instructions (https://gitlab.com/tezos/tezos/-/issues/4587#note_1248554101)
* Analyzing the results (https://gitlab.com/tezos/tezos/-/issues/4587#note_1248559713)
* The results for ICdr, ICons_list and IBlake seem reasonable
* Those instructions' allocations should be constant
* Wroking on IConcat_string
* The allocation should depend on the sizes of arguments.

##### Benchmark quality measurement (Sota)
* I calculated & investigated so that the "95% range of the normalized benchmark result" would be a good indicator for the quality of the benchmark.
* Summary is here https://gitlab.com/tezos/tezos/-/issues/4130#note_1248609706.
* The parameter is design to indicate that "the true regression results (intercepts & coefficients) will be within the range with the probability of 95%". (It depends on many assumptions & approximations so it is a very rough estimation though.)
* Like R2 scores, I will make an MR that introduces the "benchmark score" in the solution file and CSV file.

## 2023-01-17

### Announcements

* New HackMD document ([here](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng))
* The [old one](https://hackmd.io/kaD-o_RPS7OSxfOA_Ch86Q) is linked above.
* Meeting day changes starting next week: Mondays, 10:00 CET.

*We're setting up new projects and management processes. The meeting and the minutes may seem a bit messy, but it's supposed to be temporary!*

### Agenda

#### Current effort

* Defining work directions for Q1 and later (OKRs).
* A first draft of propositions have been discussed between Zaynah and Fedor.
* The draft needs to be reviewed by Lucas, and better include DaiLambda's work (described in [this document](https://hackmd.io/RP6CCk64Q0ivpAI73yEXJg)). **:arrow_right: Nicolas**
* Continue unfinished work from Q4 in the meantime.
* Everyone has something to do?
* Recurring activities: benchmarks monitoring, gas parameter updates, improving processes.
* Latest benchmarks run (2023-01-13) :heavy_check_mark:
* Need to document the process of updating the values of the reference run.
See this [Slack message](https://tezos-dev.slack.com/archives/C04HZHR11DW/p1673691872794659). **:arrow_right: Nicolas**
* Next injection in April.
* Create Q1 milestones. **:arrow_right: Nicolas**
* Report what milestone we will each mostly work on during the week. **:arrow_right: All**

#### Update [active projects](https://hackmd.io/T8GgL4G9QLSS6LmSnPvDng#Active-projects) status

##### DaiLambda updates

###### Allocation benchmark (Kazuhiro)

Experiments to measure the memory allocation of Michelon instructions. [#4587](https://gitlab.com/tezos/tezos/-/issues/4587)

* Measuring the allocated words by comparing the reachable_words from the interpreter's stack before/after the execution step.
* Still experimental, modifying the existing timing benchmark code with some dirty hacks
* Now investigating whether the results obtained are useful for code generation.

###### Benchmark timing distribution analysis (Sota)

Investigate benchmark stats towards finding suitable variance score for the benchmark results [#4130](https://gitlab.com/tezos/tezos/-/issues/4130)

The variance of benchmark seems to be proportional to the input size, so we may do linear regression on them and use the coefficient as the variance score.

- It might not be linear for some cases: e.g. blake2b)
- The proportional trend of variance might be vague in the current Nsamples (~1000).