# DaiLambda Tezos gas works

* Cost function generation
* Code generation command improvemnt
* Gas parameter inference score
* Benchmark quality measurement
* Memory allocation benchmark

## Cost function generation

Goal: auto-generate OCaml code of cost functions from the gas benchmark inference results.

Done:

* Output code is now compilable.
* Separate the generated code in a module `Michelson_v1_gas_costs_generated`.
* Hand written cost functions are in `Michelson_v1_gas_costs`.

Left open for 2023 Q1:

* [3834](https://gitlab.com/tezos/tezos/-/issues/3834). Some cost functions are hand written since the corresponding models have incompatible types with them. We should fix them and make them auto-generatable.
* [4018](https://gitlab.com/tezos/tezos/-/issues/4018). Some cost functions, even defined in `Michelson_v1_gas_costs_generated`, need manual tweaks for their memory allocation costs.
* They can be moved to `Michelson_v1_gas_costs`.
* Ultimately, their code should be auto-generated with the memory allocation benchmark.
* [3678](https://gitlab.com/tezos/tezos/-/issues/3678). Auto-gen code has redundant `let open S_syntax in` expressions. Should they be removed?
* Currently, their compilation warnings are disabled by annotations and I think it is ok.

## Code generation command improvement

Goal: improve `tezos-snoop` command to ease the gas handling development.

Done:

* Added a new sub-command to auto-generate all the cost functions derivable from the given solution file.

Left open for 2023 Q1:

* None

## Gas parameter inference score

Goal: measure the qualities of gas parameter inferences and include their scores in the inference result file

Done:

* We now have $R^2$ and RMSE scores for each regression and they are in the inference files.
* The scores are integrated by Nomadic gas team members into the benchmark spreadsheet generated twice a week.

Left open for 2023 Q1:

* [7190](https://gitlab.com/tezos/tezos/-/merge_requests/7190): The scores in the inference report PDFs. Already implemented. Review required.

## Benchmark quality measurement

Goal: measure the quality of benchmarks, in particular the standard deviation of individual timings.

Done:

* Some preliminary research of the timing variances: [4130](https://gitlab.com/tezos/tezos/-/issues/4130). We found the number of GCs in each test run affects the timings a lot.

Left open for 2023 Q1:

* Increase the number of test runs for each input to see how the time distribution changes.
* Exclude test runs with GCs to see how the distributon changes.
* Define a score of benchmark somehow and put it into the benchmark results.
* Linear regression of variances over the input size?

## Memory allocation benchmark

Since Kathmandu, the cost of each Michelson operator becomes the max of the runtime cost and memory allocation cost (0.5 unit per byte) in order to prevent memory exhaustion DoS attacks. However, Snoop cannot generate the cost functions with the memory allocation cost for now, since its benchmarks are only for the runtime costs.

This is a new project we would likt to work on in 2023.

Tasks for Q1 (and possibly for Q2):

* [4587](https://gitlab.com/tezos/tezos/-/issues/4587): Measure the size of the newly allocated objects for each test run, and store them in new workload files.
* Kazuhiro is already working on it.
* [4588](https://gitlab.com/tezos/tezos/-/issues/4588): Implement memory allocation models for opcodes
* [4589](https://gitlab.com/tezos/tezos/-/issues/4589): Run Lasso regression to infer the parameters and evaluate the results
* [4590](https://gitlab.com/tezos/tezos/-/issues/4590): Modify the code gen to include the memory allocation cost