2022 Archived Notes

# 2022 Archived Notes 2022-10-27 to 2022-12-22 # 2022-12-22 ## Last week action items ## Retrospective - It's been a calm week as half of the week is in holiday. No challenges with CI or reviews. ## Sprint Demo ## Refinement ## Planning ## Action Items - Andy to check the remaining tasks for succesfull erc-20 milestone - Team need to meet again to check # 2022-12-15 ## Last week action items ## Retrospective - Feeling we're doing many meetings privately. Can we use more the channels? Proposal: if you have no problem, try to use the public Discord channels. - How the specs and circuits are coordinated? i.e. just finihsed something in specs, need to streamline the process on how spec and circuits are changed. Can have a risk of un-sync changes on the spec or the circuit. Most of the times it should change. - Idea 1: if submitting a PR for circuit repo, file an issue for the spec repo. The other way around too. - Idea 2: list of test vectors to run on the circuits and specs. That way we'd make sure both are the same. Currently, very difficult to implement this but could be a goal in the future. - Adria will talk with friend to learn how a team working on the consensus layer is working on this. (handling specification and implementation in sync) - Congrats CC! Fantastic work creating the roadmap document :D. It could be nice to talk more about it. - https://hackmd.io/9VPGcq1CTJ6_ynGQIaGxhQ?both - Interesting, we could spend 2-3 hours reviwing/talking about it. - We could do a call talking about that. - If you'd like to include any topic, feel free to send it to the chat! We should do more writings to share information ## Sprint Demo ## Refinement ## Planning ## Action Items - Adria will share any interesting findings about circuits-spec sync - Schedule early 2023: schedule call to organize next milestone - Next week? - Review list of technical debt - We could discuss circuits-spec sync - Coudl serve as an input https://hackmd.io/074MdV2zRYKSo7N8KGqZcw - CC will send a poll in case we can start with this next week. - Han will check with Pinkie if they can move root circuit part to zkevm-circuits repo - Schedule early 2023: schedule 2-3h call "PSE Zkevm: Where are we and where are we heading?". Pre-work: everybody reads the document first. ## Notes - Brecth is refactoring and optimizing MPT with constraint builder. This is good as it reduces the code. Miha agrees with those suggestions. - Time Off - Leo, Edu, Adria will be off the next two weeks. - Andy, Miha (+1) will be off the last week of the year - David the last week and the first week of 2023 - Agreement - Skip the 29th scrum session - Continue with whomever is available although mindful many people will be out # 2022-12-08 ## Retrospective - CI / PR / Merges conversation happened - Removed integration tests req to be merged. This significantly easy off the queue. - Now, running integration tests in cascade mode. Has some drawbacks, but a positive aspect is we don't need to rebase constantly. (3-6 PRs / day now) - Requires the role of "Sheriff". Makes sure merge PRs don't break any test. This person will be in charge to do change on main branch and request another PR to fix the issues. Haven't implemented this yet. Pinkie mentioned it could be automated, however we're not sure if this is true yet. - Automation flow: reverts back if the PR breaks something, creates a new issue and taggs the original owner of the PR to fix the breaking changes. - Action Item: discuss with the rest of the team if it's worth for the CI / zkEVM team to invest in this automation. - Halo2/Exploratory work - Meeting with Barry last week. Invest more time on Halo2 improvements and exploratory work. Some work that we could start taking, i.e. debugging and performance improvements for mock prover. - This will take time from people that are part of both Halo and Circuits team. - How do we balance it? - New member joined 2 weeks ago. Kimi from Taiwan (yei!) He will be joining the circuits team. - In february a new person joining if everything goes right. - Will need some time to learn and adjust but in the future some people from circuits team can focus more on the Halo2 side. - We're coming to a point in circuits team that there are less thigns that can be done in parallel. That is a good opportuntiy to do a transition for people to dedicate more hours to Halo work. - Would it be worth have a scrum session for Halo2? - Some reasons might be: need to coordinate between the team and with other teams, there's 3+ people, there are dependencies, need to track against a deadline. - External Contributor (not from Scroll / Taiko) - Did a PR to replace print with logs. Quality was not very good. Also got the mentioned from Scroll - Reminder: we're not obligated to accept any PR from everyone. It is fine to reject a PR or ask them to do something else. - It seems this person is learning how to program/ interact with GitHub. It's no need to become a teacher ## Action Items - Wait for Brecht to do an initial pass of MPT spec and PR review. Afterwards add another reviewer from our team https://github.com/privacy-scaling-explorations/zkevm-specs/issues/289 - Kimi, Edu and David will have a meeting on how we handle public inputs. This could influence https://github.com/privacy-scaling-explorations/zkevm-circuits/issues/856 - Challenge API Strategy - Continue on that direction - Or close the milestone without the challenge API migration and put effort on HIGH/LOW approach. - -- - Action Item: discuss this meeting next week (vote for your availability in the circuit channel) - Han will double check if EQ OPCODE was already implemented or not https://github.com/privacy-scaling-explorations/zkevm-specs/pull/336 # 2022-12-01 ## Retrospective - CI: This week we had many problems with the PR (i.e. of 12 PRs we could merge 4). This slows down considerably. We need to be more careful because of the rebasing. - Issue: integration tests were run for every single PR we needed to merge. As integration test are run for every single machine, it can't be paralellized. If you're the 13th, - The waiting PRs need to be rebased with the ones ahead in the line. - 3rd problem: there was a problem on how a job waited for the next one. One job would end and stop the machine and then the next job would have to be stuck waiting for the other. (out of memory problems because of the size of the tests?) - 4rd problem: the machines ended the job suddenly, they disconnected randomly, etc (while doing benchmarking) - **Short Term Workaround: for this time, integration tests are not required for merging a PR, just unit test** - Mid Term Goal: improve this CI. Test-next team and Marios was going to focus and help with a workaround. Trying to have more than 1 server and for them to be more robust. - **Short Term solution for avoiding rebasing?: need to double check if we keep this check on the CI or if there is another way.** - PSE PR Reviewers - Current backlog (29-30 open PRs) - Is it well distributed as a team? - David is happy to take and collaborate on some of those. - PSE Review Coordinator: CC (Edu in the interim) - External Review Coordinators: Haichen (Scroll) and Brecht (Taiko) - Review coordinator: the goal is to make sure PRs have assigners, check availability from people on their team. ## Action Items - As a team, discuss if we want to keep the rebase a pending PR after each new PR is merged. - One idea is splitting this repo more. (mono vs many repos) - Andy (Edu) to get another PR reviewer from Scroll or Taiko https://github.com/privacy-scaling-explorations/zkevm-circuits/pull/873 - For PRs that don't have issues, add it to the board. - For PRs that have link, link them and add them to the board. - Edu to check if the person in the comments can take https://github.com/privacy-scaling-explorations/zkevm-specs/issues/259 # 2022-11-24 ## Retrospective - The code review process with external teams is working really well. - Experience is 50%. One random reviewer did it pretty quickly, one is taking longer but well overall. - Question: what if the reviewer is taking to much time? - Ping them, tag them - Ask if you want to find someone else - If the current approach doesn't work, we can try assigning it to Scroll Tech account or Taiko account. - Ping Scroll Taiko and ask if it works for them? Or if they want to have a reviewer coordinator who owns the assigning review?- . ## Action Items - Wait for pinkie's feedback on how to proceed with #790 () - CC will ask Scroll, Taiko which way they prefer assinging the code reviewers - Either we assigning it directly - We assign it to their company accounts - We assing it to a reviewer coordinator - Team: need to have a deeper discussion about following: - Remove word RLC and change word to HIGH and LOW? Outcome> need to be yes or no and with yes when. - The Future of challenge API: less urgent discussion. How it affects if Scroll is not using it and how it affects splitting in different proofs. - Andy and Edu to review and clean the milestone task list https://github.com/privacy-scaling-explorations/zkevm-circuits/milestone/1 - Han to share concenrs about on identity precompile # 2022-11-17 ## Retrospective - Strategy to assign random Scroll reviewers is working fine. Useful to have a reviewer you don't usually work with, having external opinions and perspectives. - There a still a few open PRs, asking for help for anyone not doing reviews. Take a look at PRs that have not been looked at anyone from PSE. - Added Halo2curve repo to Halo2 subteam. - Conversation about milestones and priorities of PRs review. - How to balance reviewing PRs from our priorities (successful erc-20) with Scroll & Taiko's priorities? i.e. out of gass, delegate calls, etc. - We should give priorities to reviewing PR's for erc-20 milestone but should also give time to review PRs for other things. For example, reviewing a PR that's old is important so they don't lose the collaboration spirit. - The Sprint Board doesn't show all the work that we're doing. Are there better ways? - Start doing again Daily Updates: - What did we do? - Do I have any blockers? - What am I going to do next? ## Action Items - Edu or Adria to check the changes and confirm the additions are sufficient. Afterwards we can close this issue for #268 - Andy reviewing age of PRs and making sure they're not after 15 & 21 days. - Andy to share the dependencies of circuits with the testnet team which are #790 (pinkie) and with halo2 #105 --- # 2022-11-18 "Scrum of Scrums" ## Done - We made a list of external contributors and have been asking for second reviews by picking people from the list. This is working. - We find it a bit hard to balance erc-20 milestone tasks with the PRs from external contributors, which have less priority for us, but shouldn't be left out. - We'll be back to doing async dailies to show work that we're doing that is not in issues. - notable work in zkevm-circuits (from any contributor): - real prover integration tests (in review) - RETURNDATACOPY - small fixes here and there - ETH transfer integration tests passing (from BeginTx impl update) - notable work in zkevm-specs: - OutOfGas from call - insufficient balance state (in review) ## Next - challenge API refactors for EVM, Keccak, PublicInputs circuits - MPT review spec - write keccak-packed-multi spec - unify circuits names and interfaces - refactor Bytecode Circuit as a state machine ## Discussion - Offer help to pinkie debugging prover failures # 2022-11-10 ## Retrospective - Last week discussed a lot queue of PRs. Managed to get in a good track. - This means there was a bit less progress completing other tasks. - Discord: shall we create a new channel to have "circuits" and "circuits scrum"? - Discord: we can figure out how to manage roles and permissions in the future. - Last week we did Scrum of Scrums. - Andy schedule: good after 12:00 CET - We spend a lot of time reviewing other teams. We can ask other teams also to review things. -> - Selected group of external contributors from Scroll and Taiko we can automatically assign it to them. - Scroll and Taiko agreed that PR send from their team will support 1 reviewer from their team. - ~~**Action item: ask them if they agree, find the group of people**~~ - Goal: for each PR review at least 1 internal and 1 external - We should be more active and consistent requesting Scroll Tech or Taiko. - Idea: Create tags in the PRs? After review is done remove the tag. - ~~**Action item: Create a list of Scroll Tech and Taiko on Github. When we have a PR, we pick one of them.~~** ## Action Items - Edu or Adria to check the changes and confirm the additions are sufficient. Afterwards we can close this issue for #268 - Public Inputs Circuit: have a sound randomness #856 is dependanat on Remove blinding factors #105 from Halo2 subteam. Halo2 subteam aware. ~~- Edu/CC can share the contact to Miha. Miha to contact the Geth developer for #268~~ - (done) Andy will ask Pinkie if he's workin on Define and implement the Root Circuit #664 if so, move it to in progress, else leave it there. - (done) CC - will ask someone from Scroll if they can review this PR https://github.com/privacy-scaling-explorations/zkevm-specs/pull/307 - (done) Han to ask someone from Scroll to review PR https://github.com/privacy-scaling-explorations/zkevm-circuits/pull/872 - (done) Miha also helping with PR - Add challenge API to state circuit #885 - (888) Andy will ask Marios if this task is done Create Infra for integration tests with real prover #888 otherwise we can add it as a dependency for Add real prover tests for the integration tests #789 in Scrum of Scrums. ## Notes: - Next week Scroll, Polygon and people from team will join a session Monday to Thursday 7pm-9pm CET. - Talks of security analysis on zkEVM. - If anyone interested let Edu know. - interested people: Andy, Adria --- # 2022-11-11 "Scrum of Scrums" # Done - Review many PRs from the queue - less progress on completing assigned tasks - Some months ago we discussed that every PR would be reviewed by 1 internal + 1 external; but we often don't request the external so we have a lot of load. We'll try to be more active on requesting external reviews. - MPT scope document is done and reviewed by Miha, and now integrated into the MPT spec PR - MPT spec is ready for review - Super Circuit spec in review - ongoing challenge API refactors - Reviewing the last missing pieces to get milestone ERC20 complete - notable work in zkevm-circuits (from any contributor): - Fix EndBlock for EVM circuit padding, fix and enable integration tests on PRs. - Update BeginTx to support tx to account without code (simple ETH transfers). In review - bus-mapping: Fixes and add missing data - PublicInput circuits bench - Keccak circuit can be of fixed capacity independent of input lengths - notable work in zkevm-specs: - DELEGATECALL opcode - PublicInput circuit optimization (lower verifier cost) - invalid error jump state # Next - challenge API refactors for EVM, PublicInputs circuits - real prover integration tests - MPT review spec - write keccak-packed-multi spec - Continue debugging the prover issues # Discussion - Andy is in Costa Rica timezone, which limits his availability: he can only be available after 12:00 CET. We could have Friday meetings at 12:00 so that Andy can come and be the facilitator. - We have a Discord server now. Currently Edu and Tyler (AtHeartEngineer) can create new channels; but the plan is to let anyone do that (needs role/permissions assignments), so we can add new channels as needed. --- # 2022-11-03 ## Retrospective - Lots of open PRs - Think abour priorities - First: Fixes, small updates, and ERC20 milestone tasks - Second: new opcodes - https://github.com/privacy-scaling-explorations/zkevm-circuits/issues/782 - Everything is ready from the Circuits team ## Sprint Demo ## Refinement ## Planning ## Action Items - Edu: Figure out why zkevm-chain tasks appear in the SubTeam: zkEVM-circuits tab in the project board - DONE: added filters - Adria: Explore the idea of having a configurable SuperCircuit where we can disable some circuits to run tests faster - Discuss with testnet team how is better to have access to a server to run tests with the real prover (instead of having direct ssh access) - Ask Miha about the status of https://github.com/privacy-scaling-explorations/zkevm-specs/issues/268 and wait for his approval (once he comes back from vacation) --- # 2022-11-04 "Scrum of Scrums" # Done - Mostly spent time checking open PRs and reviewing - notable work in zkevm-circuits (from any contributor): - simple but effective memory/cpu optimization https://github.com/privacy-scaling-explorations/zkevm-circuits/pull/852 - added EXP and RETURNDATASIZE opcode circuits - out of gas exception circuits for constant gas opcodes - Add Super Circuit to integration tests - notable work in zkevm-specs: - added BALANCE opcode - Review PublicInputs circuit design https://github.com/privacy-scaling-explorations/zkevm-circuits/issues/856 - Will require removing blinding factors in halo2 https://github.com/privacy-scaling-explorations/halo2/issues/105 # Next - challenge API refactors for State, Keccak, EVM and PublicInputs circuits - write keccak-bit spec - write super circuit spec - review PRs (we have a long queue of open PRs) - continue working on fixing Prover failure issues - real prover integration tests - continue working on making fixed size circuits work (some open bugs reported by lispc) # Discussion - Request for halo2 team: - To have a sound working PublicInputs circuit we need the removal of blinding factors from halo2 - Discussion with testnet team: - Soon we'll have easy to reproduce real prover tests, that may need lot of resources. What's the best way to run tests on a powerful server? - Status update on https://github.com/privacy-scaling-explorations/zkevm-circuits/issues/782 --- # 2022-10-27 ### Action Items - ~~Andy will check with Leo about status tasks "zkevm-chain #57"~~ - Miha and Edu will reach out a Geth developer if it's possible to have a 17th value and (something else I didn't took notes) - Adria will add the additional case mentioned by Miha in the specification scoping document. https://hackmd.io/nI_STO5HQ4G-ugKnQEzctg?view - ~~Andy will check with Nick about the status of #47 and #782~~ - ~~Andy will check with Marios if he's still blocked at "zkevm-chain #52" due to Keccak error.~~ And ask Pinkiebell if this is blocking his task "zkevm-chain #18" - ~~cc check with brecht on if he has people working on keccak spec~~ - Edu and David will discuss "Public Inputs: tricky parts needed with randomness. and write summary at "zkevm-circuits #856" - Miha will re-arrange the spec document to match the structure of the spec scoping document. Collaborate with Adria. - Miha to add a PR to https://github.com/privacy-scaling-explorations/zkevm-circuits/issues/740 ERC-20 Compatibility - Need to do a proper review of everything needed for ERC-20 milestone. Bugs, issues that might come up. - Adaptation all sub-circuits for the challenge API. ### Retrospective - Goal: reflecting how we did on the sprint. Things we can improve, things that are going well. ### Sprint Demo - Goal: Showing what we completed on the week. ### Refinement - Goal: How to break tasks that are ambiguous or big into smaller more detailed tasks. Also estimating. - Main question: what are we missing to accomplish the milestone? Can ### Planning - Goal: Deciding how much work we can take for the next sprint. ###### tags: `zkevm`