HackMD
PoS Implementers’ Call #94 - 2022-08-25
[Quick contemporaneous notes by Ben Edgington; fka "Eth2 Implementers' Call"]
Agenda: https://github.com/ethereum/pm/issues/598
Livestream: https://youtu.be/tjmpu8O-xsA
Merge
[Pari] Last shadow fork went well. Updated to latest Geth version on Monday. Mainnet shadow fork 12 will happen next week, using client versions recommended on the EF blog announcement.
[Marius] Still seeing bad blocks from Erigon on MSF11. Erigon team is aware and has reproduced the issue on Ropsten where they will continue to work on it.
EF Bounty Programme - between now and 8th of September, critical vulnerabilities get $1m (all bounties are temporarily 4x from normal).
[Lukasz] From Nethermind, there are ongoing issues if the consensus client doesn't allow enough time for FCU call. Lodestar has now resolved this. Not sure if Nimbus will fix their behaviour around this pre-Merge, so may implement a workaround - still deciding on this. [Zahary] It was supposed to be implemented in the last Nimbus release but had some issues; the fix will be included in the Nimbus hardening release pre-Merge. [Mikhail] Has a general preference for not trying to fix/workaround issues on the wrong layer. Hopefully this will be only a temporary workaround.
[Lukasz] Also some edge cases in syncing with consensus clients. Nethermind syncs a little differently. Can process new blocks at the head even if it has not downloaded the entire block history. Advise that consensus clients should include some testing around this scenario. Nethermind will now return eth_syncing=true when it doesn't have all the historical blocks, which is a change from previous behaviour.
[Mikhail] There was a discussion on Discord about changing the status of the payload via a restart, when the execution client previously responded invalid then changed its mind. What are the recovery options? Clients differ in their behaviour. Most are fixed by a restart, but Lighthouse probably not. [Danny] Forgetting invalid status on a restart is probably a reasonable strategy. [Potuz] Prysm does not store invalid blocks at all. The fork choice is not persisted either, so it is recalculated fine on restart. Also Nimbus, Lodestar.
[Saulius] Is it a problem for clients that they have a limited view after a restart? Grandine does the same - forgets everything and starts from a checkpoint after a restart. [Danny] Rebuilding the fork choice from gossip in real time should be fine.
MEV Boost
[Raul] Post about removing the trusted relay via threshold encryption. Tackles privacy at the core level.
[Jacek] Nimbus will be adding info to documentation to highlight that relays may have different transaction inclusion strategies.
Circuit breaker implementation
[AlexS] Lighthouse and Teku look to have merged PRs. [Terence] Was merged in Prysm v3 recently.
Research, spec, etc.
EIP-4844
Topic 1
https://github.com/ethereum/pm/issues/594#issuecomment-1206713478
[Micah] This is about how we adjust the gas price between blocks: smooth or bursty. What is easier from a client implementation perspective?
Action: open an issue for discussion offline.
Coupling of syncing of blobs and blocks? [Terence] wrote a design doc and feels that the difference between approaches is small. But would like to hear feedback about having them coupled. How does this interact with checkpoint sync, and potentially backfilling a month of blobs? [Danny] Intuition is to keep them independent. Terence and Enrico spoke in favour of this. As far as the spec goes, treat blobs in a modular fashion. We can reopen this if it becomes a problem.
Topic 2
https://github.com/ethereum/pm/issues/594#issuecomment-1211765927
Protolambda was not on the call to discuss.
[Mamy] Blst team contacted him last week to gather some requirements for KZG use cases. EF research team have also been talking to Supranational.
AOB
User experience feedback
[Raul] How are teams finding user-readiness for the Merge? Lots of confusion is showing up in the Prysm Discord.
[Tomasz] getting some negative feedback about the configuration experience. [Marek] A lot of users asking questions about the need to run a consensus client.
[Jacek] Lots of users having trouble with the JWT secret, and the need to switch to a different port on the execution engine. Also lack of clarity around suggested fee recipient.
[Danny] Check out the EthStaker Merge workshops. Number 4 was yesterday. Also Somer's updated installation guides.
[Jacek] Nimbus does not start calling the execution API until Bellatrix has happened. This leads to execution clients complaining in the meanwhile.
[Lightclient] How far away are we from automatic set up of the two clients? [Danny] Handling the JWT secret is always going to be an issue. [Lightclient] A default location for the JWT token might help.
[Lukasz] Nethermind devops developed sedge as a way to easily set up nodes. It uses Docker. It is not officially stable, but works well. Only supports Geth and Nethermind on execution side so far, and Lighthouse, Lodestar, Prysm and Teku on consensus side. Feel free to make PRs to this tool.
Light client protocol
[Jacek] One way of driving an execution client is to use the light client protocol. There is a beta of a Nimbus light client that just follows the protocol and can substitute for a full beacon node. Upsides are reduced network bandwidth, CPU, storage. Downside is that you are following 15 seconds behind head, and rely on light client security assumptions. Might get bundled with the Nimbus Eth1 client eventually. Also planning to publish it as a C library.
Also working on a light proxy. An eth_getProof call in Geth allows you to get a Merkle proof of anything in state. The light proxy can validate these proofs. Useful as a wallet verifier.
[Gajinder] Have a similar demo in Lodestar, running in-browser.
User activated soft fork to deal with bad actors
[Micah] Reprise of conversation from the last ACD call.
Lots of people don't understand that having this capability is a deliberate part of the protocol design.
Call for core devs to signal that we would be prepared to do such a UASF if circumstances merit it. That is, to ensure that people know that this is part of the protocol; it is an intended part of how proof of stake works.
[Mamy] We ought to have some kind of constitution setting out our values, against which we come to a decision. [Micah] Tend to agree. [Danny] We already have a written protocol, and that could be enough to define bad behaviour. It defines how the head of the chain should be identified.
Chat highlights
- From Marius to Everyone 03:06 PM
- bounty @danny
- From Marius to Everyone 03:07 PM
- 8 sept
is erigon etc under bounty? - From danny to Everyone 03:07 PM
- yes
known bugs do not count - From Micah Zoltu to Everyone 03:08 PM
- My vote is to not add workarounds because other clients have bugs.
- From Marius to Everyone 03:08 PM
- nice, I rebuild my fuzzer today, will start this evening
- From Phil Ngo to Everyone 03:09 PM
- Lodestar included the fix Nethermind on payload timings for v1.0.0
- From Łukasz Rozmej to Everyone 03:17 PM
- Micah I would love to do that, but user won't be interested if its Nethermind or Nimbus issue and will just switch
So it hurts our brand, increases (already estimated high) noise around the merge and reduces market share - From terence(prysmaticlabs) to Everyone 03:17 PM
- We added a flag: https://github.com/prysmaticlabs/prysm/pull/11303
- From danny to Everyone 03:18 PM
- nice
- From Raul Jordan (Prysm) to Everyone 03:19 PM
- https://ethresear.ch/t/removing-trusted-relays-in-mev-boost-using-threshold-encryption/13449
- From Micah Zoltu to Everyone 03:24 PM
- If we run out of things to talk about in this call Danny, we can have the same conversation from ACD again (but with CL devs this time). 😬
- From danny to Everyone 03:25 PM
- weren’t they there???
- From Micah Zoltu to Everyone 03:25 PM
- Not all of them. :)
In particular, I still think it is very important for CL devs (individually) to make a public credible commitment to a UASF/UAHF if we start seeing intentional reorging by validators. - From danny to Everyone 03:26 PM
- https://github.com/ethereum/pm/issues/594#issuecomment-1206713478
- From terence(prysmaticlabs) to Everyone 03:29 PM
- https://hackmd.io/_3lpo0FzRNa1l7XB0ELH7Q?view
- From danny to Everyone 03:34 PM
- https://github.com/ethereum/pm/issues/594#issuecomment-1211765927
- From terence(prysmaticlabs) to Everyone 03:35 PM
- https://hackmd.io/@inphi/SJMXL1P6c#Endpoints
- From Victor Zhou (xinbenlv) to Everyone 03:37 PM
- QQ: is there a way to find stats of client adoptions of the new verson?
e.g. the GETH v1.10.23/v1.10.22 - From danny to Everyone 03:39 PM
- ethstaker validator https://www.youtube.com/watch?v=Jra2cx0Wcss
- From Pooja Ranjan to Everyone 03:39 PM
- https://ethernodes.org/merge
- From Trent to Everyone 03:40 PM
- Merge Community Call #7 will be sept 9
https://github.com/ethereum/pm/issues/599 - From danny to Everyone 03:41 PM
- https://twitter.com/SomerEsat/status/1562541042120814592
- From Łukasz Rozmej to Everyone 03:43 PM
- https://github.com/NethermindEth/sedge
- From Parithosh Jayanthi to Everyone 03:45 PM
- How do we handle different OS-es?
I doubt there is any path that is standard across operating systems - From Mario Vega to Everyone 03:46 PM
- maybe not even requiring jwt key if request is coming from localhost ?
- From Marek Moraczyński to Everyone 03:46 PM
- https://twitter.com/nethermindeth/status/1555571498894725123
- From Trent to Everyone 03:46 PM
- https://github.com/NethermindEth/sedge
- From danny to Everyone 03:47 PM
- the fear on not requiring jwt is if ports are accidentally exposed
light CL -> full EL - From Łukasz Rozmej to Everyone 03:51 PM
- If any of CL/EL clients that is missing want to add sedge integration please contact our devops or do a PR :)
- From danny to Everyone 03:58 PM
- failure modes that are attributable but not auto-punishable
the protocol for finding the head is spec’d - From Potuz to Everyone 04:04 PM
- it gets tricky with Michael's PR to fork late blocks
We'll need to add that to the protocol I think
