PoS Implementers’ Call #86 - 2022-05-05

[Quick contemporaneous notes by Ben Edgington; fka "Eth2 Implementers' Call"]

Agenda: https://github.com/ethereum/pm/issues/521
Livestream: https://www.youtube.com/watch?v=nnjeqZK7jgU

The Merge

Testing

There was a Merge testing call on Monday. No need to repeat here.

Mainnet shadow fork #3 was today. TTD was hit around 1pm (UTC?). Participation pre-TTD was 99.8%. Only pre-TTD issue was with Besu–Geth sometimes missing blocks. Post TTD 97.6 % participation since the Besu–Geth issue got worse. Now being triaged. Later some intermittent issue with Prysm–Nethermind. Being fixed.

Almost bugless this time! Not seeing any late blocks. All clients are in sync.

An additional syncing through the Merge test was carried out on this shadow fork. Some issues, still analysing.

Governance update

As decided at ACD call last Friday, intention is not to proactively defuse the difficulty bomb yet. Further monitoring and decision making to be done during May on forking testnets and resetting the bomb if required.

Ropsten Beacon Chain launch

Ropsten, Sepolia, and Goerli are planned to transition to the Merge.

To do Ropsten, we need to make a Ropsten beacon chain (and same for Sepolia), at least a couple of weeks in advance. Is the plan to maintain Ropsten post-Merge? Client devs would prefer not to have to maintain Ropsten for much longer.

Any views on: Low/high validator counts? Permissioned/unpermissioned? Permissioned makes it easier to test chaotic states, and stuff we'd rather not do on "production" testnets. Large operators/individuals could still participate by request.

• The Goerli beacon chain will be unpermissioned.
• For Ropsten, makes sense to be unpermissioned, but for devs to have the majority of validators.
• Sepolia might be best permissioned to allow for a wide range of experiments. Goerli is too heavily used to do this without disrupting users. Smallish validator set. Would a huge validator set (e.g. 2x Mainnet) be interesting to test?

Expectation that client teams would help run validators.

Danny to open issues in the repo to describe the plans for Ropsten and Sepolia beacon chains.

Should try to launch the beacon chains in the next 2-3 weeks to allow for forks in June. Ropsten first, then Sepolia a little later (if it's bigger) makes sense.

Discussion

Mikhail has opened a PR around invalid terminal block that he'd like to merge asap. Also fixes a hole in the spec.

Several other PRs are open that should be merged soon: 216, and some error code things. Will be bundled into a release.

Other Client updates

Lodestar
Released v35 with lots of improvements (esp. networking). Enabled proposer bost on Mainnet.

Lighthouse
Issue around running fork choice before making block proposals. Some pressure from the community to implement IPv6 - interested in feedback from other teams. Will it cause issues (partition the network)?

Nimbus
Focused on shadow forks. Nimbus can now work with Web3Signer - is able to connect to multiple remote signers and combine the signatures. Useful to protect against rogue employees - no employee has access to a full validator key. Could be possible to extend the key manager API to support this configuration.

Prysm
v2.1.1 released with Merge support and Web3Signer support and some other nice bits.

Teku
Some optimisations around the epoch transition. Implementing the Builder API (MEV Boost). Minor bits and pieces on the Merge.

Grandine
Optimisations. Arm64 support. Web3Signer support - initial functionality is in place.

Adoption of episub

Makes the mesh more efficient (reduces bandwidth consumption) without changing too much in gossipsub. Modifications to gossipsub are very minor. Not yet in use in Filecoin or elsewhere yet, though (gossipsub is doing OK there).

Sigma Prime is looking for a Go dev to make an implementation of this.

The episub spec is not very useful - we really only need to add a single extra control message to gossipsub.

Builder API

[Lighclient] Discussed the Builder API on ACD last week. How important is it for validators to control the gas limit of blocks? Overwhelming desire was yes, it's important. Thus spec updated to add this parameter in the signed data.

Builder API PR has received a lot of feedback. Teams implementing it felt there was a lot of work going into reworking the serialisation - prefer to pivot to HTTP REST API. Lightclient is rewriting the API in REST format (no significant logic changes). Would like feedback when the draft is done.

All this currently lives in the execution API repo. Should we move it to its own repo? [Danny] Yes, this is not core, so makes sense to be separate.

There is a pending PR that's been sitting around for a week and a half: how to get that moving? (206). Danny to review.

Can client teams get implementations done in the next couple of weeks?

There is no way to compare blocks received externally vs my local execution client's block. We should have some heuristic to check whether we locally have better blocks than the relays give us. Can we use the eth_api to get some info? The blocks would need to be executed to compare them. This will introduce delays. [Mikhail] Suggest a get_payload_v2 API as discussed at Devconnect, can be done post-Merge if we trust the relays a little meanwhile. [Terence] what sort of time-out should we give relays? Something like 1s.

Plan a release of the REST API spec version in the next week.

Spec discussion and AOB

[Micah] We should continue the IPv6 discussion, perhaps on Discord. Seems like a legitimate problem if many nodes can speak only IPv6. Although clients must support IPv4, users' networks may not support this.

[Age] Changes to support IPv6, especially in Discovery, are quite extensive, so we do not plan to do anything on this pre-Merge.

Plan for next shadow fork next week. Pari to announce it later.

---

Chat highlights

From danny to Everyone 03:04 PM

https://github.com/ethereum/pm/issues/521

From pari to Everyone 03:06 PM

https://notes.ethereum.org/@parithosh/HyiESUWIc

From Potuz to Everyone 03:18 PM

That may be useful, we don't seem to be catching up on Prater

From Enrico Del Fante to Everyone 03:18 PM

Makes sense to me

From Micah Zoltu to Everyone 03:18 PM

I'm fine with either.
Weakly in favor of 2x to 10x mainnet, but maybe just because of recency bias. 😬

From pari to Everyone 03:21 PM

Weakly in favour of like a 16k validator set size for ropsten, so relatively easy to setup when split amongst ~10 teams

From Mikhail Kalinin to Everyone 03:22 PM

https://github.com/ethereum/execution-apis/pull/217

From danny to Everyone 03:24 PM

https://github.com/ethereum/execution-apis/pull/216

From pari to Everyone 03:24 PM

Where are we bike shedding the testnet names?

From Tim Beiko to Everyone 03:25 PM

Call the testnet cappella and move to single names for future hard forks 🙂

From Trenton Van Epps to Everyone 03:25 PM

yessss this

From danny to Everyone 03:25 PM

ropsten-beacon-chain

From Tim Beiko to Everyone 03:25 PM

The RBC sounds neat
SBC

From danny to Everyone 03:25 PM

SBC
lol

From pari to Everyone 03:27 PM

Oh just for the record, in case I didn’t mention it properly earlier. Every client stayed in sync + are in sync in msf3. Even if we have issues with proposals, they are all in sync 🙂

From Tim Beiko to Everyone 03:27 PM

Nice, Pari!

From Age Manning to Everyone 03:27 PM

https://github.com/ethereum/consensus-specs/pull/2878

From Micah Zoltu to Everyone 03:28 PM

So we aren't bad.

From Jamie Lokier to Everyone 03:28 PM

I saw a few IPv6 addresses advertised in the discv4/devp2p EL network. Didn’t actually connect to them to see if there’s a giant second clique of nodes hiding over there.

From Micah Zoltu to Everyone 03:29 PM

At the least, we should strive to have as many nodes supporting both IPv6 and IPv4 so we can make sure we have a strong bridge between the two.

From Age Manning to Everyone 03:29 PM

Yeah. I'm not sure entirely whats going to happen when we allow it and advertise ipv6 in discovery. How connected ipv6-only nodes will work, or dual stack ones

From Micah Zoltu to Everyone 03:29 PM

There will certainly be some clients that are IPv4 only because their ISP doesn't support IPv6. If there are also IPv6 only nodes, then we should try hard to protect against network partitions.

From Age Manning to Everyone 03:30 PM

agreed
Fork choice before block proposal if anyone hasn't seen it: https://github.com/ethereum/consensus-specs/pull/2878

From pari to Everyone 03:31 PM

Are ipv6 only nodes a thing? I thought most ISPs would still offer a 6to4 tunnel so things don’t break

From Micah Zoltu to Everyone 03:31 PM

For peer discovery, I think that won't help, right?

From Age Manning to Everyone 03:31 PM

Apparently, I can link you to some issues on our repo where some community members are asking for it if you're interested pari

From Micah Zoltu to Everyone 03:32 PM

Your peers would tell their peers "hey, I know a guy advertising on <ipv6>" and if you can't speak IPv6 you can't connect to them.

From Jamie Lokier to Everyone 03:32 PM

For peer discovery I think the IPv6->v4 tunnel would be equivalent to having a NAT you can’t punch a hole in, and there are plenty of those already.

From pari to Everyone 03:32 PM

That’d be cool, thank you :D Didn’t know the future arrived already

From Micah Zoltu to Everyone 03:34 PM

If someone is running IPv6 only and their ISP does tunneling, would they be able to open a connection to an IPv4 address (this may be a client-specific question)?
IIUC, they would only have IPv6 network interfaces locally.

From Jamie Lokier to Everyone 03:37 PM

Micah: Generally no they wouldn’t, if they only have IPv6 locally and no upstream configured tunnel address to map IPv4 requests to.

From danny to Everyone 03:40 PM

https://github.com/ethereum/builder-specs

From lightclient to Everyone 03:41 PM

https://github.com/ethereum/beacon-APIs/pull/206

From Jamie Lokier to Everyone 03:42 PM

Are ipv6 only nodes a thing?

From Micah Zoltu to Everyone 03:42 PM

Sounds like it according to Age I think (sorry if I forgot who said it).

From Jamie Lokier to Everyone 03:43 PM

I’ve read that there are users out there who can’t connect to IPv4 only websites, but I’d be surprised if there are many of them.

From Age Manning to Everyone 03:44 PM

Yeah. I think the push is for dual stack. But someone is trying with ipv6 only. See eg here: https://github.com/sigp/lighthouse/issues/1617#issuecomment-1117029351

From stokes to Everyone 03:45 PM

imo in the early day you can do "social slashing"

From lightclient to Everyone 03:57 PM

micah, the discussoooor

From danny to Everyone 03:58 PM

https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#transport

All implementations MUST support the TCP libp2p transport, and it MUST be enabled for both dialing and listening (i.e. outbound and inbound connections). The libp2p TCP transport supports listening on IPv4 and IPv6 addresses (and on multiple simultaneously).

Clients must support listening on at least one of IPv4 or IPv6. Clients that do not have support for listening on IPv4 SHOULD be cognizant of the potential disadvantages in terms of Internet-wide routability/support. Clients MAY choose to listen only on IPv6, but MUST be capable of dialing both IPv4 and IPv6 addresses.

ah, says one or the other in the spec! ha

From Jamie Lokier to Everyone 03:59 PM

At least it ought to be measured. I not now, then in future an IPv6-only clique (or effectively outbound-only clique) may grow large.
I have seen discv4 nodes advertising IPv4 addresses using IPv6 format as well. (::ffff:a.b.c.d). Suggests some clients are a bit confused out there.

From Potuz to Everyone 04:01 PM

Prysm should work with ipv6, but there are a very small number of ipv6 nodes running if at all
That's a quote from Nishant