# Hoodi Fusaka Prysm peers dropping issue
This documents presents the Prysm peering issue regarding Hoodi Fusaka in a "top down" fashion: Starting with symptoms, then digging into root issue, then establishing some hypothesis.

:::info
**TL;DR:**

1. Starting at Hoodi, Fusaka, a few attester nodes are struggling, producing attestations with `target.epoch - source.epoch > 2`. These nodes may be Prysm, or not.
2. When a Prysm node receives such an attestation, in some case it may reject it, and then downscore, and eventually disconnect, the peer gossiping it.
:::

## Symptom

If a Prysm (super) beacon node is started on Hoodi, Fusaka, then after a few minutes the Prysm peer count increases over time, and the non Prysm peer count (mainly, Lighthouse) decreases over time.
The reduction can take from a few to a dozen of hours.

A few examples:


## Analysis

:::info
The word "bad" attestation refers to an invalid attestation **from the point of view of the receiving Prysm node.**
:::

The non-Prysm peer count decreases over time, because Prysm disconnect non Prysm peers.

[2025-11-03 11:31:56.72] DEBUG p2p: Peer disconnected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux direction=Inbound multiAddr=/ip4/185.209.177.242/udp/59324/quic-v1/p2p/16Uiu2HAm3ckhuZbMDnmAnhwuvJNMczpo3C9t3TFwkMu2xsMNM9EX remainingActivePeers=73

A few moments before this disconnection event, we have:

[2025-11-03 11:31:54.25] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1663009, attSlotInEpoch: 1, attOldCommitteeIndex: 0, attCommitteeIndex: 24, attBlockRoot: 0x356dfc9ff70f501fe60e3729fc465aee3140b66bb6abdec97f72fba334053c0e, attSource: {root: 0x46f89f3700f2f2bacee1fdfb6f4674b71cc1b2547746c951fc2fe1aa8ac55e3b, epoch: 51966}, attTarget: {root: 0x356dfc9ff70f501fe60e3729fc465aee3140b66bb6abdec97f72fba334053c0e, epoch: 51969}: attester 1136516 is not a member of the committee gossipScore=14.971348702484992 multiaddress=/ip4/185.209.177.242/udp/59324/quic-v1 peerID=16Uiu2HAm3ckhuZbMDnmAnhwuvJNMczpo3C9t3TFwkMu2xsMNM9EX topic=/eth2/e2abcca4/beacon_attestation_24/ssz_snappy
[2025-11-03 11:31:54.25] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1663009, attSlotInEpoch: 1, attOldCommitteeIndex: 0, attCommitteeIndex: 24, attBlockRoot: 0x356dfc9ff70f501fe60e3729fc465aee3140b66bb6abdec97f72fba334053c0e, attSource: {root: 0x46f89f3700f2f2bacee1fdfb6f4674b71cc1b2547746c951fc2fe1aa8ac55e3b, epoch: 51966}, attTarget: {root: 0x356dfc9ff70f501fe60e3729fc465aee3140b66bb6abdec97f72fba334053c0e, epoch: 51969}: attester 1135832 is not a member of the committee gossipScore=14.971348702484992 multiaddress=/ip4/185.209.177.242/udp/59324/quic-v1 peerID=16Uiu2HAm3ckhuZbMDnmAnhwuvJNMczpo3C9t3TFwkMu2xsMNM9EX topic=/eth2/e2abcca4/beacon_attestation_24/ssz_snappy

The `16Uiu2HAm3ckhuZbMDnmAnhwuvJNMczpo3C9t3TFwkMu2xsMNM9EX` Lighthouse peer sent us the following attestation via the `/eth2/e2abcca4/beacon_attestation_24/ssz_snappy` topic:

slot: 1663009 (slot in epoch: 0)
validator index: 1136516
committee index: 24
root: 0x356dfc9ff70f501fe60e3729fc465aee3140b66bb6abdec97f72fba334053c0e
source:
  epoch: 51966
  root: 0x46f89f3700f2f2bacee1fdfb6f4674b71cc1b2547746c951fc2fe1aa8ac55e3b
target:
  epoch: 51969
  root: 0x356dfc9ff70f501fe60e3729fc465aee3140b66bb6abdec97f72fba334053c0e

The receiving Prysm node complains about validator with index `1136516` is **not** in the committee with index `24` for slot `1663009`.

Indeed, according to [beaconcha.in](https://hoodi.beaconcha.in/validator/1136516#attestations), the corresponding validator had **not** to vote on slot `1663009`, but on slot `1663021` for this epoch.


Analyzing a few dozen of bad attestations, the following pattern appears:
- `target.Epoch - source.Epoch > 2`
- `slot % SLOTS_PER_EPOCH` is very low (very often: `0`, `1` or `2`)
- Corresponding validators miss the attestaion for this epoch

## Hypothesis

On ~24H run, validators corresponding to only 4 "entities" are responsible of `>50%` of these bad attestations:


Past blocks proposed by these entities are the following:
- `Whale_0xafa025`: no graffiti
- `Whale_0x45085d`: no graffiti
- `Whale_0x4473dc`: ebunker.io
- `Whale_0xe0716b`: SST

Assuming only Prysm (is that true?) has an blank default graffiti, we can assume `Whale_0xafa025` and `Whale_0x45085d` run Prysm.

Crafting an attestation with `target.Epoch - source.Epoch > 2` may be a sign of a struggling node. Such a struggling node may use the wrong RANDAO (in the canonical chain point of view), such sending an attestion for the wrong (in the canonical chain point of view) slot/committee.

2 issues arise here:
1. A set of attesters produces "bad" (from the receiver Prysm point of view) attestations. (Note: These attestations are very probably "good" in the point of view of the attester).
2. Peers (any implementations) receive these attestations, and then gossip them. Then the receiving Prysm node rejects them and downscore the peer which gossiped them (because it thinks they are "bad").

[2025-11-02 23:22:07.71] DEBUG sync: Gossip message was rejected agent=nimbus error=attSlot: 1659360, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 39, attBlockRoot: 0xdf5f08f5766d12b21651d8d793041f294d8aececd81a4fd2eb6851f4c69aab91, attSource: {root: 0x7f80d45caf92608d03ba7406acb5dad6db30040886f0770be08d9e77559770f7, epoch: 51852}, attTarget: {root: 0xdf5f08f5766d12b21651d8d793041f294d8aececd81a4fd2eb6851f4c69aab91, epoch: 51855}: attester 1237849 is not a member of the committee gossipScore=0.010784395411173386 multiaddress=/ip4/68.72.211.75/tcp/51336 peerID=16Uiu2HAmTZFGon4CX2MF6A83RiBSMs4JG11o315PQVundF8zPkyt topic=/eth2/e2abcca4/beacon_attestation_39/ssz_snappy
[2025-11-02 23:34:55.78] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659424, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 40, attBlockRoot: 0x2b80c7f9ebb91fcdceb6ff2fb4276e7ba77abc247807bda66f96ea4c10f132cd, attSource: {root: 0xf768bee768e25b677233ae8ae7cbbf7b7e698be9e0db92498eadcad29f2aa4c6, epoch: 51854}, attTarget: {root: 0x2b80c7f9ebb91fcdceb6ff2fb4276e7ba77abc247807bda66f96ea4c10f132cd, epoch: 51857}: attester 1227317 is not a member of the committee gossipScore=3.8793362436731655 multiaddress=/ip4/15.204.105.76/udp/9001/quic-v1 peerID=16Uiu2HAmDMFTkxG4PCb4gEB1fBMbayAQ4jr4WjrdnFHXabmyNUZW topic=/eth2/e2abcca4/beacon_attestation_40/ssz_snappy
[2025-11-02 23:41:27.58] DEBUG sync: Gossip message was rejected agent=nimbus error=attSlot: 1659456, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 24, attBlockRoot: 0x2fa4f0342f31f0994444d0cabb1a0f9fc8a982b40844ce96d9670b711743c562, attSource: {root: 0x2ef0e9fa907a2d844a47a0b325b92035272b4d1a893b9f839912b47b4b5ead73, epoch: 51855}, attTarget: {root: 0x2fa4f0342f31f0994444d0cabb1a0f9fc8a982b40844ce96d9670b711743c562, epoch: 51858}: attester 1141209 is not a member of the committee gossipScore=32.72 multiaddress=/ip4/134.209.67.238/tcp/44832 peerID=16Uiu2HAmBPYaRJX3gFnDbH3KXy76WkPLj8Ch5KnZbwtiPMwfvbQf topic=/eth2/e2abcca4/beacon_attestation_24/ssz_snappy
[2025-11-03 00:00:54.20] DEBUG sync: Gossip message was rejected agent=nimbus error=attSlot: 1659554, attSlotInEpoch: 2,attOldCommitteeIndex: 0, attCommitteeIndex: 39, attBlockRoot: 0x74b612f64930d083052b2b4cef388831296b8856e8253bb4706f9aac945de9c6, attSource: {root: 0xecde84140a8deb46c85dd6a0d891ae9d5940f195a8d80f1e1ae2f55bdbf708f3, epoch: 51858}, attTarget: {root: 0x74b612f64930d083052b2b4cef388831296b8856e8253bb4706f9aac945de9c6, epoch: 51861}: attester 1237804 is not a member of the committee gossipScore=4.5658568730361715 multiaddress=/ip4/38.97.200.229/tcp/57232 peerID=16Uiu2HAm3774mKuC8Po87avCcdd8Lh5ahgfAB8a9sLmtScex8fLR topic=/eth2/e2abcca4/beacon_attestation_39/ssz_snappy
[2025-11-03 00:06:56.21] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659584, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 62, attBlockRoot: 0xe1e24dc0b29441fb541f3b76474e55dbe83a91f79d5c1aa815260392bd04cede, attSource: {root: 0x2820fd2f174e849d2b0a9753bcf0b6d767892cb9c191dfd16acbd9424aefb37e, epoch: 51859}, attTarget: {root: 0xe1e24dc0b29441fb541f3b76474e55dbe83a91f79d5c1aa815260392bd04cede, epoch: 51862}: attester 1141203 is not a member of the committee gossipScore=32.72 multiaddress=/ip4/57.129.73.144/udp/47264/quic-v1 peerID=16Uiu2HAmPuVk37d2XS7Mc4jsKiAfHwhxUuz2zd1m9vRUUBHRdcAs topic=/eth2/e2abcca4/beacon_attestation_62/ssz_snappy
[2025-11-03 00:07:06.19] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659585, attSlotInEpoch: 1,attOldCommitteeIndex: 0, attCommitteeIndex: 12, attBlockRoot: 0xe1e24dc0b29441fb541f3b76474e55dbe83a91f79d5c1aa815260392bd04cede, attSource: {root: 0x2820fd2f174e849d2b0a9753bcf0b6d767892cb9c191dfd16acbd9424aefb37e, epoch: 51859}, attTarget: {root: 0xe1e24dc0b29441fb541f3b76474e55dbe83a91f79d5c1aa815260392bd04cede, epoch: 51862}: attester 1142490 is not a member of the committee gossipScore=0 multiaddress=/ip4/118.153.253.215/udp/59001/quic-v1 peerID=16Uiu2HAm13HA23csqTEJsgH9CTLkcT5jpF58K9r6Eniryo46qdwh topic=/eth2/e2abcca4/beacon_attestation_12/ssz_snappy
[2025-11-03 00:07:07.78] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.1-1fb94ce/x86_64-linux error=attSlot: 1659585, attSlotInEpoch: 1,attOldCommitteeIndex: 0, attCommitteeIndex: 30, attBlockRoot: 0xe1e24dc0b29441fb541f3b76474e55dbe83a91f79d5c1aa815260392bd04cede, attSource: {root: 0x2820fd2f174e849d2b0a9753bcf0b6d767892cb9c191dfd16acbd9424aefb37e, epoch: 51859}, attTarget: {root: 0xe1e24dc0b29441fb541f3b76474e55dbe83a91f79d5c1aa815260392bd04cede, epoch: 51862}: attester 1131307 is not a member of the committee gossipScore=28.340906481446822 multiaddress=/ip4/37.27.100.226/udp/9010/quic-v1 peerID=16Uiu2HAmRVAmJWs5B9vTwKVeq4Lsb4Z71RYh5F6Nknr6CwLRVM2X topic=/eth2/e2abcca4/beacon_attestation_30/ssz_snappy
[2025-11-03 00:13:19.45] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659616, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 59, attBlockRoot: 0xa2efcac5c2af69a7a6508f254832087170f6708cef1fec604b71cfbb04880fce, attSource: {root: 0x4f3bd096787024f2240bc79de354e547419b90ccd863ea8d4d65887605ad069f, epoch: 51860}, attTarget: {root: 0xa2efcac5c2af69a7a6508f254832087170f6708cef1fec604b71cfbb04880fce, epoch: 51863}: attester 1237861 is not a member of the committee gossipScore=3.5708335911394884 multiaddress=/ip4/51.161.13.48/tcp/9000 peerID=16Uiu2HAmP9GmF3ngFrSVdK5cvLhqSjFQHSyVzxq1x4MnA6KxCS9S topic=/eth2/e2abcca4/beacon_attestation_59/ssz_snappy
[2025-11-03 00:19:44.87] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659648, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 2, attBlockRoot: 0x2431459a567b49317a1706885e0a9d79a0ec1fb50906f3728ff883ac3a9a61bc, attSource: {root: 0x789075b18c00a8052c5b184591198b943b61197d7b5c3ac062208125d1c199da, epoch: 51861}, attTarget: {root: 0x2431459a567b49317a1706885e0a9d79a0ec1fb50906f3728ff883ac3a9a61bc, epoch: 51864}: attester 1141860 is not a member of the committee gossipScore=31.896329034970442 multiaddress=/ip4/34.185.169.230/udp/9001/quic-v1 peerID=16Uiu2HAkxe8shWX6p6T8rPJFp26bwyimfCLbtuJCWxg16yyfL48e topic=/eth2/e2abcca4/beacon_attestation_2/ssz_snappy
[2025-11-03 00:19:45.08] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.1-1fb94ce/x86_64-linux error=attSlot: 1659648, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 34, attBlockRoot: 0x2d7b883b1cd2efd0fabaa3bef1ffa8dacb8d98eae142910f01d1eebc3fa821d2, attSource: {root: 0x789075b18c00a8052c5b184591198b943b61197d7b5c3ac062208125d1c199da, epoch: 51861}, attTarget: {root: 0x2d7b883b1cd2efd0fabaa3bef1ffa8dacb8d98eae142910f01d1eebc3fa821d2, epoch: 51864}: attester 1235214 is not a member of the committee gossipScore=1.2009145124740417 multiaddress=/ip4/51.158.203.60/udp/9001/quic-v1 peerID=16Uiu2HAmLr1HKYXTz22rjCz8ocecdj8Styr14usC7RyWSBV331RT topic=/eth2/e2abcca4/beacon_attestation_34/ssz_snappy
[2025-11-03 00:20:07.70] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659650, attSlotInEpoch: 2,attOldCommitteeIndex: 0, attCommitteeIndex: 40, attBlockRoot: 0x2d7b883b1cd2efd0fabaa3bef1ffa8dacb8d98eae142910f01d1eebc3fa821d2, attSource: {root: 0x789075b18c00a8052c5b184591198b943b61197d7b5c3ac062208125d1c199da, epoch: 51861}, attTarget: {root: 0x2d7b883b1cd2efd0fabaa3bef1ffa8dacb8d98eae142910f01d1eebc3fa821d2, epoch: 51864}: attester 1225556 is not a member of the committee gossipScore=32.72 multiaddress=/ip4/57.129.73.144/udp/47264/quic-v1 peerID=16Uiu2HAmPuVk37d2XS7Mc4jsKiAfHwhxUuz2zd1m9vRUUBHRdcAs topic=/eth2/e2abcca4/beacon_attestation_40/ssz_snappy
[2025-11-03 00:26:30.68] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659682, attSlotInEpoch: 2,attOldCommitteeIndex: 0, attCommitteeIndex: 40, attBlockRoot: 0x2aecc4500c66d6731d6b92fc77a5315cabd231804c66ec92b9f48d0230f3addc, attSource: {root: 0xc23f6ed05264d80e3a89312790acb65e8d58109645e7c3479075610c266c753b, epoch: 51862}, attTarget: {root: 0x2aecc4500c66d6731d6b92fc77a5315cabd231804c66ec92b9f48d0230f3addc, epoch: 51865}: attester 1237820 is not a member of the committee gossipScore=32.72 multiaddress=/ip4/51.79.99.122/tcp/9000 peerID=16Uiu2HAmHbzKbPaD2QcyNVMoPRV1SSuGBFXZd3bbf9RNk8syrNDc topic=/eth2/e2abcca4/beacon_attestation_40/ssz_snappy
[2025-11-03 00:32:42.61] DEBUG sync: Gossip message was rejected agent=nimbus error=attSlot: 1659713, attSlotInEpoch: 1,attOldCommitteeIndex: 0, attCommitteeIndex: 40, attBlockRoot: 0x51a4bcf4fc721230c5721842fd0b7f356d92a52a90b62f402432d7e20f159ce2, attSource: {root: 0xb9f7420d35c9d66b27896c68917b3e9b547136cf58792473589f16cfe9dfdd0b, epoch: 51863}, attTarget: {root: 0x51a4bcf4fc721230c5721842fd0b7f356d92a52a90b62f402432d7e20f159ce2, epoch: 51866}: attester 1183916 is not a member of the committee gossipScore=0.19348377248195822 multiaddress=/ip4/194.33.40.246/tcp/42156 peerID=16Uiu2HAmVJD4hYVmpgxKkPhDQzRVb3wCwSDcj48XgGBEesK9P3QR topic=/eth2/e2abcca4/beacon_attestation_40/ssz_snappy
[2025-11-03 00:38:55.18] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659744, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 41, attBlockRoot: 0x2a92b55e51514a37aa1ee7982b6a88a45e0ac657500fce89befe441ad338406e, attSource: {root: 0xb4f77ea416965f59fddb72ade677173ece17027680ba2335e365d8beb8ca6080, epoch: 51864}, attTarget: {root: 0x2a92b55e51514a37aa1ee7982b6a88a45e0ac657500fce89befe441ad338406e, epoch: 51867}: attester 1227307 is not a member of the committee gossipScore=12.45718621237272 multiaddress=/ip4/65.109.123.254/udp/49001/quic-v1 peerID=16Uiu2HAm5B1y2x8sDqvw8re5zTDW1tbSEhdsQZrpfCuHFeSCadXQ topic=/eth2/e2abcca4/beacon_attestation_41/ssz_snappy
[2025-11-03 00:51:43.26] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659808, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 18, attBlockRoot: 0xbf939507e504fe87f86c33ad16c0c56b09e528bcdc1bd901e1ef25ebaa71a4df, attSource: {root: 0xf12e56406f19aa69c38c1e4ccc2dde3623c8dbde3294d4b7d66b749e1e182ed5, epoch: 51866}, attTarget: {root: 0xbf939507e504fe87f86c33ad16c0c56b09e528bcdc1bd901e1ef25ebaa71a4df, epoch: 51869}: attester 1237818 is not a member of the committee gossipScore=4.796610414161459 multiaddress=/ip4/71.90.237.100/udp/9001/quic-v1 peerID=16Uiu2HAkwxCPfQzLhC7fZRiPWp7Lez5CK1kmVNt7pa2tDrSsCV8h topic=/eth2/e2abcca4/beacon_attestation_18/ssz_snappy
[2025-11-03 00:52:05.89] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659810, attSlotInEpoch: 2,attOldCommitteeIndex: 0, attCommitteeIndex: 50, attBlockRoot: 0xbf939507e504fe87f86c33ad16c0c56b09e528bcdc1bd901e1ef25ebaa71a4df, attSource: {root: 0xf12e56406f19aa69c38c1e4ccc2dde3623c8dbde3294d4b7d66b749e1e182ed5, epoch: 51866}, attTarget: {root: 0xbf939507e504fe87f86c33ad16c0c56b09e528bcdc1bd901e1ef25ebaa71a4df, epoch: 51869}: attester 1237852 is not a member of the committee gossipScore=9.655407025144855 multiaddress=/ip4/91.134.22.131/tcp/9000 peerID=16Uiu2HAkyx5wyRX4P9ep7K7tUu11VjPYxTczxSL6adYqiSa5bMgz topic=/eth2/e2abcca4/beacon_attestation_50/ssz_snappy
[2025-11-03 00:52:06.24] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659810, attSlotInEpoch: 2,attOldCommitteeIndex: 0, attCommitteeIndex: 23, attBlockRoot: 0xbf939507e504fe87f86c33ad16c0c56b09e528bcdc1bd901e1ef25ebaa71a4df, attSource: {root: 0xf12e56406f19aa69c38c1e4ccc2dde3623c8dbde3294d4b7d66b749e1e182ed5, epoch: 51866}, attTarget: {root: 0xbf939507e504fe87f86c33ad16c0c56b09e528bcdc1bd901e1ef25ebaa71a4df, epoch: 51869}: attester 1237847 is not a member of the committee gossipScore=32.72 multiaddress=/ip4/18.212.219.236/udp/9001/quic-v1 peerID=16Uiu2HAm3rh17cEj5ouLuNYnRQwLduD3XLtJajSg7rCHt3VsHrDu topic=/eth2/e2abcca4/beacon_attestation_23/ssz_snappy
[2025-11-03 01:04:32.49] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659872, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 40, attBlockRoot: 0xff76b2b2c95cdbdea6fc8e4ae7e0de1c629bbea53b97b346cc24d0b8ce9f3a68, attSource: {root: 0xf094bc2adb9ae1fcfe5846ee2aa087c50fb59c05a7894e37b0259725a39c8e89, epoch: 51868}, attTarget: {root: 0xff76b2b2c95cdbdea6fc8e4ae7e0de1c629bbea53b97b346cc24d0b8ce9f3a68, epoch: 51871}: attester 1237822 is not a member of the committee gossipScore=32.72 multiaddress=/ip4/37.187.132.95/udp/8402/quic-v1 peerID=16Uiu2HAmHmEJncQ61YYjzLUzga3G67vXxHb5bwfqxa1paa3GMGwE topic=/eth2/e2abcca4/beacon_attestation_40/ssz_snappy
[2025-11-03 01:04:33.11] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=attSlot: 1659872, attSlotInEpoch: 0,attOldCommitteeIndex: 0, attCommitteeIndex: 39, attBlockRoot: 0xeea5bfff5e6c543694b7cf200f9a45fa0b46e7b07fad3b94439308c6c9dd7797, attSource: {root: 0xf094bc2adb9ae1fcfe5846ee2aa087c50fb59c05a7894e37b0259725a39c8e89, epoch: 51868}, attTarget: {root: 0xeea5bfff5e6c543694b7cf200f9a45fa0b46e7b07fad3b94439308c6c9dd7797, epoch: 51871}: attester 1086575 is not a member of the committee gossipScore=0.009055670461624936 multiaddress=/ip4/57.129.73.156/udp/26797/quic-v1 peerID=16Uiu2HAmJDjMeEqKcaYm34UoF3DnfjtuUhLqfMLL84rJiT1yvLxk topic=/eth2/e2abcca4/beacon_attestation_39/ssz_snappy
...

https://gist.github.com/nalepae/e00dff7ebde0bffe514e683d037ae208

For some reason, these "bad" attestations are mainly received from Lighthouse nodes.
(But there is also some more rare cases from other clients, including Prysm.)

:::info
Issue `2.` is probably here for a long time in Prysm.

Prysm, in some cases, pulls the wrong state when validating incoming attestation (when `target.epoch - source.epoch > 2`), and then rejects unfairly the attestation.

However, this issue is visible only when such attestations with `target.epoch - source.epoch > 2` happens.

For some reason, this case seems to happen only on Hoodi, Fusaka. (Probably because some nodes are now de-facto super nodes without enough Hardware.)

**We expect a fix early this week.**
:::

**Additional note:**
The corresponding "aggregated attestation" issue exists on the `topic=/eth2/e2abcca4/beacon_aggregate_and_proof/ssz_snappy` topic.

Very probably, both issues are related.

[2025-11-03 11:31:56.57] DEBUG sync: Gossip message was rejected agent=Lighthouse/v8.0.0-rc.2-b59feb0/x86_64-linux error=could not validate index in committee: validator index 1135271 is not within the committee: [824762 66421 15155 945569 212090 1126843 172314 699742 1050755 735971 475983 958746 596150 520209 193326 149385 279605 912131 611397 586127 739060 297921 19979 237858 92222 457650 350101 823805 167360 1092284 1225498 38404 1153746 235376 81476 958375 130965 906150 1004899 734325 52891 153289 494095 1019096 1029922 992008 818282 1234869 141693 70277 434349 780087 156881 814813 838898 269567 581976 189518 1227886 778302 171380 207636 941593 446850 494517 446630 588808 1057943 1173662 976512 613521 1223786 956554 1182279 238701 980023 262079 630359 468976 103389 1122935 566010 693178 544109 962468 371480 261147 733520 52556 680967 365174 44183 512513 500665 910766 84440 1006674 962188 1039196 805947 878657 302764 1016072 816656 549899 1519 890606 1068374 675852 562883 473909 172861 362070 147799 405312 610738 1186134 1019917 781532 805766 319739 1007770 361829 616165 769566 184599 1007088 882438 681816 607586 572232 311605 714689 788716 629732 961415 235547 423071 1007658 342900 1231729 1239256 259698 920225 18069 591535 916694 44632 34548 120428 444548 1034800 991620 1049505 870133 930413 758975 942061 230756 445722 1132855 331565 315994 63261 411843 859103 1227243 925493 723043 144384 421358 290902 953023 1195999 375740 267389 621500 816017 40308 288320 1074835 1176269 257820 1233252 697788 63689 61189 1209366 1226643 1234220 158661 659600 616509 80981 1002931 553573 643488 698016 927211 774508 277317 718314 888409 571233 943865 785476 174768 432450 697909 191591 773605 546408 363931 1161921 796700 519499 495869 671385 1113737 110994 223090 524745 446317 429529 353977 1218865 727316 777397 138428 1146789 630498 136697 1118006 316446 1238142 831945 241759 381797 931984 118957 903465 939437 563705 555888 344209 961380 612048 505125 296911 1165292 40860 910810 1502 982423 318747 829464 643009 272734 1166879 12657 597240 1106311 1238823 1095548 441676 210535 687859 893801 1010598 571881 862449 515270 227404 520525 186962 236760 133450 1194875 233676 688239 396131 1111434 624208 551315 394811 1127697 194871 1078861 1205079 685096 580615 677864 188065 96517 661295 150250 367120 813294 86151 787982 328079 190125 894482 1202134 591865 769797 488974 963859 1559 676220 96224 76501 1079619 537797 930173 734453 819391 582651 1073303 1071333 170264 410298 270845 873620 640 361040 1046998 627971 1035366 415057 555033 186748 511282 353861 85218 1078514 327393 712665 948514 1037060 542497 147945 869705 1130489 302237 938803 622908 560535 670047 83654 43393 309562 412953 703114 533509 558549 406001 154727 1063286 708502 949291 403979 454120 1099649 961323 931879 191478 537915 374382 744928 515269 1097377 175597 100060 1111124 313545 803333 721117 542283 1236903 896662 38157 242165 43332 1072618 606775 1020739 931148 503348 252211 412504 53424 397818 517609 119545 744004 289329 869517 12029 323977 1146024 772461 401385 443555 193007 1054721 33444 73509 656988 796516 650454 889862 12488 403415 194960 187596 585603 799426 178228 253780 823678 478123 174661 532788 850426 221734 1028602 41520 582627 277590 892460 552593 215452 13388 1128699 389444 74566 41650 732145 92339 863601 876403 288859 752477 853521 84443 1048775 419391 56047 1042976 620285 909750 930859 3639 157969 1108208 10124 424635 1034540 1209480 1160037 96054 23631 62736 673523 743534 408510 1161653 369727 211019 365409 488585 680175 650353 822559 837121 322087 708202 388905 5605 710993 458235 1190694 202539 116104 221592 1039046 412695 421740 1188487 90866 639168 637431 1215902 193481 1164259 568624 503653 460822 922232 688254 1209587 1002040 1111526 445219 844773 356952 477094 893271 979390 191341 373562 191829 571631 594928 1064284 781189 594354 198302 53346 555234 852490 1163709 834385 774658 795128 963239 800482 1028079 9265 1158714 330552 975665 1161826 1149657 536796 491024 522534 625761 30814 723343 840126 825254 1166500 1218881 332681 1209870 290890 482943 751539 566961 1127972 1051088 119939 300059 1002843 568971 369640 907025 782996 163023 399729 708275 724869 1052955 39140 185022 885726 841414 495567 437213 127010 320043 1115343 644946 497044 786753 641009 491491 462091 226010 676384 970463 241003 397656 660847 1190633 585890 39878 752575 1149246 331521] gossipScore=25.414366971875225 multiaddress=/ip4/86.111.48.83/udp/9001/quic-v1 peerID=16Uiu2HAmSAn2s3VS1AKxW9uF2fBn5zDDL4htnsqpp21cXnfqopY8 topic=/eth2/e2abcca4/beacon_aggregate_and_proof/ssz_snappy