---
# System prepended metadata

title: Antithesis Report 31.10.2023

---

# Antithesis Report 31.10.2023

In this report we looked at interesting logs for our sync test experiments. We used devnet-9 images throughout this report.

## Summary
* Lodestar:
    * TypeError: Cannot set property message of which has only a getter

* Lighthouse:
    * Chain removal due to RPC error, ChainFailed
    * Peer sent an invalid block, NonLinearParentRoots log emitted by lighthouse block requested from nimbus
    * RPC Error, Peer does not support the protocol

* Prysm:
    * Could not get forkchoice update argument, block does not exist

## Consensus

### Lodestar

1. Validator Client (Lodestar paired with geth)
```l!
[ 1281020] [  1459.896976] [system;capturefs;files;eth/service_lodestar-geth-0--lodestar-vc]        [I] Oct-20 20:24:59.099[]                [31merror[39m: Error proposing block slot=118, validator=0x820c…520e - Cannot set property message of  which has only a getter
source_service = "system;capturefs;files;eth/service_lodestar-geth-0--lodestar-vc"
state_id = " 1281020 "
error_message = ` TypeError: Cannot set property message of  which has only a getter`
message_count = 2
```

[Antithesis Report](https://ethereum.antithesis.com/report/-O8fyT5GFs3F3Fo5_gWqljEBVI7YypqV/_6O48QVlngUEroUR6rpwYHFx8_UVOQ_FHKCKWwi4Rvg.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiXzZPNDhRVmxuZ1VFcm9VUjZycHdZSEZ4OF9VVk9RX0ZIS0NLV3dpNFJ2Zy5odG1sIiwicmVwb3J0X2lkIjoiLU84ZnlUNUdGczNGM0ZvNV9nV3FsakVCVkk3WXlwcVYifX0sIm5iZiI6IjIwMjMtMTAtMzFUMTI6MjA6NDZaIn24m4Dsr79Jk4Rc2wi-NvR9ljWASz8ZSpwjyJoxLi8iRm7HAEglgnL9mCOEpeL4xpOY-XEIbFw-6HkXN88YhFoL)

### Lighthouse

1. Beacon node (Lighthouse paird with nethermind)
```l!
source_service = "system;capturefs;files;eth/service_lighthouse-nethermind-0--bn "
state_id = " 1221139 "
error_message = ` Oct 20 20:27:03.168 DEBG Chain removed                           op: RPC error, reason: ChainFailed { blacklist: false, failing_batch: Epoch(1) }, id: 2517229546392799496, from: 0, to: 3, end_root: 0x55ff…56aa, current_target: 0, batches: 2, peers: 1, validated_batches: 0, state: Syncing, sync_type: Head, service: sync`
message_count = 35
```

[Antithesis Report](https://ethereum.antithesis.com/report/VlWr6mLTnXV7OKcvTaAoVs693-NVaZZG/eXE6pN53n_pSd9ivxS_v8wRTwgZHpgm4LBawgAdQqT0.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiZVhFNnBONTNuX3BTZDlpdnhTX3Y4d1JUd2daSHBnbTRMQmF3Z0FkUXFUMC5odG1sIiwicmVwb3J0X2lkIjoiVmxXcjZtTFRuWFY3T0tjdlRhQW9WczY5My1OVmFaWkcifX0sIm5iZiI6IjIwMjMtMTAtMzFUMTE6NDE6MjZaIn0QAa3iXgdyKMCCIKnQ09ZoR2s-o-QF-SdWxgtHsQ_KA7k5bnbklFVA2Zi35z8g2qCoGxPjPgwnY5eC71UNozIJ)

2. Beacon node (Lighthouse paird with geth)

```l!
source_service = "system;capturefs;files;eth/service_lighthouse-geth-0--bn "
state_id = " 302657 "
error_message = ` Oct 20 20:25:52.253 DEBG Parent lookup failed                    error: Peer sent invalid block. Reason: NonLinearParentRoots`
message_count = 2
```

Nimbus peer id: 16Uiu2HAmUu85bA2otfKKyfSJRo7MwMLVm8Qjt5V2jcN4K65fiZaY
Related logs:
```
[  302609] [  1512.443844] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:51.699 DEBG Sending BlocksByRoot Request            lookup_type: Parent, peer: 16Uiu2HAmUu85bA2otfKKyfSJRo7MwMLVm8Qjt5V2jcN4K65fiZaY, block_roots: [0xd81a01ce13987be7d5ab6ea60c1875b36a874514685f0920721f214f4eda0305], method: BlocksByRoot, service: sync
[  302609] [  1512.480223] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:51.700 DEBG Sending BlobsByRoot Request             lookup_type: Parent, peer: 16Uiu2HAmUu85bA2otfKKyfSJRo7MwMLVm8Qjt5V2jcN4K65fiZaY, blob_indices: [0, 1, 2, 3, 4, 5], block_root: 0xd81a01ce13987be7d5ab6ea60c1875b36a874514685f0920721f214f4eda0305, method: BlobsByRoot, service: sync
[  302657] [  1512.610665] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:51.866 DEBG No change in canonical head             head: 0xa1c472129706ee61021006704d2af6e3450027b72bbc9b50d78f44c170913153, service: beacon
[  302657] [  1512.641856] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:51.897 DEBG Issuing engine_forkchoiceUpdated        current_slot: 123, head_block_root: 0xa1c472129706ee61021006704d2af6e3450027b72bbc9b50d78f44c170913153, head_block_hash: 0xf36663c3886a39356d24637c3604a048b9022a54b3051c2d1d334a2d7f129989, justified_block_hash: 0x0000000000000000000000000000000000000000000000000000000000000000, finalized_block_hash: 0x0000000000000000000000000000000000000000000000000000000000000000, service: exec
[  302657] [  1512.726244] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:51.982 DEBG Processing parent lookup                blocks: 5, chain_hash: 0xd81a…0305
[  302657] [  1512.997211] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.239 DEBG Invalid block received                  outcome: NonLinearParentRoots, msg: peer sent invalid block
[  302657] [  1512.997277] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.253 DEBG Parent lookup failed                    error: Peer sent invalid block. Reason: NonLinearParentRoots
[  302657] [  1513.223000] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.477 DEBG Processed HTTP API request              method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 3.677785ms
[  302657] [  1513.233050] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.487 DEBG Processed HTTP API request              method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 6.669223ms
[  302657] [  1513.233638] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.489 DEBG Processed HTTP API request              method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 6.672919ms
[  302657] [  1513.240784] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.494 DEBG Processed HTTP API request              method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 7.442235ms
[  302657] [  1513.251966] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.500 DEBG Processed HTTP API request              method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 522.494µs
[  302657] [  1513.254084] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.508 DEBG Processed HTTP API request              method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 5.504131ms
[  302657] [  1513.593619] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.847 DEBG Execution engine online                 service: exec
[  302657] [  1513.593695] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.850 DEBG Execution engine upcheck complete       state: Synced, service: exec
[  302657] [  1513.620271] [system;capturefs;files;eth/service_lighthouse-geth-0--bn]               [I] Oct 20 20:25:52.868 DEBG Parent chain processed                  result: NonFaultyFailure, chain_hash: 0xd81a…0305, service: sync
```

[Antithesis Report](https://ethereum.antithesis.com/report/OLvW8-4Tkimpm4b8SxguiFcnFX-WVntr/rQgmHuhoPrFXoL1g_ww7IF60pJqa0KjaKQvxyABuDIA.html?auth=v2.public.eyJuYmYiOiIyMDIzLTEwLTMxVDExOjA1OjAwWiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJyUWdtSHVob1ByRlhvTDFnX3d3N0lGNjBwSnFhMEtqYUtRdnh5QUJ1RElBLmh0bWwiLCJyZXBvcnRfaWQiOiJPTHZXOC00VGtpbXBtNGI4U3hndWlGY25GWC1XVm50ciJ9fX1pyow22QUJmqTYKKIu1gBNLsRbWGaOWdJaQLr7YZ6WmxbJif_YBSTy0WneonGa7AnnEJl1NRzE_Ji5Un0L3BYE)

3. Beacon node (Lighthouse paird with geth)
```l!
source_service = "system;capturefs;files;eth/service_lighthouse-geth-0--bn "
state_id = " 201883 "
error_message = ` Oct 20 20:24:53.396 DEBG RPC Error                               direction: Outgoing, score: inf, peer_id: 16Uiu2HAmGsTzCmnVTNJ2maPs1pj4jpLub1xdeMPQrMH5PucG4qLj, client: Nimbus: version: unknown, os_version: unknown, err: Peer does not support the protocol, protocol: beacon_blocks_by_root, service: libp2p`
message_count = 3
```

[Antithesis Report](https://ethereum.antithesis.com/report/Pzv_B-MjVMi-OmSYv35Le0RKWSSHf4-o/SVefQUf_HxnufagipqEvSEgRNbeFK8lnw33jr0tMZ30.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiU1ZlZlFVZl9IeG51ZmFnaXBxRXZTRWdSTmJlRks4bG53MzNqcjB0TVozMC5odG1sIiwicmVwb3J0X2lkIjoiUHp2X0ItTWpWTWktT21TWXYzNUxlMFJLV1NTSGY0LW8ifX0sIm5iZiI6IjIwMjMtMTAtMzFUMTE6MTc6NTVaIn3E8kRRSnB7w3kD9Vbj1qocf4Rjsdq8nkZdUTj8nXZh3Ln_PNH5CXz2mc66Pgok2alPIWcyElvAT7SPdujZtcYK)

4. Beacon node (Lighthouse paird with nethermind)
```l!
source_service = "system;capturefs;files;eth/service_lighthouse-nethermind-0--bn "
state_id = " 1144875 "
error_message = ` Oct 20 20:28:29.091 WARN Execution engine call failed            error: ServerMessage { code: -38001, message: "unknown payload" }, service: exec`
message_count = 2
```

Related logs:
```
[ 1144875] [  1669.400419] [system;capturefs;files;eth/service_lighthouse-nethermind-0--nethermind] [I] 20 Oct 20:28:28 | Block production for payload with id=f5abb7711b5009a6 failed - unknown payload.
[ 1144875] [  1669.401896] [system;capturefs;files;eth/service_lighthouse-nethermind-0--nethermind] [I] 20 Oct 20:28:28 | Error when handling ID 1, engine_getPayloadV3(0xf5abb7711b5009a6) | {"jsonrpc":"2.0","error":{"code":-38001,"message":"unknown payload"},"id":1}
[ 1144875] [  1669.837317] [system;capturefs;files;eth/service_lighthouse-nethermind-0--bn]         [I] Oct 20 20:28:29.091 WARN Execution engine call failed            error: ServerMessage { code: -38001, message: "unknown payload" }, service: exec
```

[Antithesis Report](https://ethereum.antithesis.com/report/HVw99cQ9_pVXz9bjbiPkGTumHl8aAsgM/CHklWL8Ca1oEBU1b9mSbNlqnirb2bZEls4-YYxgPiwc.html?auth=v2.public.eyJuYmYiOiIyMDIzLTEwLTMxVDExOjQxOjM0WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJDSGtsV0w4Q2Exb0VCVTFiOW1TYk5scW5pcmIyYlpFbHM0LVlZeGdQaXdjLmh0bWwiLCJyZXBvcnRfaWQiOiJIVnc5OWNROV9wVlh6OWJqYmlQa0dUdW1IbDhhQXNnTSJ9fX3vWGSsy8vU3cqj3KW1fIgQrhPvljlTYh-GGOOl_Q_6H2pKUwChVKWWu3QzMRuuKWHlKDAIWFPFdnAOb_NSiX0F)

### Prysm

1. Beacon node (Prysm paired with geth)
```l!
source_service = "system;capturefs;files;eth/service_prysm-geth-0--prysm-bn "
state_id = " 1292982 "
error_message = ` time="2023-10-20 20:25:13" level=error msg="Could not get forkchoice update argument" error="block does not exist" prefix=blockchain`
message_count = 1
```

Related logs:
```
[ 1292982] [  1473.798484] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn]              [I] time="2023-10-20 20:25:13" level=warning msg="Could not update head" error="could not apply weight changes: context deadline exceeded" prefix=blockchain
[ 1292982] [  1473.877188] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn]              [I] time="2023-10-20 20:25:13" level=warning msg="could not determine node weight" prefix=blockchain root=0x0000000000000000000000000000000000000000000000000000000000000000
[ 1292982] [  1473.880793] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn]              [I] time="2023-10-20 20:25:13" level=debug msg="Head block is not the received block" headRoot=0x0000000000000000000000000000000000000000000000000000000000000000 headWeight=0 prefix=blockchain receivedRoot=0xaae3a85a60ce86283533bd873ea0d8cf90ceb91514c21162f6b7ef0f6e50155b receivedWeight=0
[ 1292982] [  1473.953665] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn]              [I] time="2023-10-20 20:25:13" level=error msg="Could not get forkchoice update argument" error="block does not exist" prefix=blockchain
```

[Antithesis Report](https://ethereum.antithesis.com/report/-jVolAETwVNFUaGerWWMCpHYOqKpVswQ/eNJdOkc2CFjLqFWk8-5ioWOZpO7fBpk8TrcBVAP-tAA.html?auth=v2.public.eyJuYmYiOiIyMDIzLTEwLTMxVDEyOjEwOjMzWiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJlTkpkT2tjMkNGakxxRldrOC01aW9XT1pwTzdmQnBrOFRyY0JWQVAtdEFBLmh0bWwiLCJyZXBvcnRfaWQiOiItalZvbEFFVHdWTkZVYUdlcldXTUNwSFlPcUtwVnN3USJ9fX2L-YZp9U5JGI-lFAKgQUinWgRsqEN7CVqML-KiUVejseJPewIPbzzZaQxnB3BlceC5UUM4MPwQSGxCjOhihgcN)