# Antithesis Report 31.10.2023
In this report we looked at interesting logs for our sync test experiments. We used devnet-9 images throughout this report.
## Summary
* Lodestar:
* TypeError: Cannot set property message of which has only a getter
* Lighthouse:
* Chain removal due to RPC error, ChainFailed
* Peer sent an invalid block, NonLinearParentRoots log emitted by lighthouse block requested from nimbus
* RPC Error, Peer does not support the protocol
* Prysm:
* Could not get forkchoice update argument, block does not exist
## Consensus
### Lodestar
1. Validator Client (Lodestar paired with geth)
```l!
[ 1281020] [ 1459.896976] [system;capturefs;files;eth/service_lodestar-geth-0--lodestar-vc] [I] Oct-20 20:24:59.099[] [31merror[39m: Error proposing block slot=118, validator=0x820c…520e - Cannot set property message of which has only a getter
source_service = "system;capturefs;files;eth/service_lodestar-geth-0--lodestar-vc"
state_id = " 1281020 "
error_message = ` TypeError: Cannot set property message of which has only a getter`
message_count = 2
```
[Antithesis Report](https://ethereum.antithesis.com/report/-O8fyT5GFs3F3Fo5_gWqljEBVI7YypqV/_6O48QVlngUEroUR6rpwYHFx8_UVOQ_FHKCKWwi4Rvg.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiXzZPNDhRVmxuZ1VFcm9VUjZycHdZSEZ4OF9VVk9RX0ZIS0NLV3dpNFJ2Zy5odG1sIiwicmVwb3J0X2lkIjoiLU84ZnlUNUdGczNGM0ZvNV9nV3FsakVCVkk3WXlwcVYifX0sIm5iZiI6IjIwMjMtMTAtMzFUMTI6MjA6NDZaIn24m4Dsr79Jk4Rc2wi-NvR9ljWASz8ZSpwjyJoxLi8iRm7HAEglgnL9mCOEpeL4xpOY-XEIbFw-6HkXN88YhFoL)
### Lighthouse
1. Beacon node (Lighthouse paird with nethermind)
```l!
source_service = "system;capturefs;files;eth/service_lighthouse-nethermind-0--bn "
state_id = " 1221139 "
error_message = ` Oct 20 20:27:03.168 DEBG Chain removed op: RPC error, reason: ChainFailed { blacklist: false, failing_batch: Epoch(1) }, id: 2517229546392799496, from: 0, to: 3, end_root: 0x55ff…56aa, current_target: 0, batches: 2, peers: 1, validated_batches: 0, state: Syncing, sync_type: Head, service: sync`
message_count = 35
```
[Antithesis Report](https://ethereum.antithesis.com/report/VlWr6mLTnXV7OKcvTaAoVs693-NVaZZG/eXE6pN53n_pSd9ivxS_v8wRTwgZHpgm4LBawgAdQqT0.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiZVhFNnBONTNuX3BTZDlpdnhTX3Y4d1JUd2daSHBnbTRMQmF3Z0FkUXFUMC5odG1sIiwicmVwb3J0X2lkIjoiVmxXcjZtTFRuWFY3T0tjdlRhQW9WczY5My1OVmFaWkcifX0sIm5iZiI6IjIwMjMtMTAtMzFUMTE6NDE6MjZaIn0QAa3iXgdyKMCCIKnQ09ZoR2s-o-QF-SdWxgtHsQ_KA7k5bnbklFVA2Zi35z8g2qCoGxPjPgwnY5eC71UNozIJ)
2. Beacon node (Lighthouse paird with geth)
```l!
source_service = "system;capturefs;files;eth/service_lighthouse-geth-0--bn "
state_id = " 302657 "
error_message = ` Oct 20 20:25:52.253 DEBG Parent lookup failed error: Peer sent invalid block. Reason: NonLinearParentRoots`
message_count = 2
```
Nimbus peer id: 16Uiu2HAmUu85bA2otfKKyfSJRo7MwMLVm8Qjt5V2jcN4K65fiZaY
Related logs:
```
[ 302609] [ 1512.443844] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:51.699 DEBG Sending BlocksByRoot Request lookup_type: Parent, peer: 16Uiu2HAmUu85bA2otfKKyfSJRo7MwMLVm8Qjt5V2jcN4K65fiZaY, block_roots: [0xd81a01ce13987be7d5ab6ea60c1875b36a874514685f0920721f214f4eda0305], method: BlocksByRoot, service: sync
[ 302609] [ 1512.480223] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:51.700 DEBG Sending BlobsByRoot Request lookup_type: Parent, peer: 16Uiu2HAmUu85bA2otfKKyfSJRo7MwMLVm8Qjt5V2jcN4K65fiZaY, blob_indices: [0, 1, 2, 3, 4, 5], block_root: 0xd81a01ce13987be7d5ab6ea60c1875b36a874514685f0920721f214f4eda0305, method: BlobsByRoot, service: sync
[ 302657] [ 1512.610665] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:51.866 DEBG No change in canonical head head: 0xa1c472129706ee61021006704d2af6e3450027b72bbc9b50d78f44c170913153, service: beacon
[ 302657] [ 1512.641856] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:51.897 DEBG Issuing engine_forkchoiceUpdated current_slot: 123, head_block_root: 0xa1c472129706ee61021006704d2af6e3450027b72bbc9b50d78f44c170913153, head_block_hash: 0xf36663c3886a39356d24637c3604a048b9022a54b3051c2d1d334a2d7f129989, justified_block_hash: 0x0000000000000000000000000000000000000000000000000000000000000000, finalized_block_hash: 0x0000000000000000000000000000000000000000000000000000000000000000, service: exec
[ 302657] [ 1512.726244] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:51.982 DEBG Processing parent lookup blocks: 5, chain_hash: 0xd81a…0305
[ 302657] [ 1512.997211] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.239 DEBG Invalid block received outcome: NonLinearParentRoots, msg: peer sent invalid block
[ 302657] [ 1512.997277] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.253 DEBG Parent lookup failed error: Peer sent invalid block. Reason: NonLinearParentRoots
[ 302657] [ 1513.223000] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.477 DEBG Processed HTTP API request method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 3.677785ms
[ 302657] [ 1513.233050] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.487 DEBG Processed HTTP API request method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 6.669223ms
[ 302657] [ 1513.233638] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.489 DEBG Processed HTTP API request method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 6.672919ms
[ 302657] [ 1513.240784] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.494 DEBG Processed HTTP API request method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 7.442235ms
[ 302657] [ 1513.251966] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.500 DEBG Processed HTTP API request method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 522.494µs
[ 302657] [ 1513.254084] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.508 DEBG Processed HTTP API request method: GET, path: /eth/v1/node/version, status: 200 OK, elapsed: 5.504131ms
[ 302657] [ 1513.593619] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.847 DEBG Execution engine online service: exec
[ 302657] [ 1513.593695] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.850 DEBG Execution engine upcheck complete state: Synced, service: exec
[ 302657] [ 1513.620271] [system;capturefs;files;eth/service_lighthouse-geth-0--bn] [I] Oct 20 20:25:52.868 DEBG Parent chain processed result: NonFaultyFailure, chain_hash: 0xd81a…0305, service: sync
```
[Antithesis Report](https://ethereum.antithesis.com/report/OLvW8-4Tkimpm4b8SxguiFcnFX-WVntr/rQgmHuhoPrFXoL1g_ww7IF60pJqa0KjaKQvxyABuDIA.html?auth=v2.public.eyJuYmYiOiIyMDIzLTEwLTMxVDExOjA1OjAwWiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJyUWdtSHVob1ByRlhvTDFnX3d3N0lGNjBwSnFhMEtqYUtRdnh5QUJ1RElBLmh0bWwiLCJyZXBvcnRfaWQiOiJPTHZXOC00VGtpbXBtNGI4U3hndWlGY25GWC1XVm50ciJ9fX1pyow22QUJmqTYKKIu1gBNLsRbWGaOWdJaQLr7YZ6WmxbJif_YBSTy0WneonGa7AnnEJl1NRzE_Ji5Un0L3BYE)
3. Beacon node (Lighthouse paird with geth)
```l!
source_service = "system;capturefs;files;eth/service_lighthouse-geth-0--bn "
state_id = " 201883 "
error_message = ` Oct 20 20:24:53.396 DEBG RPC Error direction: Outgoing, score: inf, peer_id: 16Uiu2HAmGsTzCmnVTNJ2maPs1pj4jpLub1xdeMPQrMH5PucG4qLj, client: Nimbus: version: unknown, os_version: unknown, err: Peer does not support the protocol, protocol: beacon_blocks_by_root, service: libp2p`
message_count = 3
```
[Antithesis Report](https://ethereum.antithesis.com/report/Pzv_B-MjVMi-OmSYv35Le0RKWSSHf4-o/SVefQUf_HxnufagipqEvSEgRNbeFK8lnw33jr0tMZ30.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiU1ZlZlFVZl9IeG51ZmFnaXBxRXZTRWdSTmJlRks4bG53MzNqcjB0TVozMC5odG1sIiwicmVwb3J0X2lkIjoiUHp2X0ItTWpWTWktT21TWXYzNUxlMFJLV1NTSGY0LW8ifX0sIm5iZiI6IjIwMjMtMTAtMzFUMTE6MTc6NTVaIn3E8kRRSnB7w3kD9Vbj1qocf4Rjsdq8nkZdUTj8nXZh3Ln_PNH5CXz2mc66Pgok2alPIWcyElvAT7SPdujZtcYK)
4. Beacon node (Lighthouse paird with nethermind)
```l!
source_service = "system;capturefs;files;eth/service_lighthouse-nethermind-0--bn "
state_id = " 1144875 "
error_message = ` Oct 20 20:28:29.091 WARN Execution engine call failed error: ServerMessage { code: -38001, message: "unknown payload" }, service: exec`
message_count = 2
```
Related logs:
```
[ 1144875] [ 1669.400419] [system;capturefs;files;eth/service_lighthouse-nethermind-0--nethermind] [I] 20 Oct 20:28:28 | Block production for payload with id=f5abb7711b5009a6 failed - unknown payload.
[ 1144875] [ 1669.401896] [system;capturefs;files;eth/service_lighthouse-nethermind-0--nethermind] [I] 20 Oct 20:28:28 | Error when handling ID 1, engine_getPayloadV3(0xf5abb7711b5009a6) | {"jsonrpc":"2.0","error":{"code":-38001,"message":"unknown payload"},"id":1}
[ 1144875] [ 1669.837317] [system;capturefs;files;eth/service_lighthouse-nethermind-0--bn] [I] Oct 20 20:28:29.091 WARN Execution engine call failed error: ServerMessage { code: -38001, message: "unknown payload" }, service: exec
```
[Antithesis Report](https://ethereum.antithesis.com/report/HVw99cQ9_pVXz9bjbiPkGTumHl8aAsgM/CHklWL8Ca1oEBU1b9mSbNlqnirb2bZEls4-YYxgPiwc.html?auth=v2.public.eyJuYmYiOiIyMDIzLTEwLTMxVDExOjQxOjM0WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJDSGtsV0w4Q2Exb0VCVTFiOW1TYk5scW5pcmIyYlpFbHM0LVlZeGdQaXdjLmh0bWwiLCJyZXBvcnRfaWQiOiJIVnc5OWNROV9wVlh6OWJqYmlQa0dUdW1IbDhhQXNnTSJ9fX3vWGSsy8vU3cqj3KW1fIgQrhPvljlTYh-GGOOl_Q_6H2pKUwChVKWWu3QzMRuuKWHlKDAIWFPFdnAOb_NSiX0F)
### Prysm
1. Beacon node (Prysm paired with geth)
```l!
source_service = "system;capturefs;files;eth/service_prysm-geth-0--prysm-bn "
state_id = " 1292982 "
error_message = ` time="2023-10-20 20:25:13" level=error msg="Could not get forkchoice update argument" error="block does not exist" prefix=blockchain`
message_count = 1
```
Related logs:
```
[ 1292982] [ 1473.798484] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn] [I] time="2023-10-20 20:25:13" level=warning msg="Could not update head" error="could not apply weight changes: context deadline exceeded" prefix=blockchain
[ 1292982] [ 1473.877188] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn] [I] time="2023-10-20 20:25:13" level=warning msg="could not determine node weight" prefix=blockchain root=0x0000000000000000000000000000000000000000000000000000000000000000
[ 1292982] [ 1473.880793] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn] [I] time="2023-10-20 20:25:13" level=debug msg="Head block is not the received block" headRoot=0x0000000000000000000000000000000000000000000000000000000000000000 headWeight=0 prefix=blockchain receivedRoot=0xaae3a85a60ce86283533bd873ea0d8cf90ceb91514c21162f6b7ef0f6e50155b receivedWeight=0
[ 1292982] [ 1473.953665] [system;capturefs;files;eth/service_prysm-geth-0--prysm-bn] [I] time="2023-10-20 20:25:13" level=error msg="Could not get forkchoice update argument" error="block does not exist" prefix=blockchain
```
[Antithesis Report](https://ethereum.antithesis.com/report/-jVolAETwVNFUaGerWWMCpHYOqKpVswQ/eNJdOkc2CFjLqFWk8-5ioWOZpO7fBpk8TrcBVAP-tAA.html?auth=v2.public.eyJuYmYiOiIyMDIzLTEwLTMxVDEyOjEwOjMzWiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJlTkpkT2tjMkNGakxxRldrOC01aW9XT1pwTzdmQnBrOFRyY0JWQVAtdEFBLmh0bWwiLCJyZXBvcnRfaWQiOiItalZvbEFFVHdWTkZVYUdlcldXTUNwSFlPcUtwVnN3USJ9fX2L-YZp9U5JGI-lFAKgQUinWgRsqEN7CVqML-KiUVejseJPewIPbzzZaQxnB3BlceC5UUM4MPwQSGxCjOhihgcN)