# Antithesis Report 28.11.2023
* Error producing empty block, null reference exception in eth_getTransactionReceipt.
* Null reference exception notifying a peer about transactions.
* Payload timestamp must be greater than block timestamp.
* InvalidOperationException in eth_gasPrice.
* Error executing consensus method engine_forkchoiceUpdatedV3.
* Failed to build payload with invalid timestamp for the given parent.
* Debug message about an invalid epoch, warnings about batch processing and inconsistent votes.
* Warning about the inability to process attestation for fork choice.
* Error updating deposit contract cache.
* Warning about execution engine call failure with a 400 Bad Request status.
* Warnings about fork and execution engine call failure.
* Warnings and debug messages about API request errors and peer disconnection.
* Internal error while importing a block.
## Nethermind
Nethermind (Prysm)
```l!
08 Nov 14:54:47 | Error during method execution, request: ID 472, engine_forkchoiceUpdatedV3({
"headBlockHash": "0x4864d6d28b4b4aa44f0a30b76ee8d1c89430330bfde00a21421cf53040c7ca7e",
"safeBlockHash": "0x70b2efccdb84dfbe920e83bae2c1c91cad4b2d16c837a01b95ac59911845a802",
"finalizedBlockHash": "0x7bb7e398a2e65606c8f6685296397249375af806a04ed66a58cb6a13c06deebc"
},{
"timestamp": "0x654ba13c",
"prevRandao": "0xf9083e83b007cbb6ffa53e8c872eec2a83042f7890892cbc197ef7d628e21181",
"suggestedFeeRecipient": "0x00000000219ab540356cbb839cbe05303d7705fa",
"withdrawals": [],
"parentBeaconBlockRoot": "0x449fec86c2f414a7594aaa89bf5013638274c02a871a6af2107c8beff1947983"
}) Nethermind.Merge.Plugin.BlockProduction.EmptyBlockProductionException: Couldn't produce empty block: Setting state for processing block failed
```
[Antithesis Report](https://ethereum.antithesis.com/report/DvX_PNXmQV4zsm253o7KoiB3/aiMfDxBejLO11mUauqRep-gMHQYFHNjxy-rryr7qbgU.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiYWlNZkR4QmVqTE8xMW1VYXVxUmVwLWdNSFFZRkhOanh5LXJyeXI3cWJnVS5odG1sIiwicmVwb3J0X2lkIjoiRHZYX1BOWG1RVjR6c20yNTNvN0tvaUIzIn19LCJuYmYiOiIyMDIzLTExLTIxVDEwOjUzOjU3WiJ9qjPsXvZIuM4DMcO47zPJbtCS1KI0HjgP9C8z_z--j6Jw5j6n8sYNqH0zpxkgxANDgBIYCjjZh5v4HKUhT98ADw)
Nethermind (Prysm)
```l!
08 Nov 14:56:01 | Error during method execution, request: ID 82, eth_getTransactionReceipt(0x314746ccfeaa3938b3a285de0a3eba6abc6f664623a111546b67460c9306eb42) System.NullReferenceException: Object reference not set to an instance of an object.
```
[Antithesis Report](https://ethereum.antithesis.com/report/4NudPhQRSz8Z46jbnMAXeK47/UroVVAAqWRc7usakhCIu5M24YPlKhVOIUiKEdNlE3NI.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiVXJvVlZBQXFXUmM3dXNha2hDSXU1TTI0WVBsS2hWT0lVaUtFZE5sRTNOSS5odG1sIiwicmVwb3J0X2lkIjoiNE51ZFBoUVJTejhaNDZqYm5NQVhlSzQ3In19LCJuYmYiOiIyMDIzLTExLTIxVDExOjAwOjU4WiJ9D1KBTWAUBtS_yvL35Q4JLSS2RJKW-DR4R9V7IC6Eqzyf1pNejWk8ZfHtX6skbjOwSMSHSoRQJq7Z7nZhTctXBg)
Nethermind (nimbus)
```l!
17 Nov 12:30:36 | Failed to notify [Peer|eth68| 0| 10.0.20.50:56896| In] about transactions. System.NullReferenceException: Object reference not set to an instance of an object.
```
[Antithesis Report](https://ethereum.antithesis.com/report/yYfWwnihPDRMR5pvmFDB1N_o/HhYw6Fmnc-bIt6pW-sey9hskAXGSkKghaJMOvyYLpHo.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiSGhZdzZGbW5jLWJJdDZwVy1zZXk5aHNrQVhHU2tLZ2hhSk1PdnlZTHBIby5odG1sIiwicmVwb3J0X2lkIjoieVlmV3duaWhQRFJNUjVwdm1GREIxTl9vIn19LCJuYmYiOiIyMDIzLTExLTI4VDE1OjE5OjEwWiJ9buo7p6J8-KuHab0Hj9iiXQYGr4AVapPuyoNTZ8foO1AcJ749JFSCVu7Afusd98mPR-YLo2vwx9VjTcLtGrTPBg)
Nethermind (prysm)
```l!
}) | {"jsonrpc":"2.0","error":{"code":-38003,"message":"Payload timestamp 1700225374 must be greater than block timestamp 1700225374."},"id":3924}
```
[Antithesis Report](https://ethereum.antithesis.com/report/51ppliA6a8UDXCxsW6r2Vvyk/O0Y0I-6C_dkQ0qbaZUe9h6k1KlgVid2mWuaEAO7vbPM.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiTzBZMEktNkNfZGtRMHFiYVpVZTloNmsxS2xnVmlkMm1XdWFFQU83dmJQTS5odG1sIiwicmVwb3J0X2lkIjoiNTFwcGxpQTZhOFVEWEN4c1c2cjJWdnlrIn19LCJuYmYiOiIyMDIzLTExLTI4VDE2OjQ2OjE5WiJ9WKMN4t7Y921AvA8as-_z1niemvZvIa4cyvHx2wJVpY-9lVlRoBdDGs7YOVaaIQVe4zIaJJzsC3Kh-OABxg-LCg)
Nethermind (lighthouse)
```l!
[ 1732.779941] [system;capturefs;files;/service_lighthouse-nethermind-0--nethermind] [I] 17 Nov 13:50:15 | Error during method execution, request: ID 23, eth_gasPrice() System.InvalidOperationException: Nullable object must have a value.
[ 1732.789] [system;capturefs;files;/service_lighthouse-nethermind-0--nethermind] [inf] at System.Nullable`1.get_Value()
[ 1732.804] [system;capturefs;files;/service_lighthouse-nethermind-0--nethermind] [inf] at Nethermind.JsonRpc.Modules.Eth.GasPrice.GasPriceOracle.GetGasPriceEstimate()
[ 1732.820] [system;capturefs;files;/service_lighthouse-nethermind-0--nethermind] [inf] at Nethermind.JsonRpc.Modules.Eth.EthRpcModule.eth_gasPrice()
[ 1732.832] [system;capturefs;files;/service_lighthouse-nethermind-0--nethermind] [inf] at InvokeStub_IEthRpcModule.eth_gasPrice(Object, Object, IntPtr*)
[ 1732.838] [system;capturefs;files;/service_lighthouse-nethermind-0--nethermind] [inf] at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
[ 1732.901] [system;capturefs;files;/service_lighthouse-nethermind-0--bn] [inf] Nov 17 13:50:15.544 DEBG ENR written to disk service: libp2p
[ 1732.904] [system;capturefs;files;/service_lighthouse-nethermind-0--nethermind] [inf] 17 Nov 13:50:15 | Error when handling ID 23, eth_gasPrice() | {"jsonrpc":"2.0","error":{"code":-32603,"message":"Internal error","data":"System.InvalidOperationException: Nullable object must have a value.\n at System.Nullable`1.get_Value()\n at Nethermind.JsonRpc.Modules.Eth.GasPrice.GasPriceOracle.GetGasPriceEstimate()\n at Nethermind.JsonRpc.Modules.Eth.EthRpcModule.eth_gasPrice()\n at InvokeStub_IEthRpcModule.eth_gasPrice(Object, Object, IntPtr*)\n at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)"},"id":23}
```
[Antithesis Report](https://ethereum.antithesis.com/report/QzTFuOoKewzSK2vQ_9d6yt8T/4LsaCJS37ZIGxF3Q9d1BKB5wvj7wNyCpwewF7bCnG64.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTI4VDE2OjU0OjA2WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiI0THNhQ0pTMzdaSUd4RjNROWQxQktCNXd2ajd3TnlDcHdld0Y3YkNuRzY0Lmh0bWwiLCJyZXBvcnRfaWQiOiJRelRGdU9vS2V3elNLMnZRXzlkNnl0OFQifX19CQUzidj3olfuEuWReoZqsx5p0kCBhA3wBCkvjm_Kc1YNIVjmUO8iwxC_1pGJN27uIJyMF4E74_N2_TV_TT_dCw)
## Besu
Besu (Lighthouse)
```l!
2023-11-08 15:02:30.874+00:00 | vert.x-eventloop-thread-1 | ERROR | ExecutionEngineJsonRpcMethod | failed to exec consensus method engine_forkchoiceUpdatedV3, error: null
```
[Antithesis Report](https://ethereum.antithesis.com/report/JUScc-p0iMrH0AKT27mSjIHJ/Hib6COyjX61DNVqO2gw83sf8rD0hSz9CmlTN_YL5Guc.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTIxVDEyOjUwOjU1WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJIaWI2Q095alg2MUROVnFPMmd3ODNzZjhyRDBoU3o5Q21sVE5fWUw1R3VjLmh0bWwiLCJyZXBvcnRfaWQiOiJKVVNjYy1wMGlNckgwQUtUMjdtU2pJSEoifX197GI3VQ7k6pOtbg_hyyefjn89OT40pUKDmWAHjuy2axSMz7n_F_oMwB__wx7SbjpKhvi6vzhMRH0LMZ1Mu6BlBQ )
## Geth
Geth (Teku)
```l!
ERROR[11-08|14:58:29.347] Failed to build payload err="invalid timestamp, parent 1699455508 given 1699455496"
```
[Antithesis Report](https://ethereum.antithesis.com/report/O71Qjm-opBO3PFBCdkBBruCs/vy817JIi8ZxKMfBVAjtHxGVKUiIZUbTfB0GJCR_vMoM.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTIxVDE1OjIxOjM5WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJ2eTgxN0pJaThaeEtNZkJWQWp0SHhHVktVaUlaVWJUZkIwR0pDUl92TW9NLmh0bWwiLCJyZXBvcnRfaWQiOiJPNzFRam0tb3BCTzNQRkJDZGtCQnJ1Q3MifX19dhMh-jzRE0dilRwNdjx75aN0dwWNTIE9ImFo4AIdbWL6E7iJRLl1bneET_RGqnGXufao30SUGmRz-2q9dXhoDw)
## Prysm-bn
Prysm-bn
```l!
time="2023-11-08 15:03:27" level=debug msg="Invalid status message from peer" error="invalid epoch" handler=status peer=16Uiu2HAmHtj8uzmeNGAq7yVvfd1Lpwuy9JwBexAydPHvF4ukTBCW prefix=sync
```
[Antithesis Report](https://ethereum.antithesis.com/report/IW5M_j_TVQynGt9Qp4P3GTMh/cfgAhmyB5d8zf-wjWSd_-_M_Z8xSAP6jRLrXv225_mY.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTI4VDE3OjEwOjA1WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJjZmdBaG15QjVkOHpmLXdqV1NkXy1fTV9aOHhTQVA2alJMclh2MjI1X21ZLmh0bWwiLCJyZXBvcnRfaWQiOiJJVzVNX2pfVFZReW5HdDlRcDRQM0dUTWgifX198NbtG4YlTXFNqddcoU2UXyOqY_9NqybMnybdlbrgEQS_M7PMIctXiFqqdguMEU_h4OAu_fBv0z_IFxEEAKsTDg)
Prysm-bn
```l!
time="2023-11-08 15:16:58" level=warning msg="Skip processing batched blocks" error="could not process block in batch: could not insert last block in batch to forkchoice: could not update justified balances: could not get justified balances: could not get ancestor state: signed beacon block can't be nil" prefix=initial-sync
```
[Antithesis Report](https://ethereum.antithesis.com/report/71So9mXBXrXYsfi20B41YbGS/fmVy0dwvjjOuEv-ETfGNA-kdnaH-ztUw5312k0o6lHc.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiZm1WeTBkd3Zqak91RXYtRVRmR05BLWtkbmFILXp0VXc1MzEyazBvNmxIYy5odG1sIiwicmVwb3J0X2lkIjoiNzFTbzltWEJYclhZc2ZpMjBCNDFZYkdTIn19LCJuYmYiOiIyMDIzLTExLTI4VDE3OjIxOjM5WiJ9S3eVj_SO6-3L6WONVOE9RNmZq46OluFx_WYcsQE-2WW0yoSGdrxAsrQIOv30V8ikTg3eBNoRdbuhNEzcvw4kBQ)
Prysm-bn
```l!
FFG and LMD votes are not consistent, block root: 0x5cb4036799820e6c4c9044ba4b48ed5011307f9b85c5b3e73e3c0d85b5d1019a, target root: 0x5cb4036799820e6c4c9044ba4b48ed5011307f9b85c5b3e73e3c0d85b5d1019a, canonical target root: 0x9c86f967f151b9b43bb01db48240e450bb82ac0207b7eea75fec18f46a86bb77" gossip score=32.72 multiaddress="/ip4/10.0.20.110/tcp/4000" peer id=16Uiu2HAm8LKsFhZtMMVtERALpd558pGiFrim1FoQhsLoAqx1nm9X prefix=sync topic="/eth2/2d46aec5/beacon_attestation_1/ssz_snappy"
```
[Antithesis Report](https://ethereum.antithesis.com/report/noxYj8zmYCveSlOZmxxdWQnw/AbdDbwlVBn_VQlMdiFVK7nQrK22pKt2CKD3Of9BOVlI.html?auth=v2.public.eyJuYmYiOiIyMDIzLTEyLTA3VDE2OjMzOjA3WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJBYmREYndsVkJuX1ZRbE1kaUZWSzduUXJLMjJwS3QyQ0tEM09mOUJPVmxJLmh0bWwiLCJyZXBvcnRfaWQiOiJub3hZajh6bVlDdmVTbE9abXh4ZFdRbncifX19ucezhbjw1_ECf53PvsJ5QDuYbHJGvca38_A1cTjRZg-kort9UiMnSKE4Nh0Wy--SLg9NIwecoVf83Y5eoXw4Cg)
Prysm-bn (geth)
```l!
time="2023-11-17 12:45:22" level=error msg="invalid prev root 0x6c14e59370fd2c42e33aae164851f9552cd6fd9dc5ce2577fcc94af79169f660" error="invalid proposer boost root" prefix=forkchoice-doublylinkedtree
```
[Antithesis Report](https://ethereum.antithesis.com/report/Zrk92WkxwLzA_EGjFP0MdWBy/Zkd8ZwLbRM8Qfo4A7d77qnyyG6fTv1LlI9nxm32XND4.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiWmtkOFp3TGJSTThRZm80QTdkNzdxbnl5RzZmVHYxTGxJOW54bTMyWE5ENC5odG1sIiwicmVwb3J0X2lkIjoiWnJrOTJXa3h3THpBX0VHakZQME1kV0J5In19LCJuYmYiOiIyMDIzLTExLTI4VDE1OjI3OjE4WiJ97gU1nbvryI8fp3a8baeZXn8MJrDXu9WkmZ2zWJoWgOi25y4Tl6BgAMjgr6E1m1LlZ5CskGWhKQhwSpAKQdhnAA)
Prysm-bn (geth)
```l!
time="2023-11-17 12:51:44" level=warning msg="Could not process attestation for fork choice" aggregationCount=2 beaconBlockRoot=0x824eea59b1ea committeeIndex=0 error="could not process attestation: target epoch 6 does not match current epoch 8 or prev epoch 7" prefix=blockchain slot=223 targetRoot=0x1d6090a39f85
```
[Antithesis Report](https://ethereum.antithesis.com/report/eOhwD4rl6zE9mvJICnNPWvv8/nDCMG8WXTV-IfoCJ2f5BYzyfs0YhD3ZOlpbXxJkxkNc.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoibkRDTUc4V1hUVi1JZm9DSjJmNUJZenlmczBZaEQzWk9scGJYeEpreGtOYy5odG1sIiwicmVwb3J0X2lkIjoiZU9od0Q0cmw2ekU5bXZKSUNuTlBXdnY4In19LCJuYmYiOiIyMDIzLTExLTI4VDE1OjMyOjQwWiJ9fwYRdLuD6u6x1ekhT0eZrJ-mGoHvV777ieDzMgBM-iE0g9IHEfHkywoCPbA1kn2EZ_6nUZ3c51hUhWh0VcIiDA)
## Lighthouse
Lighthouse (Nethermind)
```l!
Nov 08 14:27:40.350 ERRO Error updating deposit contract cache error: Failed to update deposit cache: GetDepositLogsFailed(\"eth_getLogs call failed ServerMessage { code: -32001, message: \\\"1 could not be found\\\" }\"), retry_millis: 60000, service: deposit_contract_rpc
```
[Antithesis Report](https://ethereum.antithesis.com/report/BF6sCwrBQqPxIzlQvq2t6-cd/szo14Z0laHEfYNur-sCsosE-zvhKQ45LjYvcAEatMuk.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0Ijoic3pvMTRaMGxhSEVmWU51ci1zQ3Nvc0UtenZoS1E0NUxqWXZjQUVhdE11ay5odG1sIiwicmVwb3J0X2lkIjoiQkY2c0N3ckJRcVB4SXpsUXZxMnQ2LWNkIn19LCJuYmYiOiIyMDIzLTExLTI3VDA5OjM1OjM4WiJ9dB28pDVZ7ldngCmWcn4-t-Mob8adVeG6B0pOJOg87gXnn2B66s4MGrmS1EAOpKK7vBOfTvERxMJZRTgIR1rwCw)
Lighthouse (Besu)
```l!
Nov 08 14:50:43.329 WARN Execution engine call failed error: HttpClient(url: http://127.0.0.1:8551/, kind: status, status_code: 400 Bad Request, source: unknown), service: exec localhost-8551 . kind- status. status_code- 400 bad request. source- unknown), service=untracked,
```
[Antithesis Report](https://ethereum.antithesis.com/report/sYgN_dwzXap_PY-dI1UQYQIY/KQ3Sq_E8I7PKKQW3mCN7PJrkrzNPDXVA2DZvuwBKLZ4.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTI4VDE0OjQ0OjIxWiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJLUTNTcV9FOEk3UEtLUVczbUNON1BKcmtyek5QRFhWQTJEWnZ1d0JLTFo0Lmh0bWwiLCJyZXBvcnRfaWQiOiJzWWdOX2R3elhhcF9QWS1kSTFVUVlRSVkifX19RzVewLrKTp9xnXJuR7It41AsnSKO2VjE_9AXMnZ7uQJt1-tzhjPvxJOQDW8LIkjhImoSiAP6ohKdvqDQvJV1AQ)
Lighthouse (Nethermind)
```l!
Nov 12 16:54:05.707 WARN Unable to prepare proposer with lookahead, slot: 145, error: ExecutionForkChoiceUpdateFailed(EngineError(Api { error: ServerMessage { code: -32603, message: \"Internal error\" } })), service: state_advance
```
[Antithesis Report](https://ethereum.antithesis.com/report/Y-GZLHgqL_OhCSBEne_iQWGl/SNN8AUk26Hke050TYjYxZCcaZqcIzsNiNIPA50Lh27M.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTI4VDE0OjU0OjIzWiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJTTk44QVVrMjZIa2UwNTBUWWpZeFpDY2FacWNJenNOaU5JUEE1MExoMjdNLmh0bWwiLCJyZXBvcnRfaWQiOiJZLUdaTEhncUxfT2hDU0JFbmVfaVFXR2wifX19Q6vuc95WWqiBJYpyUgvlAhJhroa8kfmXaFz6Nc9ZIz9iWMrOaMqyvOSlDJ84KC1v6w21o3g7bcc88UyKB3K4Dw )
Lighthouse (nethermind)
```l!
Nov 12 16:31:28.969 ERRO Fork subscription scheduled but no fork scheduled, service: network
```
[Antithesis Report](https://ethereum.antithesis.com/report/KlFTEYcsb8kpcuaC0TnfA-Uo/N-PxArE7A7Na2UEutOU7tnv2wgzyfAMbF78y1hDviPo.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTI4VDE0OjI4OjE4WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJOLVB4QXJFN0E3TmEyVUV1dE9VN3RudjJ3Z3p5ZkFNYkY3OHkxaER2aVBvLmh0bWwiLCJyZXBvcnRfaWQiOiJLbEZURVljc2I4a3BjdWFDMFRuZkEtVW8ifX19xPIAkptQDX9KaBfGtF0qWfQRX6zy07Z-Dksuk_fryje1QZt4q5RnAfMvlwF6cNMSRLiokGCK0EllXf-VFOgEBA )
Lighthouse (nethermind)
```l!
Nov 12 16:56:37.546 WARN Execution engine call failed error: ServerMessage { code: -38001, message: \"unknown payload\" }, service: exec
```
[Antithesis Report](https://ethereum.antithesis.com/report/fhMzOUfSPPYFCZXzgIVe4R5t/4cfMNHzOKFb9_e8jGCL7kCEOSV4lnqMMP7kC8E-qte0.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiNGNmTU5Iek9LRmI5X2U4akdDTDdrQ0VPU1Y0bG5xTU1QN2tDOEUtcXRlMC5odG1sIiwicmVwb3J0X2lkIjoiZmhNek9VZlNQUFlGQ1pYemdJVmU0UjV0In19LCJuYmYiOiIyMDIzLTExLTI4VDE0OjExOjUzWiJ925tBUsr3ElEvLxnnlPaIXfqpBSPjaC265HEwQD1yO9IaMop091DsnG5SuVoua1wJjoKCTllJvPIbV9Pz53RcBw )
Lighthouse (geth)
```l!
Nov 12 16:57:17.534 WARN Error processing HTTP API request method: POST, path: /eth/v1/validator/contribution_and_proofs, status: 400 Bad Request, elapsed: 154.482186ms
```
[Antithesis Report](https://ethereum.antithesis.com/report/pDHdk6VblDBfKnp6u2Ncb9jC/Q8IzITZfsmi5wTru9TSazv-oqO7ZKumuWeC-WtOVfGg.html?auth=v2.public.eyJuYmYiOiIyMDIzLTExLTI4VDE0OjAxOjU1WiIsInNjb3BlIjp7IlJlcG9ydFNjb3BlVjEiOnsiYXNzZXQiOiJROEl6SVRaZnNtaTV3VHJ1OVRTYXp2LW9xTzdaS3VtdVdlQy1XdE9WZkdnLmh0bWwiLCJyZXBvcnRfaWQiOiJwREhkazZWYmxEQmZLbnA2dTJOY2I5akMifX19dy2BCf8t78IBmTSE9LVfIglsnWwm7Ozy334p1dm2UEdhXLsfSBQQUAwc55QJ2mHnm0IEMNyu3bC-xQQKxEd2CA )
Lighthouse (geth)
```l!
Nov 12 16:55:07.977 DEBG Marking peer disconnected in DHT error: Transport error: Protocol negotiation failed., peer_id: 16Uiu2HAm8RY6xR6cW2g2ccvsctN1GMjJZxptYzngd37Tefgewqzp, service: libp2pservice=untracked,
```
[Antithesis Report](https://ethereum.antithesis.com/report/o2xwxxJmW7S-GGG2D19p3w1-/Ki69XZKvU_0sIwMDupC9Hd1XCeT7eAKSzFtTw6J5DZc.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiS2k2OVhaS3ZVXzBzSXdNRHVwQzlIZDFYQ2VUN2VBS1N6RnRUdzZKNURaYy5odG1sIiwicmVwb3J0X2lkIjoibzJ4d3h4Sm1XN1MtR0dHMkQxOXAzdzEtIn19LCJuYmYiOiIyMDIzLTExLTI4VDEzOjQwOjEwWiJ9sQ-UalAf5zUspyi8x4qtk2GBji2SszXNRvG1DG3cSPMR91WKdsIUXiLX215Srl7mCyVPKRrLWnd-pDOv_4Z_Aw )
## Lodestar
Lodestar (geth)
```l!
Nov 12 17:01:30.372 DEBG Error response to dropped lookup error: ErrorResponse(ServerError, \"Inconsistent state, blobSidecar blockRoot=0xc6c20693c03143197d593f56c2a256d6e61418a9cccbe72656ddc8c5e02f1898 index=0 blobSidecarBytes=0 expected=131256\"), service: sync",source:"system;capturefs;files;/service_lighthouse-geth-0--bn
```
[Antithesis Report](https://ethereum.antithesis.com/report/vS1cuXav0QPDBi-R-1ZrpomP/ER7jzZmvSuQTjwOS-Nw5Hm0bTMy10Cq8GRWtb3kX9Ro.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiRVI3anpabXZTdVFUandPUy1OdzVIbTBiVE15MTBDcThHUld0YjNrWDlSby5odG1sIiwicmVwb3J0X2lkIjoidlMxY3VYYXYwUVBEQmktUi0xWnJwb21QIn19LCJuYmYiOiIyMDIzLTExLTI4VDE0OjI2OjEyWiJ9AxhAZmfwWVCs7sYiEKuUHK9T7nBrOoM4uxERt2Cb7UnZcwfQwlywm4vdeSjKYQbCcwMxTxHdUFRsWnToVqBhCA )
## Teku
Teku (geth)
```l!
error_message = `2023-11-17 12:32:36.786 ERROR - Internal error while importing block: c598c44ef8ec629c2c76261bf824717f04c6ae2a384f8bc9c2d5d5ceb580695e (121). Block content: 0x64000000a0209a303904b05af99e5c0ece5a6872d812454557fd787ed6d99c43b4ad8d43739dbaec801a04c32c70785f097c5dd7001859e29a3203eb4bd29a7f19f25341e54ecbd38e5a241ff1789ab3842c856ad10f493def5f639940a0471030d78c697900000000000000280000000000000081ba77b368441be27d63f495eb42242fa22bb70faa3f08f6d4f044df03a542fec28d7a73a465c0cccbf46a554e39bfe55600e482cf4445132c4baa08768b4c7054000000b48b93ede9fb3725cee04ad6a0c1b22f36988fa6b22f909eb54feaba43c5ea52f915e81c91a9379f50ff5ec0a6ca45a000d8e8d3559ee42a2c4f68240f4dac242b0c1560f75fc8a1f3f0f68761d8e7b0e9e66594cfb9d3c4777ac7335e364769d70a234731285c6804c2a4f56711ddb8c82c99740f207854891028af34e27e5e00000000000000003bc9cb34b9eddfa69828aca22b6cefd03c61f58f3631cd480b437d42dfc2c690707279736d2d6e65746865726d696e642d3000000000000000000000000000008801000088010000880100004304000043040000004000000820460708001000000288d100000000000000000000000000000000000000000000000000000000000000000288d101020004008000627480000001a37dd0de0f6e2dc021d8a9316c3ecae1590966a30f7ea715f9dd5f571d0fa4ad038173b71c0c1c1f70e9dacede298ca411514b3cd5836a7f78425735c544928720ada6cd0c1ceb4dc1ed9a729c15a07bdcec2ea379cbc1da9273d716d4c557a243040000970d0000970d00000c000000f1000000d6010000e400000075000000000000000000000000000000729c557b81e72c4fc230ac84cfd6d45a192b3335ce5a1e3f6db2004860e1feaa02000000000000000b195b6c7ad8da0d4e6179aae94e0ca762ee1891ab757654be287d51546312c40300000000000000729c557b81e72c4fc230ac84cfd6d45a192b3335ce5a1e3f6db2004860e1feaab67976c5d143ac058c4a4b41d2f4b143c6a861885ebd36d35d006e139050b566c60a66bcbc3a3f62c069a859c9f68c740eb38eab2d952826df696c1730d7f84debf65c5383efebd2edba85be9b9765f0a9e9fe7017c121517bfc15d046c1337d09e40000007400000000000000000000000000000081ba77b368441be27d63f495eb42242fa22bb70faa3f08f6d4f044df03a542fe02000000000000000b195b6c7ad8da0d4e6179aae94e0ca762ee1891ab757654be287d51546312c40300000000000000729c557b81e72c4fc230ac84cfd6d45a192b3335ce5a1e3f6db2004860e1feaaa79ec2223110615567a2ae6e1fe36f1eb16172b1b97bd1e269e66a7cb0a849311f83ed4bc68905b2f8533fc69a976e6c07d97f9999ac2919280c2b5cf13f15f9b043ccf03a68842fe94d2d7e656089b6275a33931f23cce3ac90dcc4cc55fe7906e40000007300000000000000000000000000000081ba77b368441be27d63f495eb42242fa22bb70faa3f08f6d4f044df03a542fe02000000000000000b195b6c7ad8da0d4e6179aae94e0ca762ee1891ab757654be287d51546312c40300000000000000729c557b81e72c4fc230ac84cfd6d45a192b3335ce5a1e3f6db2004860e1feaab9357e1f1550654518ffdf58e12863f69ee6ad423fc9d077342bf6d4cca8909c5169c3362ecb0c14e798a6d6a518d3120dac1cec0e3448b59c8ecfb15514f8523ef2baac9f0a9c4b89e52ee513ac996c354ea4d4d8ddc0b33b047de40ced500e060a78a95bf0a9304a9d28aa508ba9b12195b7bc95f04dd11caedda5c9ee14d01600000000219ab540356cbb839cbe05303d7705fab970fea9582e98c5cc315eccc7bdc7d097da8c0754021c7905da1e8d109dcfa332d66b78c96b8dde58daf37f057691d942ed95f7f268b6166df32f0bc293de3c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006e9349c214453493f8348daede263bb093772d3109d002ce964ac7d0d8aab2cd4700000000000000176686010000000080200500000000008e5b57650000000010020000aee408000000000000000000000000000000000000000000000000000000000009c28a023711bef3fe6e3dffd8b411174ee8926366b6be6a6096359aa2281fe91a02000054090000000000000000000000000000000000004e65746865726d696e6440000000af0000001e0100008d010000fc0100006b020000da0200004a030000ba0300002a0400009a0400000a0500007a050000ea0500005a060000ca060000f86d81dc8420472286825208946177843db3138ae69679a54b95cf345ed759450d8738d7ea4c68000080820a95a0b787479012550f782c7dfb78b91f7fa101e4e8d3a75ca81574963bd535e577f6a00885c82addb9f844215c56974f0bea40bae791af6b6a0933fb512ecca297aca1f86d81dd842047228682520894687704db07e902e9a8b3754031d168d46e3d586e8738d7ea4c68000080820a95a0f88b01387228a0709d5c5d648fa7e109e90de42ae026e8d58a668706944649daa01cf0aa7d3fe22029779fae27dbce5bbfc318c70f8ae8f0dd48c2b06c96f6d587f86d81de84204722868252089415e6a5a2e131dd5467fa1ff3acd104f45ee5940b8738d7ea4c68000080820a96a08d087f3ca7e13df84f7356e1858d46020dbbeb1fcc811e808fae927c5f21dae0a00e80d55b0fd6f239698b9990070f616c539233eccc6f0d88aaefe5849ab310b3f86d81df84204722868252089480c4c7125967139acaa931ee984a9db4100e0f3b8738d7ea4c68000080820a96a09715a463dc93a0f8ad0cce17a3306efd4825334deb47ca7198ba002e6ca7b74ba039f8e5b76257fba5e8a078c4a1a4d2e8f29b18bc75fb1383653c03ce3f582ebbf86d81e0842047228682520894d08a63244fcd28b0aec5075052cdce31ba04fead8738d7ea4c68000080820a95a0bd9bc8fd82c154a13a60461480613b102c85fc54519ebcfa09371b47b73c5d08a02dd284efdf94ebd4b131982ec78eaae8e7b079db43c43b0dfd2b57f22467adc0f86d81e18420472286825208940b06ef8be65fcda88f2dbae5813480f997ee8e358738d7ea4c68000080820a95a04e3280e09ba7507c862e10ce34b603430d66bb77a58a0280344778541f3a362ea0329ffc60c3b6a80338c00f7bdc3bcceba8cec890f4048a63f71d431a1808508ef86e820190842030fe5f825208946177843db3138ae69679a54b95cf345ed759450d8738d7ea4c68000080820a95a02c64be682bfd0fc9d17bd20a5b85997c64fd341463fd29ebdfbfcf5abb1cd9a2a05dcfd519e672ec28947573012294964ed9c3de8902226f853b3409db5da9b11ff86e
[ 1951.385] [system;capturefs;files;/service_teku-geth-0--teku] [inf] 820191842030fe5f82520894687704db07e902e9a8b3754031d168d46e3d586e8738d7ea4c68000080820a95a07ee2129885b00d95b59df5b36dfbda2a37178ec2303f91ef3dabf0dbd26b1763a010b7e6d4953154c23bbf3b31a1f4e802a36d5ab0b99578cb66c57723aa6d1ab6f86e820192842030fe5f8252089415e6a5a2e131dd5467fa1ff3acd104f45ee5940b8738d7ea4c68000080820a96a09917229e4ced7c22256eec6f2aec17df5b95ef4db75f513db646388e0a006ae4a02dc2797358447d471876e2e53d2fdc91be7bc54ed38963c08a8575eaac7694caf86e820193842030fe5f8252089480c4c7125967139acaa931ee984a9db4100e0f3b8738d7ea4c68000080820a96a0f703d9d8979ce6f51a1a5a45aa2a708c5f14f74265285950437892a2b25d6fc0a0339a4b2572ab6ba8df5e007a5126f867d5bacd9eb63dc0070d84c0f0bf50028cf86e820194842030fe5f82520894d08a63244fcd28b0aec5075052cdce31ba04fead8738d7ea4c68000080820a96a081ae2a0817051d99ad773cb6d77a066c2c90a7e9f23569f4ecbc0fd6eb0da09da05af39509cc4440d767f989926aeb3de1de8daf92e53f0f76c93afbfee2c586c5f86e820195842030fe5f825208940b06ef8be65fcda88f2dbae5813480f997ee8e358738d7ea4c68000080820a96a07dad4451f192c4aed0b0542b3af51cf4e9c8639116eb0fd26e86590fb0df348aa012a3cfbce74d232076d8101483bfa5feee9e2c8f9b62e7edb8100d282ad89647f86e820196842030fe5f825208941cb96c5809da5977f99f69c11ee58bae5711c5f18738d7ea4c68000080820a95a0da1987a8080850c94ac511a9ff5062dd84bb90ec6e8ec18cdb7f006dbce1e0f8a04098088cd80dd4e3a8abf02f2cc0c13cefd4785388c968b3978a7b8940c10122f86e820197842030fe5f825208942aa48ee899410a6d97c01b0bb0eeaf1771cc435b8738d7ea4c68000080820a95a04039f6b43937906b8acfe30cd875dc52ad101b8c0933f4ad901f0450b75509e8a0392834cccc674543fd4cdb80b16a155fc4330afbfb333deba2eaaf0addfedaaaf86e820198842030fe5f8252089407b9d920dd8e8d83dc1125c94fc0b3cdcdf602fb8738d7ea4c68000080820a95a0c7afd0929a6bb97d761fa4b460e89303b744341b4ec4866b2a6343c3012e76bda051f1205e347ea9d933b8ba16b71e5906c112b229fc33e49b33a9b87e24728577f86e820199842030fe5f82520894fcb6e353ad4f79245c7cb704abcffe2f486842418738d7ea4c68000080820a96a058ae03a368c2104a95e540202e9efb2da0d7506a49ac4e0d1b25840462a87894a01a9cf5d3e78e923b4a6a71558b2b557b8b833568ea630cd7b76e5b6cbff86ad5
[ 1951.385] [system;capturefs;files;/service_teku-geth-0--teku] [inf] java.util.concurrent.CompletionException: java.lang.IllegalStateException: Failed to retrieve 4 / 32 blocks building on state at slot 64: 0x7e0c6a07ae0312f0be2b558bbde6174a82486f9e3be864ad6f2fd24e4d2a546c, 0xfa182bf0580af5d4e83157c5ae74485ad2aa9a285b0437d5f4b1ab45784206c9, 0x9e5b664ebeedc57c9702e0fb086ec33ba09589f6d5a01405d5d6f6e4d30f1851, 0x64e2b115391ffa2988777b61e0de9a7a2e5166522dc0a58a9b63e86aacfbd1e7
[ 1951.385] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) ~[?:?]
[ 1951.385] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) ~[?:?]
[ 1951.385] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:649) ~[?:?]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[?:?]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$propagateResult$3(SafeFuture.java:146) ~[teku-infrastructure-async-develop.jar:23.10.0+16-gd418f8dae7]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863) ~[?:?]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841) ~[?:?]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[?:?]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$propagateToAsync$20(SafeFuture.java:323) ~[teku-infrastructure-async-develop.jar:23.10.0+16-gd418f8dae7]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at tech.pegasys.teku.infrastructure.async.SafeFuture.of(SafeFuture.java:80) ~[teku-infrastructure-async-develop.jar:23.10.0+16-gd418f8dae7]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at tech.pegasys.teku.infrastructure.async.AsyncRunner.lambda$runAsync$2(AsyncRunner.java:47) ~[teku-infrastructure-async-develop.jar:23.10.0+16-gd418f8dae7]
[ 1951.393] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at tech.pegasys.teku.infrastructure.async.SafeFuture.of(SafeFuture.java:72) ~[teku-infrastructure-async-develop.jar:23.10.0+16-gd418f8dae7]
[ 1951.393] [system;capturefs;files;/service_nimbus-nethermind-0--nimbus] [inf] DBG 2023-11-17 12:32:36.815+00:00 Current syncing state topics="syncman" workers_map=ssssssssss sleeping_workers_count=10 waiting_workers_count=0 pending_workers_count=0 wall_head_slot=160 local_head_slot=159 pause_time= avg_sync_speed=0.0 ins_sync_speed=0.0 sync_ident=main direction=forward
[ 1951.398] [system;capturefs;files;/service_nimbus-nethermind-0--nimbus] [inf] DBG 2023-11-17 12:32:36.819+00:00 Synchronization loop sleeping topics="syncman" wall_head_slot=160 local_head_slot=159 difference=1 max_head_age=32 sync_ident=main direction=forward
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at tech.pegasys.teku.infrastructure.async.ScheduledExecutorAsyncRunner.lambda$createRunnableForAction$1(ScheduledExecutorAsyncRunner.java:124) ~[teku-infrastructure-async-develop.jar:23.10.0+16-gd418f8dae7]
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.lang.Thread.run(Thread.java:833) [?:?]
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] Caused by: java.lang.IllegalStateException: Failed to retrieve 4 / 32 blocks building on state at slot 64: 0x7e0c6a07ae0312f0be2b558bbde6174a82486f9e3be864ad6f2fd24e4d2a546c, 0xfa182bf0580af5d4e83157c5ae74485ad2aa9a285b0437d5f4b1ab45784206c9, 0x9e5b664ebeedc57c9702e0fb086ec33ba09589f6d5a01405d5d6f6e4d30f1851, 0x64e2b115391ffa2988777b61e0de9a7a2e5166522dc0a58a9b63e86aacfbd1e7
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at tech.pegasys.teku.dataproviders.generators.AsyncChainStateGenerator.lambda$processBlockBatch$8(AsyncChainStateGenerator.java:165) ~[teku-ethereum-dataproviders-develop.jar:23.10.0+16-gd418f8dae7]
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646) ~[?:?]
[ 1951.398] [system;capturefs;files;/service_teku-geth-0--teku] [inf] ... 15 more
```
[Antithesis Report](https://ethereum.antithesis.com/report/oRZfOluRdHCiJjcLnR36fL8Z/KFl6R4tZ0usVBfs51Kk3-KFDvlkC8uPPu0u-ofWF1Sk.html?auth=v2.public.eyJzY29wZSI6eyJSZXBvcnRTY29wZVYxIjp7ImFzc2V0IjoiS0ZsNlI0dFowdXNWQmZzNTFLazMtS0ZEdmxrQzh1UFB1MHUtb2ZXRjFTay5odG1sIiwicmVwb3J0X2lkIjoib1JaZk9sdVJkSENpSmpjTG5SMzZmTDhaIn19LCJuYmYiOiIyMDIzLTExLTI4VDEzOjI1OjIwWiJ9gsedyoWNTpVrQaM7yzjt7RO9MiXNBEoTfzC8YT_DTQ7-Af99MT7Ry_H7PDdzAa9cShLL88DADBUzK4Lz4zkhBw )
Sign in with Wallet
Connect another wallet