---
# System prepended metadata

title: 'Teku attestation subnets revamp '

---

Observed improvements (approximations) summary:  
- -36% transmission bandwidth
- -32% reception bandwidth
- -17% VC+EL CPU usage
- -15% VC CPU usage
- +40% Non validating node gossip participation
- +45% Non validating node transmission bandwidth usage (no effect on the reception bandwidth)  

This note summarizes the performance improvements observed after implementing the [Attnet revamp change](https://github.com/ethereum/consensus-specs/pull/3312).  
The new behavior is described in the spec section [Attestation subnet subscription](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#attestation-subnet-subscription).  

The main change is to subscribe to 2 ([SUBNETS_PER_NODE]( https://github.com/ethereum/consensus-specs/blob/726b453332ab8dc3a2f2508b8d19e91d0e28a870/specs/phase0/p2p-interface.md?plain=1#L203)) persistent subnets for 256 epochs ([EPOCHS_PER_SUBNET_SUBSCRIPTION](https://github.com/ethereum/consensus-specs/blob/726b453332ab8dc3a2f2508b8d19e91d0e28a870/specs/phase0/p2p-interface.md?plain=1#L194)) based on the node id regardless of the number of running validators (which we used to do).  

The new change has been tested on the Goerli network with nodes running 5000 validators (Teku-Besu and Teku-Geth) and an archive node not running any validator (Teku-Besu).  

## Subnet subscriptions effects  

After releasing the change to a 5000 validators Teku-Besu node we can see that it's now connected to 2 persistent subnets plus other subnets but for aggregation duties (once the aggregation duty is fulfilled, the node un-subscribes from the correspondant subnet).
This behaviour is expected since in addition to the persistent subnets, the validators could be assigned attestation aggregation duties and hence should subscribe to the correspondant subnets. In this case, there are 5000 validators running in the node so we end up connected to all subnets.  
Figure 1 shows:
1. Aggregation subnet subscriptions: 62 total
2. All subnet subscriptions: 62 for aggregation and 2 persistent
3. Persistent subnet subscription: 2 subnets (18 and 19)

![](https://hackmd.io/_uploads/r1y28hd3h.png)
*Figure 1 - Attestation subnets subscriptions of a 5000 validators node*  

For a non validating node, the change makes it subscribe to 2 persistent subnets only (No aggregation subnets since no validators)  
Figure 2 shows:
1. No aggregation subnet subscriptions
2. All subnet subscriptions: 2 persistent subnets only
3. Persistent subnet subscription: 2 subnets (62 and 63)

![](https://hackmd.io/_uploads/SkbOjg933.png)
*Figure 2 - Attestation subnets subscriptions of an archive non validating node*  


---

## Network effects  
One of the expected improvements of the attnet revamp is the network bandwidth reduction since the node will subscribe to less persistent attestation subnets and hence will participate less in their gossiping.
We noticed approximately 36% drop in the transmission bandwidth and approximately 32% drop in the reception.  
The graphs in Figure 4 and 5 compare the transmission/reception bandwidth between a regular node and a node running the change.
The graph in Figure 6 shows the change deployment effect on the network bandwidth usage.  

![](https://hackmd.io/_uploads/BkzNAJSon.png)
*Figure 4 - Transmission bandwidth usage*  
Green line: Change deployed  
Yellow line: Change not deployed

---

![](https://hackmd.io/_uploads/ryBr0kBoh.png)
*Figure 5 - Reception bandwidth usage*  
Bleu line: Change deployed  
Orange line: Change not deployed

---

![](https://hackmd.io/_uploads/ByA8-bSo2.png)
*Figure 6 - Network traffic drop after deploying the change*  
Green line: Transmission bandwidth  
Yellow line: Reception bandwidth

---

## CPU Usage effects  
Subscribing to less subnets means processing less attestations and hence less workload which explain the drop in CPU usage we noticed after the change.  
The graph in Figure 7 shows the total CPU usage (CL+EL, Teku+Besu). We noticed a drop of approximately 17% in a 12 hour period (from an average of 3.37 to 2.8).  
The graph in Figure 8 shows the CL (Teku) CPU usage. We noticed a drop of approximately 15% on a 12 hours period (from an average of 3.27 to 2.78).  

![](https://hackmd.io/_uploads/rJUfgxHj2.png)
*Figure 7 - Total CPU usage (CL+EL) comparison*  
Purple line: Change deployed  
Green line: Change not deployed

---

![](https://hackmd.io/_uploads/Bkh5yeHi2.png)
*Figure 8 - Teku CPU usage (CL only) comparison*  
Green line: Change deployed  
Yellow line: Change not deployed


---


## Non validating nodes effects  
One of the goals of this change is to make the non validating nodes participate in the attestation processing and gossiping in order to reduce the workload on the validating nodes.  
As expected, the non validating node attestation gossiping participation increased significantly.   

![](https://hackmd.io/_uploads/B1eHQZrs2.png)
*Figure 8 - Non validating node gossip participation*  
Green line: Total participation  
The colored purple, red and green lines show the 256 epochs persistent subnet subscriptions

---

![](https://hackmd.io/_uploads/rk31umHo3.png)
*Figure 9 - Non validating node network traffic*  
Green line: Transmission bandwidth (rising)  
Yellow line: Reception bandwidth (no effect)




