# **The ongoing Waku scaling**

*to infinity and beyond*!

With the ongoing efforts of teams to scale Waku, also remains the need to have ongoing practical tests aimed at our theoretical calculations. You and others following or learning about Waku might get a bit worried once you realize about the ongoing scaling process. Scaling is one thing. Scaling securely is another.

As we scale our tech, we guarantee to build upon our [principles](https://our.status.im/our-principles/) while maintaining full transparency.

We are not just promising that we're not running your messages through a firebase backend—no need to trust only our words. Find out for yourself on our and follow our live work through this pocket guide that scales. In this article, you'll find a pocket guide to our current scaling goals throughout 2023, our new ongoing tests, and points to relevant resources to this workstream.

---

## **Waku, Waku?**
*first, a refresher of Waku.*

[Waku](https://waku.org/) is a decentralized, open-source suite of privacy-preserving, peer-to-peer messaging protocols. It is designed for general messaging with humans, machines, or a hybrid situation. Waku runs everywhere. On your phone, a server, or through your browser.


Waku is our communication protocol for our [Status Messaging](https://status.im/) application. See also [others that use waku](https://waku.org/#who-is-using-waku).

We apply [Waku2-Relay](https://hackmd.io/@vac/main/%2FYYlZYBCURFyO_ZG1EiteWg#11WAKU2-RELAY-gossipsub) on top of gossip protocol [GossibSub](https://docs.libp2p.io/concepts/pubsub/overview/) from libp2p to route messages & information with a subset of connected peers.
- **A gossip protocol** is a peer-to-peer protocol in which nodes periodically exchange information about themselves and other nodes they know.
*I recommend playing with this [(web)simulator by flopezluis](https://flopezluis.github.io/gossip-simulator/) to visually understand how a gossip protocol intends to work.*

This implementation was a large improvement in perspective to Waku V1. In V1 we applied a [flooding mechanism](https://hackmd.io/@vac/main/%2FYYlZYBCURFyO_ZG1EiteWg#Routing-example-flooding-1) to distribute the messaging. This meant that every peer forwarded every new incoming message, which led to unnecessary duplication and wasted bandwidth.

#### Bandwith improvements on chart
*[learn more here about how v1 & v2 compare](https://vac.dev/waku-v1-v2-bandwidth-comparison)*

---

## **Ongoing Processes**
Even after our successful adaption of Waku V2, the nature of scaling requires a constant need to innovate within our product capacities. These current milestones are mainly focused on the scaling of Waku Relay. As we securely increase the technical capacities of Relay, its goals are to increase the number of users our protocol can securely handle.
##
### **Status MVP: Scaling Communities to 10K**
[Communities](https://https://status.im/get-involved/community-groups/) are groups within Status run either by communities or by Status themselves. They can communicate on a shared channel.

Primary goal:

* Scale to 10K Status Community users, spread across ~10 to ~100 communities

Only desktop users, primarily those using Relay but with experimental/beta support for filter and light push for clients with poor connectivity. Communities, private group chats, and 1:1 chats are considered. Public chats are excluded.

Have a real-time look at how far we are on our ongoing technical objectives [here.](https://github.com/waku-org/pm/issues/12) On this tracker, you will not only find where we are at, but it will also offer you a look at what's happening under the hood as all of our code is [open-source](https://github.com/waku-org)!

Deadline: May 31st, 2022
VAC, Status, and Waku are directly involved.

##
### **Waku Relay Scaling Plan to 1m**
[SeM](https://github.com/vacp2p/research/issues/187) has their main priority in 2023 to reach 1 million user capacity and devised a 3-step plan to do so.

1. Scale a single shard (pubsub mesh) to 10k users.
2. Scaling the Waku network used by Status App to 1 million users through [static sharding](https://rfc.vac.dev/spec/51/#static-sharding).
3. specifications for both Status protocols and Scaling extensions for Waku protocols.

On successfully scaling communities to 10K, we aim to create similar shards of those communities. Then by creating a large collective supported by smaller independent collectives, we can reduce the load on a single larger collective.

Expected Deadline: June 31st, 2022 [depedent on 10k plan](https://github.com/waku-org/pm/issues/12)

---

## **Testing**

Everything is supposed to work as we assume and have researched. However, assumptions are only rarely viable in building robust systems.

As introduced by Corey P. in [this forum post](https://forum.vac.dev/t/waku-v2-scalability-studies/142/3), the upcoming tests are important for various our workstream participants to know the thresholds till where we can maintain our principle and quality standard

With our ongoing partnership with Kurtosis, we're strongly armed for these challenges. Kurtosis will help us simulate complex distributed network operations and ship a test network.

The roadmap of these ongoing network tests can be followed here [here](https://github.com/waku-org/pm/issues/2).

| Test Name | Link | Summary | Status |
| -------- | -------- | -------- | -------- |
| WaKurtosis First Network test | [Github](https://github.com/logos-co/wakurtosis) | Setting up the Kurtosis environment and Benchmark Waku performance as it scales |ongoing
| Waku Relay Scaling | [Github](https://github.com/vacp2p/research/issues/158) | How does a single pubsub mesh scale, including an outlook towards what topic sharding can achieve | [results](https://github.com/vacp2p/research/issues/158#issuecomment-1434485991)

Metrics we're looking for:
- latency
- reliability
- resource usage (bandwidth, CPU, memory) against network size and message rate.

These test results will help us get closer to calculating the exact requirements to support 1MM users, how far we can push communities, draw boundaries for our nodes & capacities, and obtain precise numbers for simulations or predictions.

## The future
What's next? Where are the big questions ahead?

There is still a lot of work to be done to scale securely, swiftly, and privacy-preserving a lot of big questions will remain involved.

From Spam-protection to better node discovery to potential evolutions of the Gossip Network. We can lean on industry-wide research and work and apply our customizations according to our principles. Im excited to see the results of our upcoming tests, just as much to see our network go through a new sharding process.

Keep revisiting this page, as you can find all our upcoming updates [here](https://hackmd.io/psCdvgKIR9St4gmT-pYVOA?both#Scaling-of-Waku)!

---

## **Dive deeper with these resources**

| Resources | Why would you read this |
|--|--|
|[Gossibsub 1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md) | Dive deeper in the specific gossip protocol details|
|[Gossibsub evaluation by Protocol labs](https://research.protocol.ai/publications/gossipsub-v1.1-evaluation-report/vyzovitis2020.pdf) | learn about Gossibsubs ressilience |
|[Waku: A Family of Modular P2P Protocols For Secure & Censorship-Resistant Communication](https://arxiv.org/ftp/arxiv/papers/2207/2207.00038.pdf)) | Technical 2-pager of Waku, condensed but strong summary! |
|[Initiation of Waku v2 Scalability studies](https://forum.vac.dev/t/waku-v2-scalability-studies/142) | Where does our need come from to do these studies & tests |
|[NEAR & Kurtosis ](https://near.org/case-studies/kurtosis/) | Learn how NEAR implemented Kurtosis for their test network |
|[Kurtosis Docs](https://docs.kurtosis.com/) | Learn more about how you can apply Kurtosis yourself|

## **Contribute to the decentralization of communication**
Help us make sure that we're not just scaling but also staying decentralized by running your own Node! We can use your help just as much as your friends help [Run a Waku Node](https://github.com/waku-org/nwaku/blob/master/docs/operators/overview.md)

#
|| Build with Waku on your application
|--|--
|Nim-Waku|https://github.com/waku-org/nwaku
|Javascript | https://js.waku.guide/
|Go | https://github.com/waku-org/go-waku
|Rust | https://github.com/waku-org/waku-rust-bindings
|More... | https://waku.org/platform