# Queue Overload - Load shedding

## Navigation

1. [Problem](https://hackmd.io/@jwdunne/HJXyhNY4h)
2. [Observability](https://hackmd.io/@jwdunne/S1pJ1CgHn)
3. [Testing](https://hackmd.io/@jwdunne/H1zKkAeSn)
4. [Throughput optimisation opportunities](https://hackmd.io/@jwdunne/H1h2k0xH3)
5. [Backpressure](https://hackmd.io/@jwdunne/B1WZeCeBh)
6. [Load shedding](https://hackmd.io/@jwdunne/BJB4MReH2)
7. [Autoscaling](https://hackmd.io/@jwdunne/Bkw_zAxHn)

## Solution

We can drop these messages outright in times of overload:

- `Leadflo\Integrations\Listeners\MonitorIntegrations`
- `Leadflo\Integrations\Listeners\MonitorSyncs`
- `Leadflo\Integrations\Listeners\Healthcheck`
- `Leadflo\Integrations\Events\IntegrationSyncStateChanged`
- `Leadflo\Integrations\Listeners\TimeoutSyncingIntegrations`
- `Leadflo\Email\Listeners\ProcessFeedbackQueue`
- `Leadflo\App\Listeners\ActionCountsBuilder`

This one is trickier as these listeners extend from the `BaseListener`, which already defines an abstract `execute` method. We would have to do something different. Perhaps `maybeExecute`, which signals that the listener may not be executed at all under times of high-load:

/**
 * @template T
 * @require-extends BaseListener<T>
 */
trait Sheddable
{
	private readonly QueueLoad $queueLoad;

	/** @param T $event **/
	abstract public function maybeExecute($event): void;

	public function execute($event): void
	{
		if (!$this->shouldQueue($event)) {
			return;
		}

		$this->maybeExecute($event);
	}

	/** @param T $event **/
	public function shouldQueue($event)
	{
		// load shedding stuff here
	}
}

The reason why we implement a guard clause that uses `shouldQueue` is because `shouldQueue` only prevents the message from being queued. If it's already on the queue, we want to ignore that too and get it off the worker ASAP.

## Implementation

- Implement `Sheddable` trait
- Make `MonitorSyncs` listener shed load
- Make `MonitorIntegrations` listener shed load
- Make `ActionCountsBuilder` listener shed load