I have narrowed down the search to two solutions ElastAlert2 and Sentinl.
Both of them require a backend server running and a plugin in Kibana.

## ElastAlert2
ElastAlert2 is the continuation of the discontinued ElastAlert.
It provides integrations to notify more services than it's predecessor.
- Provides "alerting on anomalies, spikes, or other patterns of interest".
- **Reporting of visualisations is NOT included**

### Setup
I am using a fork of ElastAlert2, which provides a REST server and a plugin to work with it.
#### [ElastAlert2 Server](https://github.com/Karql/elastalert2-server/tree/master)
A deployment can be found in **infra-650** of the infrastructure-operations repository.

It requires some configurations mentioned [here](https://github.com/Karql/elastalert2-server/tree/master#configuration). I wasn't able and did not have enough time to figure out how to provide it. An idea is to prebuild the image with the config.

A manifest of its service can be found in **infra-650** of the infrastructure-operations repository.

#### [ElastAlert Kibana Plugin](https://github.com/Karql/elastalert-kibana-plugin)
[A init container is used to install the plugin](https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-init-containers-plugin-downloads.html) before Kibana starts.
Alternatively the plugin can be [prebuild in the image that is deployed](https://www.elastic.co/guide/en/cloud-on-k8s/master/k8s-custom-images.html).

**All the manifests mention below and more can be found in branch infra-650 of the infrastructure-operations repository.**

# kibana.yml
  podTemplate:
    spec:
      initContainers: # (1)
        - name: sentinl-installer
          securityContext:
            runAsUser: 0
          command:
            - sh
            - -c
            - |
              if [ -d "/usr/share/kibana/plugins/" ]; then # (2) 
                rm -rfv /usr/share/kibana/plugins/
              else
                mkdir /usr/share/kibana/plugins/
              fi
              bin/kibana-plugin install https://github.com/Karql/elastalert-kibana-plugin/releases/download/1.6.2/elastalertKibanaPlugin-1.6.2-8.8.0.zip
          volumeMounts: # (3)
            - name: plugins-storage
              mountPath: /usr/share/kibana/plugins
      containers:
        - name: kibana
          volumeMounts:
            - name: plugins-storage
              mountPath: /usr/share/kibana/plugins
      volumes:
        - name: plugins-storage
          persistentVolumeClaim: # (4)
            claimName: kibana-pv-claim

(1) - The init container config
(2) - Since the folder is persisted delete it's content on every start or created if does not exist. Then install the plugin
(3) - Volume mount config
(4) - Persistent volume claim config

The plugin is using the Kibana config directly

# kibana.yml
  config:
    elastalertKibanaPlugin.serverHost: elastalert2-server.default.svc # backend server host
    elastalertKibanaPlugin.serverPort: 3030 # backend server port
    elastalertKibanaPlugin.serverSsl: false #insecure connection
    elastalertKibanaPlugin.serverPath: "" # idk

## [Sentinl](https://github.com/sentinl/sentinl)

**This tool ticks all the boxes when it comes to our needs. Unfortunatelly its latest release was in 2020 and it's not supported by Kibana**... So I [forked it](https://github.com/hero101/sentinl).

### Setup
The package consists of a backend server in Python and a Kibana plugin in AngularJS (v1).

The same setup from above can be used to install the plugin.
Just substitute the plugin source with this one

https://github.com/hero101/sentinl/releases/download/8.8.0/sentinl.zip

The result after the installation is Kibana not being able to start because it cannot find a module. No further investigation was conducted.

If this is resolved it will require some configuration to point it to the Elasticsearch host and credentials.

## Final thoughts
The most realistic scenario I can think of is that we create our in-house solution that we have full control of.
It can follow the same principle as the tools I have reviewed - having files with rules to alert on.
Elastic and Kibana can create rules and alerts and ingest documents via an index connector.
Then our tool can scan for documents and alert based on them.