This lab starts an OpenNMS instance, a 3-node ScyllaDB cluster, and optionally an instance for the Scylla Monitoring Stack in the cloud, for learning purposes.
To monitor a network, it is advised to enable ActiveMQ or Kafka and use Minions. For simplicity, the embedded ActiveMQ will be enabled.
Follow this guide for general information about how to configure Scylla/Cassandra for OpenNMS/Newts. Most of that knowledge is transferable to ScyllaDB.
The scripts used through this tutorial use envsubst, make sure to have it installed.
Make sure to log into Azure using az login
prior creating the VM.
If you have a restricted account in Azure, make sure you have the Network Contributor
role and the Virtual Machine Contributor
role associated with your Azure AD account for the resource group where you want to create the VM. Of course, either Owner
or Contributor
at the resource group level are welcome.
If you have a restricted account in Azure, make sure you have the Network Contributor
role and the Virtual Machine Contributor
role associated with your Azure AD account for the resource group where you want to create the VM. Of course, either Owner
or Contributor
at the resource group level are welcome.
Feel free to change the content and keep in mind that $USER
is what we will use throughout this tutorial to identify all the resources we will create in Azure uniquely.
Do not confuse the Azure Location or Region with the Minion Location; they are both unrelated things.
We're going to leverage the Azure DNS services to avoid the need to remember and using Public IP addresses.
In Azure, the default public DNS follow the same pattern:
To make the VMs FQDN unique, we're going to add the username to the VM name. For instance, the OpenNMS FQDN would be:
The above is what we can use to access the VM via SSH and to configure Minions.
This is a necessary step, as every resource in Azure must belong to a resource group and a location.
However, you can omit the following command and use an existing one if you prefer. In that case, make sure to adjust the environment variable RG_NAME
so the subsequent commands will target the correct group.
The reason for creating the VNET is that we need static IP addresses for each ScyllaDB instance (as a given Scylla node identifies itself within the cluster through its IP Address).
We will name each VM like follows, based on the chosen username and CIDR for the VNET and the subnet within it:
agalue-scylla1
(13.0.1.11
)agalue-scylla2
(13.0.1.12
)agalue-scylla3
(13.0.1.13
)Note the hostnames include the chosen username to make them unique, which is mandatory for shared resource groups and the default Azure DNS public domain on the chosen region.
Remember that each VM in Azure is reachable within the same VNet from any other VM through its hostname, and the following will take advantage of this fact when configuring seed nodes and Newts access from OpenNMS.
It is assumed that each VM would have at least 8GB of RAM (a Standard_D2s_v3 instance should be sufficient), although feel free to adjust it. The key element is that Scylla requires dedicated disks configured as RAID0 formatted as XFS, so additional disks for this purpose will be added (and the Scylla setup tool takes care of the configuration and formatting).
It is also assumed that agalue-scylla1
will be the seed node.
Network Topology will be used as that's a typical scenario in production. For this, we're going to use rack-awareness, but a single-dc. The DC name will be DC1
, and for the rack, we're going to use the hostname of the ScyllaDB instance.
Create the cloud-init YAML file as /tmp/scylla-template.yaml
with the following content for ScyllaDB 4.5, expecting to be installed in RHEL/CentOS 8:
For simplicity, I'm extracting the last digit from the hostname
and use it as part of the rack
name. You can apply other rules if needed.
Create the Scylla cluster on CentOS 8 VMs (with an additional disk of 500GB for data, as Scylla requires):
There is no need to open ports, as any VM can reach any other VM through any port by default, and the cluster won't be exposed to the internet (except for SSH via FQDN).
Also note that static IPs will be used because that's a Cassandra requirement (and Scylla is no different as it behaves the same way).
All the VMs will be created simultaneously; although, the bootstrap script takes care about having only one joining the cluster at a time.
Create a cloud-init script to deploy OpenNMS and PostgreSQL in Ubuntu with the following content and save it at /tmp/opennms-template.yaml
:
The above installs the latest OpenJDK 11, the latest PostgreSQL, and the latest OpenNMS Horizon. I added the most basic configuration for PostgreSQL to work with authentication. The embedded ActiveMQ is enabled, as well as Syslogd.
Create an Ubuntu VM for OpenNMS:
Keep in mind that the cloud-init
process starts once the VM is running, meaning you should wait about 5 minutes after the az vm create
is finished to see OpenNMS up and running.
In case there is a problem, SSH into the VM using the public IP and the provided credentials and check /var/log/cloud-init-output.log
to verify the progress and the status of the cloud-init execution.
You can SSH the Scylla VMs from the OpenNMS VM, as those don't have public IP addresses.
Work in progress (not verified).
This is an optional step, not required to use Scylla. Please note that its configuration requires listing all the nodes explicitly from the Scylla cluster, meaning if you're initializing more than 3, update scylla_servers.yml
accordingly.
Create a cloud-init script to deploy the Monitoring Stack in CentOS 8 with the following content and save it at /tmp/scylla-monitoring-template.yaml
:
Then,
Even if Scylla offers a comprehensive monitoring stack based on Prometheus, and even if the JMX statistics available in Cassandra are unavailable in ScyllaDB, it worth checking the Scylla servers like regular ones with OpenNMS.
Wait until OpenNMS is up and running and then execute the following:
Create the following cloud-init template to create a Minion (assuming the embedded ActiveMQ within OpenNMS is in place) and save it as /tmp/minion-template.yaml
:
Note the usage of environment variables within the YAML template. We will substitute them before creating the VM.
Then, create the runtime template:
Then, start the new Minion via multipass
with one core and 2GB of RAM:
When you're done, make sure to delete the cloud resources.
If you created the resource group for this exercise, you could remove all the resources with the following command:
If you're using an existing resource group that you cannot remove, make sure only to remove all the resources created in this tutorial. All of them should be easily identified as they will contain the username and the VM name as part of the resource name. The easiest way is to use the Azure Portal for this operation. Alternatively,
The reason to have two sets of deletion groups is that, by default, the list contains disks initially, which cannot be removed before the VMs. For this reason, we exclude the disks on the first set, and then we remove the disks.
Then clean the local resources: