Alejandro Galue

@agalue

Joined on Jun 13, 2018

  • The use case for having Cassandra (or ScyllaDB) as the backend for the performance metrics in OpenNMS is for storing a huge amount of non-aggregated data, which is not possible with RRDtool. RRDtool is very good for those installations with a finite and predictable amount of metrics where the size and the I/O requirements are feasible for modern SSD disks. This is important as RRDtool only scales vertically, meaning that when the current disks' limits are reached (mostly due to speed, not space), a faster disk is required. This is when Cassandra or ScyllaDB can help. Although there is a high learning curve, using any of these applications requires a commitment to have qualified personnel to manage this database. ScyllaDB is binary compatible with Cassandra (even at the SSTable level), but they are implemented very differently. Cassandra is implemented in Java, meaning that all the JVM tuning is required and other internal Cassandra tunings. On the other hand, ScyllaDB is implemented in modern C++, taking advantage of the CPU where it is running. That means ScyllaDB can manage huge machines as nodes, whereas Cassandra would require multiple instances on huge machines. It is feasible to have faster results with ScyllaDB compared with its Java sibling in terms of performance. Configuring and managing both applications is different, even if they will provide the same operational result with OpenNMS, so the decision should be carefully analyzed, especially by the team that will support this database.
     Like  Bookmark
  • This lab starts an OpenNMS instance, a 3-node ScyllaDB cluster, and optionally an instance for the Scylla Monitoring Stack in the cloud, for learning purposes. To monitor a network, it is advised to enable ActiveMQ or Kafka and use Minions. For simplicity, the embedded ActiveMQ will be enabled. :::success Follow this guide for general information about how to configure Scylla/Cassandra for OpenNMS/Newts. Most of that knowledge is transferable to ScyllaDB. ::: Requirements
     Like  Bookmark
  • This lab starts an OpenNMS instance and a 3 node ZK/Kafka cluster in the cloud and two Minions on your machine, using Kafka for communication through Multipass and Azure, for learning purposes. :::warning The lab doesn't cover security by default (user authentication and encryption), which is crucial if we ever want to expose the Kafka cluster to the Internet. A separate section covers the required changes for this. ::: :::success Keep in mind that nothing prevents us from skipping using the cloud provider and do everything with Multipass (or VirtualBox, or Hyper-V, or VMWare). The reason for using a cloud provider is to prove that OpenNMS can monitor unreachable devices via Minion. Similarly, we could use any other cloud provider instead of Azure. However I won't explain how to port the solution here. :::
     Like 1 Bookmark
  • This lab starts an OpenNMS instance in the cloud and two Minions on your machine, using ActiveMQ for communication through Multipass and Azure, for learning purposes. :::danger The lab doesn't cover security (in terms of encryption), which is crucial if you ever want to expose AMQ to the Internet. Follow this guide to learn how to do it with LetsEncrypt. ::: :::success Keep in mind that nothing prevents you from skipping using the cloud provider and do everything with Multipass (or VirtualBox, or Hyper-V, or VMWare). The reason for using a cloud provider is to prove that OpenNMS can monitor unreachable devices via Minion. Similarly, you could use any other cloud provider instead of Azure. However I won't explain how to port the solution here. :::
     Like 2 Bookmark
  • This lab starts an OpenNMS instance, a 3-node Cassandra cluster and optionally an instance for Cassandra Reaper in the cloud, for learning purposes. To monitor a network, it is advised to enable ActiveMQ or Kafka and use Minions. For simplicity, the embedded ActiveMQ will be enabled. :::success Follow this guide for general information about how to configure Cassandra for OpenNMS/Newts. ::: Requirements
     Like  Bookmark
  • :::warning I'm not a Cassandra expert. This guide should serve as a reference based on my experience with that complex piece of software when I helped customers deploying it in their environments. However, I won't be covering authentication and encryption. ::: Apache Cassandra is an open source, distributed, NoSQL database. It presents a partitioned wide column storage model with eventually consistent semantics. We use Cassandra in OpenNMS to store performance metrics through Newts. :::danger Sizing your cluster is crucial, and it is a task to be done before even think about configuring a Cassandra cluster. Follow the sizing guide for more information, as that analysis falls outside the scope of this guide.
     Like  Bookmark
  • The following explains how to perform TLS termination using NGINX and LetsEncrypt for OpenNMS and Grafana; all running on the same server. Even if you can manage TLS within the embedded Jetty available in OpenNMS, it is better and faster doing it outside of it using an external proxy. There are multiple ones out there, and NGINX is one of them. You could also do this using Apache HTTPD, Envoy, or any other. The following procedure has been designed and tested on RHEL/CentOS 8. Requirements Have an Azure Subscription ready. Install Azure CLI
     Like  Bookmark
  • A ring is a set of Calix C7 Devices that share the same IP Address. For this reason, you cannot apply the usual way to add a node to the OpenNMS inventory. The idea would be to add an OpenNMS node for each C7 entity with a unique SNMP agent, regardless of its nature: a node, a shelve, a card, a location, etc. Then, use the metadata feature to populate the specifics of the SNMP agent at the node level. This is what the SNMP Collector would use at run-time to know how to reach the SNMP Agent, as ALL the ring entities would have the same IP Address. In other words, multiple OpenNMS nodes will have the same IP address but a different port to reach their respective statistics. There are 2 ways to populate the inventory: Use requisitions via Provisiond. Manual provision via ReST API (without using requisitions or Provisiond in general).
     Like 1 Bookmark
  • This guide was designed for Meridian 2018 but should work on Horizon 22 or newer versions of OpenNMS. I'm assuming the operating system will be CentOS/RHEL 7, but all the commands should work in 8. Any yum command will internally use dnf if you're using CentOS/RHEL 8. Installation The installation process requires privileges. For this reason, the administrator must execute all the following steps as root or through sudo. User for Minion Create the minion user before installing the OpenNMS RPMs (ensure that /opt/minion doesn't exist). If the RPMs are already installed (meaning the /opt/minion directory exists and it is populated), fix the minion user to work as a regular user.
     Like  Bookmark
  • :::warning I'm not a PostgreSQL expert. This guide should serve as a reference based on my experience with that database when I helped customers deploying it in their environments. ::: PostgreSQL (PG) is a powerful, open source, object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, feature robustness, and performance. OpenNMS uses PostgreSQL to store the inventory and the core objects OpenNMS requires to be able to work properly. In other words, PostgreSQL is an essential and mandatory component, and OpenNMS cannot work without it. :::danger It is crucial to keep in mind that a PG cluster in primary-standby mode is for disaster recovery or high availability purposes. PostgreSQL is NOT a distributed database, meaning the data will always be handled by the primary server, which is the only one capable of executing mutations (i.e., add, modify and delete entries in tables). Standbys will be in replication mode all the time, and they can be used in read-only mode for reporting purposes, but not for normal OpenNMS operation.
     Like  Bookmark
  • This lab starts an OpenNMS instance and a 3 node Elasticsearch cluster in the cloud, for learning purposes. To monitor a network, it is advised to enable ActiveMQ or Kafka and use Minions. For simplicity, the embedded AMQ will be enabled, and a simple minion will be started to test flows. Requirements Have an Azure Subscription ready. Install Azure CLI Install Multipass
     Like  Bookmark
  • This guide was designed for Meridian 2018 but should work on Horizon 22 or newer versions of OpenNMS. I'm assuming the operating system will be CentOS/RHEL 7, but all the commands should work in 8. Any yum command will internally use dnf if you're using CentOS/RHEL 8. Installation The installation process requires privileges. For this reason, the administrator must execute all the following steps as root or through sudo. User for OpenNMS Create the opennms user before installing the OpenNMS RPMs (ensure that /opt/opennms doesn't exist). If the RPMs are already installed (meaning the /opt/opennms directory exists and it is populated), fix the opennms user to work as a regular user.
     Like  Bookmark
  • As described on the official documentation, the ReST API endpoint to manage scheduled outages requires two steps to have it working: Create a scheduled outage entry. Apply the created entry to one (or several) of the OpenNMS daemons through their packages (if applicable). Each scheduled outage consist of the following: A name A recurrence type A list of time range entries (each entry format depends on the type).
     Like  Bookmark
  • Some SNMP Traps codify IP Addresses as hex strings (or OCTET-STRING) as part of their varbinds; for instance, BGP Traps or CIKE Traps. Unfortunately, by default, Trapd in OpenNMS won't parse or decode them and store the content as it is. Because of this, when you render the details of the events associated with those traps in the WebUI, the IP addresses appear as unreadable content. One way to deal with this situation is by writing some custom code with Scriptd to process the hex content properly, transform it to its text form using SNMP4J (the library OpenNMS uses to handle SNMP), to then create an enhanced version of the event (and discarding the original one). To explain a working use case with an example, we're going to use the cikeTunnelStop trap from CISCO-IPSEC-FLOW-MONITOR-MIB. The event is not part of OpenNMS by default and requires compiling the MIB. Here is partial content of the MIB with the definition of that trap:
     Like 1 Bookmark
  • This lab starts an OpenNMS instance in the cloud and two Minions on your machine, using ActiveMQ for communication through Multipass and AWS, for learning purposes. This procedure is inspired by its Azure counterpart. :::danger The lab doesn't cover security (in terms of encryption), which is crucial if you ever want to expose AMQ to the Internet. ::: Requirements Have an AWS Account ready. Install AWS CLI
     Like  Bookmark
  • WSMAN support has been present in OpenNMS for a while. The primary motivator for adding support to this was Dell iDRAC. Still, it is Microsoft Windows infrastructure, the one that can have more benefits from it, as an alternative of SNMP to gather statistics from Windows. Through WinRM, everything exposed by WMI is accessible (if the user has the right permissions), which makes it a more attractive solution than using WMI directly. The challenge is always how to configure the Windows Infrastructure in terms of user authentication easily. Fortunately, thanks to Active Directory, we have GPOs (Group Policy Objects), which we can use to configure a set of Windows machines from a central place. Traditionally, WinRM (which stands for Windows Remote Management) is just a tool used by PowerShell to have remote access to a Windows machine to perform administrative tasks. For this reason, the protocol and access to WMI statistics require administrators privileges. That said, when speaking about monitoring Windows infrastructure, there is no need to have a privileged account, as the purpose of a monitoring tool is only to read statistics, and never perform changes.
     Like  Bookmark