--- tags: mads --- # (OLD) MADS IoT Platform # (OLD: do not continue to edit this) ![datakrew](https://i.imgur.com/gA5Mo9D.png) ###### Document Metadata ``` tags: `architecture`, `documentation`, `features` start-date: Feb 3, 2020 version: 1.0 authors: Arjun Singh, Sumanta Bose ``` > This document is a developer's guide for building the MADS IoT Platform. ## What is MADS IoT Platform? DataKrew's IoT platform comes with an AppStore, with dedicated apps to digitally (M)onitor, (A)utomate, (D)iagnose and (S)ecure built environments, smart cities and industrial infrastructure - hence we call it MADS! MADS apps can securely combine data visualisation with advanced analytics to provide easy-to-understand insights and actionable maintenance recommendations. It can be used to manage building and asset health, to enable you to make informed reliability decisions and optimize your operations. MADS collects, stores and transports data using secure cryptography, resilient even to quantum attacks. ## Architecture The architecture of the MADS IoT platform is based on micro-services architecture. The architecutre was chosen keeping in mind the requirments of: - Single responsibility. - Loose coupling between different services. - Indpendent deployments. - Organized around business capabilities and needs. - Ownership by small teams. - Easy maintenance and testing. - Pluggable architecture, services can come into picture and leave as and when needed. Below is an illustration diagram of the MADS architecture. ![platform_architecture](https://i.imgur.com/rXKIIA6.jpg) ### Event Driven Design Keepig in mind the above requirements as well as the natural behavior of IoT data, `event driven design` meets the requirement. In the world of micro-services traditonally `synchronous request-response` was pretty common. However, as the number of services increase the web of synchronous interactions grows with them. The synchronous request-response creates `tight coupling` as a result of which if one of the services change, others need to modify themeselves to accomodate it. This is far from ideal as business requirements evolve at a fast pace. ![request_vs_event](https://i.imgur.com/TrDq1E5.png) In the world of Internet-of-Things (IoT) all the interaction over the network can be split in to three distinct ways: - *Command*: Request for some operation which will change the state of the system. Commands typically expect completion and have a result. - *Events*: Events represent fact and notification and they are the most commonly used in IoT infrastructure. - *Queries*: Queries are used to ask for results of analysis or the data log. They don't change the state of the system. Designing event driven systems requires the presence of a broker which would act as a single source of truth for all the events on the platform. These events are shared between different services. Considering the requirement to handle huge amount of streaming data and events, `Apache Kafka` was chosen. Apache kafka is a streaming platform. It consists of clusters, distributed over multiple machines for tolerance and linear scale-out. It's throughput properties allows it to stand out among other technologies being used for stream processing. **Apache Kafka Properties** - `Efficient structure for retainging and distributing messages`: Kafka consists of a partitioned replayable log. Log structured approach allows for sequential reads writes and batched sequential operations, resulting in better overall performance compared to B-trees(B-trees need to be kept in memory which limits the retention of data for performance). - `Linear Scalability`: Since kafka works with log partitioned structure, the logs can be spread over multiple machines, which are essentially tied together. The system takes care of reliable routing, replication for fault tolerance and handles failures gracefully. - `Load segregation for multiple services`: A kafka cluster can be used by many services, but too much load may result in DDoS or instability. To counter this kafka provides throughput control features called `quotas`. These are too limit bandwidth available for different services. - `Strong Ordering Guarantees`: Some IoT applications require ordering in the data that has been sent, kafka allows ordering data based on `keys`. - `Message durability`: Kafka provides durability through replication. - `Long-term data storage`: This is one of the biggest advantages compared to other streaming services that a large amount of data can be stored in different topics. - `Security`: Client side security can be added to kafka with `kerberos` or `TLS`, additionally Unix like permissioned system can also be used to determine hierarchical access. In the MADS IoT platform, different services are communicating with each other by publishing events to topics inside kafka and subscribing to topics for events they need. ### MADS Transport Microservices Transport microservices deals with providing a mechanism for IoT gateways to send data to the cloud. The supported transports in MADS are: - `HTTP`: HTTP(S) is a general purpose network protocol that can be used in IoT application for exchanging data. Details can be checked [here](https://www.w3.org/Protocols/rfc2616/rfc2616.txt). The HTTP service is written in elixir lang with support of phoenix library and secured with TLS. - `MQTT`: [MQTT](http://mqtt.org/) is a light-weight publish-subsribe protocol making it ideal for IoT gateways. The MQTT service is written in elixir lang and the broker used for mqtt is Vernemq. The base security layer is over TLS. - `CoAP`: [CoAP](https://tools.ietf.org/html/rfc7252) is a light-weight IoT protocol for constrained devices. CoAP protocol is UDP based, but similar to HTTP it uses request-response model. CoAP observes option allows to subscribe to resources and receive notifications on resource change. The CoAP service is written in elixir lang. - `LoRaWAN`: [LoRaWAN](https://lora-alliance.org/) is a media access control (MAC) protocol for wide area networks. It is designed to allow low-powered devices to communicate with Internet-connected applications over long range wireless connections. LoRaWAN can be mapped to the second and third layer of the OSI model. It is implemented on top of LoRa or FSK modulation in industrial, scientific and medical (ISM) radio bands. The underlying stack used for creating the microservices `HTTP`, `MQTT` and `CoAP` is Elixir/OTP. The LoraWAN service is based on a service from [chirpstack](https://www.chirpstack.io/) to manage LoRa devices, chirpstack provides a network and application server which are stable and battle tested. ![transport_layer](https://i.imgur.com/hvBBKzD.png) ### MADS Services MADS services cater to different business requirements in domain of IoT. The services are modeled around business domains to provide a list of features, this is also referred to as a bounded context in DDD, domain driven design. The underlying stack for most of the services is based on erlang. Erlang was chosen exclusively because it promises a highly available system. For a system to remain highly available the following challenges needs to be tackled: - `Fault Tolerance`: The system should keep running when some unforseen bugs creep in, network connection drops, or the machine running the system crashes. In case of such an event, it should be localized and recovery should be possible while running rest of the components. - `Scalability`: A system should be able to scale as the load(users, gateways, etc) increase. - `Distribution`: The system should run on multiple machines. This allows to scale the platform horizontally as well as provide service globally with less network latency. - `Responsiveness`: The system should be reasonably fast and responsive. - `Live update`: In some cases new it should be possible to update the running software without restarting the system. ![highly_available](https://i.imgur.com/Du7UB9J.png =350x) Erlang ecosystem provides tools to handle above challenges very well. The erlang concurrency model unlike other languages does not rely on heavy weight threads and OS processes. Instead the Erlang Virtutal Machine, BEAM takes care of concurrency by itself. A typical erlang process is much lighter and faster to spin than OS processes. This allows the BEAM to run millions of such processes, with it's own schedulers on available CPU cores. Erlang ecosystem provides tools to handle above challenges very well. The erlang concurrency model unlike other languages does not rely on heavy weight threads and OS processes. Instead the Erlang Virtutal Machine, BEAM takes care of concurrency by itself. A typical erlang process is much lighter and faster to spin than OS processes. This allows the BEAM to run millions of such processes, with it's own schedulers on available CPU cores. ![beam](https://i.imgur.com/oa3PIZJ.png =450x) Although, erlang provides various functionalities for a highly available system required to businesses. It can not match the number crunching capabilities of machine compiled languages like C and C++. However erlang provides a way to call native functions in C for number crunching applications. ### Data Storage MADS uses different type of databases or caching mechanisms depending on the features a service provides. Also, since the design is extensible new services will join in to support new business domains. The base platform provides three important services. - MADS Core Service - MADS Analytics Service - MADS Notifications and Realtime Service. MADS core service allows users to configure the application. This includes adding gateways, users, providing access controls and a plethora of configuration related tasks. Keeping this in mind the database used for the core service is `postgres`. [Postegres](https://www.postgresql.org/) is a open-source relational database with better features and high throughput compared to other relational databases. MADS Analytics service is about statistical analysis and AI based learning models. To enable different domains it consists of two different databases [`Hadoop`](https://hadoop.apache.org/) and [`Cassandra`](http://cassandra.apache.org/). While the hadoop database augments batch processing, cassandra helps in stream processing. MADS notifications or messaging service needs a fast caching mechanism for different types of notifications and messaging. To enable this the database used is `Redis.`. [Redis](https://redis.io/) is an open-source in memory database which is optimized for speed. ### Deployment and Orchestration MADS services are deployed as containerized applications. Containers are the best way to bundle and deploy applications. They provide a platform and premise independent way of doing deployments. Also, using containers allows for easy continous integration and continuous deployments (CI/CD). The lightweight nature of containers also allows easy managing which will ensure that there are minimum to zero downtime. The containers are managed using a container orchestration servcie. MADS makes use of `kubernetes` as the orchestration service. The below image shows different components of `kubernetes` working together to provide orchestration. ![kube_intro](https://i.imgur.com/P9CvhX1.png =550x) Kubernetes requires a container runtime to provide the service. The container runtime required is being provided by `docker` for managing containers and providing different services. The below image shows different components of kubernetes interacting with the container runtime. ![kube_components](https://i.imgur.com/ZYhNVuM.png) Kubernetes alongwith docker runtime provides following services: __Service discovery and load balancing__: Kubernetes can expose a container using the DNS name or using their own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable. __Storage Orchestration__: Kubernetes allows us to automatically mount a storage system of our choice, such as local storages, public cloud providers, and more. __Automated rollouts and rollbacks__: We are able to describe the desired state for our deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, we can automate Kubernetes to create new containers for our deployment, remove existing containers and adopt all their resources to the new container. __Automatic bin packing__: We provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. We can tell Kubernetes how much CPU and memory (RAM) each container needs. Kubernetes can fit containers onto your nodes to make the best use of our resources. __Self-healing__: Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve. __Secret and configuration management__: Kubernetes lets us store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. We can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in the stack configuration. ## Applications ![mads_apps](https://i.imgur.com/uoXVsYP.png) ### [Core Apps](#Core-Apps-Info) Core apps are apps that are part of the core bundle of the MADS platform. There is no concept of installing or uninstalling core apps. Thus there is no way to uninstalled a core app by the user or by Datakrew. There are three core apps as enlisted in the table below. | Sl. No. | App Name | About | Details | |---------|----------|------------------------------------------------------|--------------------| | 1 | AppStore | Explore apps supported in MADS or develop your own. | [C1](#C1-AppStore) | | 2 | Settings | Configure and customise your MADS settings. | [C2](#C2-Settings) | | 3 | Support | Learn about MADS, or get in touch with support team. | [C3](#C3-Support) | All non-core apps are available in the AppStore, which can be installed or uninstalled by the user. Non-core apps include *Productivity* apps, *Management* Apps, *Analytics* Apps and *General* Apps. For a given client, based on their requirements Datakrew may pre-install certain non-core apps as and when required. <!-- 1. AppStore 2. Settings 3. Support --> ### [Productivity Apps](#Productivity-Apps-Info) | Sl. No. | App Name | About | Details | |---------|----------------------|------------------------------------------------------------------------------------------|--------------------------------| | 1 | Dashboards | Create and share custom dashboards with choice of widgets. | [P1](#P1-Dashboards) | | 2 | Digital Twin | Virtually represent real-time data sources of a site. | [P2](#P2-Digital-Twin) | | 3 | Task Manager | Assign tasks and tickets in kanban or list view for follow within an organisation. | [P3](#P3-Ticket-Generator) | | 4 | Report Wizard | Create and share custom reports with user-defined parameters (with company logo/format). | [P4](#P4-Report-Wizard) | | 5 | Alerts and Reminders | Create and share custom alerts and reminder based on any data source. | [P5](#P5-Alerts-and-Reminders) | | 6 | Feed Viewer | Tick posts (from MADS apps, or external source like RSS, twitter, etc.). | [P6](#P6-Feed-Viewer) | <!-- 1. Dashboards 2. Digital Twin 3. Ticket Generator 4. Report Generator 5. Alerts and Reminders --> ### [Management Apps](#Management-Apps-Info) | Sl. No. | App Name | About | Details | |---------|---------------------|----------------------------------------------------------------------------------|-------------------------------| | 1 | File Manager | Manage files (copy, move, open, delete, search) within the user's MADS instance. | [M1](#M1-Dashboards) | | 2 | IoT Gateway Manager | Manage credentials, security settings, JSON structure of IoT gateways. | [M2](#M2-IoT-Gateway-Manager) | | 3 | Role Manager | Manage roles and access controls of users of an organisation. | [M3](#M3-Role-Manager) | | 4 | Entity Manager | Manage sites, sections, equipments, other assets, etc. | [M4](#M4-Entity-Manager) | | 5 | Tool Manager | Manage RFID tagged tools within a site. | [M5](#M5-Tool-Manager) | | 6 | Widget Manager | Create, manage and share widgets within an organisation. | [M6](#M6-Widget-Manager) | <!-- 1. File Manager 2. IoT Gateway Manager 3. Role Manager 4. Entity (Site) Manager 5. Tool Manager 6. Widget Manager 7. Task Manager --> ### [Analytics Apps](#Analytics-Apps-Info) | Sl. No. | App Name | About | Details | |---------|-----------------|--------------------------------------------------------------------------------------|---------------------------| | 1 | Data Cruncher | Interactive visual data ingestion, utilisation and analysis tool. | [A1](#A1-Data-Cruncher) | | 2 | Trend Predictor | Tool to analyse and predict trends and patterns in telemetry data stream. | [A2](#A2-Trend-Predictor) | | 3 | Usage Analyser | Tool to analyse trends and patterns in user's dashboard interaction and screen time. | [A3](#A3-Usage-Analyser) | <!-- 1. Rule Engine 2. Trend Analyser 3. Behaviour Analyser 4. AI Recommender --> ### [Security Apps](#Security-Apps-Info) | Sl. No. | App Name | About | Details | |---------|------------------|-------------------------------------------------------------------------------|----------------------------| | 1 | MADS Security | Provides protection against malware, trojans, and other malicious agents. | [S1](#S1-MADS-Security) | | 2 | Login Tracker | Logs all MADS logins, patterns, and individual's metadata in your org. | [S2](#S2-Login-Tracker) | | 3 | Network Analyser | Records and t/s network data traffic issues by application, IP or user. | [S3](#S3-Network-Analyser) | | 4 | Secure Exporter | Manages security issues of exported links, and other content in/outside MADS. | [S4](#S4-Secure-Exporter) | | 5 | App Locker | Safeguards individual apps by asking a pin/pattern to access the app. | [S5](#S5-App-Locker) | | 6 | Password Manager | Stores passwords/pin/pattern of your apps with extra layers of security. | [S6](#S6-Password-Manager) | ### [General Apps](#General-Apps-Info) | Sl. No. | App Name | About | Details | |---------|------------|------------------------------------------------------------------------|----------------------| | 1 | Calendar | Intra-org event scheduler that syncs with your company calendar. | [G1](#G1-Calendar) | | 2 | Calculator | Perform basic, advanced, or scientific calculations and share results. | [G2](#G2-Calculator) | | 3 | Chat App | Intra-org chat to share dashboards, files, reminders, etc. | [G3](#G3-Chat-App) | <!-- 1. Calendar 2. Calculator 3. Chat App --> ## Applications Info ### Core Apps Info #### C1. AppStore - Description: The MADS Platform offers a repository of available applications (apps) in the AppStore for the client. These range of apps are suitable for many different types of generic business tasks or specific Industry 4.0 goals. - Features: - Shows a list of available apps. - Some apps can be free, other apps may be paid (either one time, or subscription basis based on tiered features, or number of user, or some other criteria) - Each app will have app screenshots, small videos to introduce the app, and a write up of the app about instllation steps, update versions, and new features in the latest update. - Each app should show the resources it will use (analogy: camera, contacts, GPS location, etc for phone apps). --- #### C2. Settings - Description: App to configure and customize your MADS settings. You can set your language and region, change the theme of your platform, choose different modes for notifications, and much more. The settings for specific apps are explained in the chapters for those apps. - Features: - Has platform settings like general, wallpaper, security, accounts, etc. - Has personal settings like user profile, status, sharing preferences, etc. - Has billing settings like subscription details, payment details, invoice, etc. - *(See settings of your phone for better understanding)* --- #### C3. Support - Description: Need help? MADS Support app is your personalized guide to the best options from MADS. Find answers with articles tailored to your products and questions. Call, chat or email with an expert right away, or schedule a callback when it’s convenient. Get a customisation or app service from a designated MADS support personnel. MADS Support is here to help. - Features: - Refer to https://apps.apple.com/us/app/apple-support/id1130498044 for features and ideas. - The Support app has following tiles. - MADS 101: Basic interactive tutorials and video tutorials about the MADS platform. - What's new: Press release about new features in MADS. - Webinars: Subscribe to regular MADS webiniars. - Community: Connect with the MADS community, raise a question or contribute in an app. - Contact: Chatbot, chat, call, email us. - Customise: Request us to develop a custom app for your business needs. --- ### Productivity Apps Info #### P1. Dashboards - Description: App to show historical as well as real time visualizations in the form of widgets from different data sources. - Features: - Upon opening this app, user will be presented with a tile view (and sneak preview) of all the dashboard they have created, or others have shared with them. For example, see https://demo.thingsboard.io/dashboards - An user can create a new dashboard, edit an existing dashoard, or share a dashboard with other user(s). - If an user A has shared a dashboard (D1) with an user B, the Dashboard app of user B will generate a notification that user A has shared a dashboard D1. - User B can view and accept the dashboard D1. - User A will give access rights to user B on the dashboard. Types of rights are: - view only - view and edit - view, edit and share with other user - User B can request user A to modify access rights type - A dashboard can show multiple widgets. - When an user creates or edits a dashboard, they can add new widgets to the dashbaord, from a list of available widgets from the **Widget Library** app. - A widget is a chart type, such as *line, spline, time-series, area chart, stacked chart, bar chart, column chart, histogram, pie chart, donut chart, bubble chart, scatter plot, meteogram, multiple axes chart, timeline, gantt chart, 3D chart, series gauge, solid gauge, activity gauge, heat map, tile maps, tree map, radar chart, wind rose, radial bar, spiderweb, error bar, waterfall, funnel chart, pyramid chart, venn diagram, etc.* (See https://www.highcharts.com/demo) - Each widget will be associated with one of multiple `data source`. - A `data source` is a stream of primary or secondary data. This is a specific term, not to be interchangeably used with any other term. - A primary `data source` is one that comes from on-ground telemetry, such as sensor data, section telemetry data, site telemetry data, or user telemetry data. - A secondary `data source` is one that is arrived at after computation on one or more primary or pre-existing secondary data. - A dashboard will have a `view` mode and an `edit` mode. - In `view` mode the widgets are fixed and can't be adjusted for size, data source, etc. However in `view` mode, user can make a widget full-window (maybe not full-screen!, similar to TB) - In `edit` mode, user can add new widgets, change widget dimensions, duplicate widgets, change data source, etc. - A dashboard has the feature of a dashnoard-wide `timewindow`. - This means means that all the widgets in the dashboard showing historical data, will show the data within the start and end timestamp specified in the `timewindow`. This can be implemented as a calendar dropdpown to choose the start and end timestamp. (See TB and Grafana for reference). - However, user can change the `timewindow` of any specific widget(s) in the dashbaord, if they want. - A dashboard can be saved and exported as a JSON file (similar to TB). The exported file can be used to reconstruct the dashboard (needs to render as it originally was). It will save the metadata of the dashboard, such as: - who created the dashboard - which users is the dashboard shared with - what are the widgets in the dashboard - what are the data sources of those widgets - etc (can add later) --- #### P2. Digital Twin - Description: Allows the user to create a digital mapping of any entity(e.g. a processs floor with multiple machines, collection of farm land, individual equipment etc.) - Features: - User is able to create a diagramatic representation of the process or product using a [draw.io][dig_draw_io] like tool. - In the edit mode the user is able to assign a data source to individual components. - The user can also select the widget in which the data would be shown. The data is real time value received from the physical equivalent of the component. The data is updated continuously without any reloads. - The user is able to define transitions and animations for a component. - A site has multiple digital twins, a digital twin can also be associated with a section or equipment. - Improvements - Add feature to co-relate data between different components of a twin. --- #### P3. Task Organiser - Description: An app to generate tickets for various tasks in the platform. Tickets are way to assign a task to be completed in the application(e.g. adding new devices, creating new dashboards, attending to telemetery alerts, maintenance on ground etc.) It allows managing tickets using agile or kanban methodologies. - Features: - User can create a ticket and assign to any user. - It has basic trello like flow where user can create different columns and move tickets in between them. - Every ticket has a completion deadline. - There is be a calendar page to show differnt tickets due on differnt dates for a particular user, the admin is able to see tickets for all the users. - An alarm is generated for user if ticket is overdue. - Tags can be added to a ticket which will assist with search. - A ticket can contain description, files and images for that particular ticket. - The task organiser has multiple project boards for independently managing different tasks. --- #### P4. Report Wizard - Description: - Features: - Reference: - https://ubiq.co/report-generation-software - https://en.wikipedia.org/wiki/Report_generator - https://en.wikipedia.org/wiki/List_of_reporting_software - https://www.inetsoft.com/business/solutions/report_generator_software/ - https://www.finereport.com/en/reporting-tools/report-generation.html - https://www.softwaretestinghelp.com/reporting-tools/ - https://www.quora.com/Which-is-the-best-report-generation-software-Open-source-preferred --- #### P5. Alerts and Reminders - Description: App to send alerts and reminders in the platform. - Features: - Gateway related alerts, for example if device is not performing properly (gateway switched off, gateway not sending data at required intervals, firmware updated, configuration updated). - Telemetery related alerts if data is above certain threshold. The threshold can be based on different policies. - Alert for new devices added. - Alert for ticket raised to solve any issue. - Alerts are of two types in app and email or sms based. User while creating an alert can select what kind of alert he wants and user groups to which the alert should be sent or assigned. - Alerts can be associated with any entity(site, section, equipment, gateway, sensors). - Alerts based on user activity. - Alerts can have a level of severity (CRITICAL MAJOR MINOR WARNING) - Alerts can also be raised via the rule engine. --- #### P6. Feed Viewer - Description: App to show posts (from MADS apps, or external source like RSS, twitter, etc.). - Features: - Any app in the MADS platform (example: Dashboard, Digital Twin, etc) can generate posts. - Posts are mainly short news headlines, blog entries, audio, video, real-time stock market figures, site specific highlights, insights, daily/hourly/monthly report card etc, - The format of a post is something similar to a facebook post or a tweet or a tumblr post, in a box. Next post is below the previous post in a long scrollable column format. - Posts can also be generated from external sources like twitter, etc. - The Feed Reader app will show a scrollable/auto ticking list of posts. - The Feed Reader app can run in two modes: - Manual mode (User scrolls) - Auto mode (Auto tick, suitable for TVs, big screens in company front office, sites, etc.) - Every post has an author, a timestamp. - An author could be a MADS app, an external app, or a human user. - Every author has a list of subscribers. Alternately every subscriber (user of the app), has a list of authors who they have subscribed to. - Reference: See https://www.open-tickr.net --- ### Management Apps Info #### M1. File Manager - Description: File Manager is a file manager app bundled with releases of MADS Platform. - Features: - It is a graphical interface to manage files (copy, move, open, delete, search, etc.). - It may or may not have a command-line equivalent. - Reference: - https://en.wikipedia.org/wiki/File_Manager_(Windows) - https://en.wikipedia.org/wiki/Comparison_of_file_managers - https://en.wikipedia.org/wiki/File_manager - https://en.droidwiki.org/wiki/File_manager --- #### M2. IoT Gateway Manager - Description: Gateway manager will help to manage credentials and attributes for gateways and sensors attached to it. - Features: - A user should be able to see a list of all the gateways added in the system under the gateways tab. - A user should be able to see a list of all the sensors for a gateway. - A gateway can have attributes related to it's `performance` which can be uploaded, there should be a provision to create a mapping between the `telemetery attributes` and `user defined attributes`. Since, the gateway also sends `sensor telemetery data` there should be a provision to map it to sensor attributes being defined by the user. - There should be a way to define alarms for device telemetery as well as sensor telemetery data. - There should be a way to define credentials with which device can upload data. In case the credentials are not found the device can not upload the data. - A device can be assigned to a particular site. - **Gateway Updates**: There are two different types of gateway updates: - Configuration update: This will update the basic configurations such as ping cycle, any actuation if it needs to be performed, telemetery data - Firmware updates: Will update the firmware of the device, have to look into the details of how this would be done. - Consensus based authentication using blockchain for advanced security while doing firmware updates. - The gateway can send data via different protocols - HTTP - MQTT - CoAP - LoraWan - The gateway can send data via different communication mediums - 2G, 4G, NbIOT,Sigfox (sim based) - Lora - WiFi --- #### M3. Roles Manager(User and Access Management) - Description: Provides a centralized control of access to different features in the application for multiple users. - Features: - **Users**: Allows the admin to perform following tasks. - Add users to the platform who can access, manage and observe different portions depending on the level of permissions. - The user gets an invitation on the email used while creating the account, and on acceptance, will be able to signin on the platform. - The system administrator has rights to decide access level of the user. The acess levels are defined with permissions. - Define a timeline for which access is allowed. - The system admin can also define the password policy for the user(length, capital letters, expiration date etc.) - Optionally the admin can choose to add Multi-Factor Authentication for users. - **User Groups**: User groups allow easier administration for users. The access level of users are determined by the permissions a system administrator gives. However, when the number of user increases then they can be grouped together and permissions can be set for the group itself, this removes the hassle of assigning it individually.! ![user_group](https://i.imgur.com/khbefH2.png) - **Policies**: It is used for assigning permissions to user, user groups and service access. A policy is made of different parts: - Resource: A resource refers to an entity and information associated with it. e.g. a gateway is a resource in the platform. - Actions: This refers to actions corresponding to the resource, which includes managing CRUD operations, handling data associated with the resources. - Conditions: A policy can contain different conditions which the permissioned has to satisfy. e.g. requests originating from a particular set of ip addresses, from particular zone etc. - Effect: This refers to the results of an action over a resource. ![policy](https://i.imgur.com/commUCD.png) Some pre-defined policies are defined by default, however the admin can create new policies using JSON schema. - **Service Access**: This is a feature to limit the level of access for a third party application. This is mostly for accessing data from the main application using REST APIs, gRPC or Pub/Sub mechanism. The access level is determined by adding permissions for the third party service. - **Auditing**: This allows the platform to monitor, log and audit actions performed by user. This allows better security analysis and preventive actions for security breaches. This expedites requirements for security compliances by independent auditors. Auditing also allows for automations related to security, incase of breach event it will automatically execute a bunch of pre-defined flow to take corrective action. --- #### M4. Entity Manager - Description: App to ... - Features: - Populates all entity (sites, blah, blah) of the organization in different views: - map view - tile view - list view --- #### M5. Tool Manager --- #### M6. Widget Manager - Description: All Dashboards are constructed using MADS widgets that are defined in Widget Manager. Each widget provides end-user functions such as data visualization, remote device control, alarms management and displaying static custom html content. - Features: - According to the provided features, each widget definition represents specific widget type. At the moment there are five widget types: - Latest values - Time-series - RPC (Control widget) - Alarm widget - Static - Each widget type has own specific datasource configuration and corresponding widget API. Each widget requires datasource for data visualization. Types of the available datasource depend on widget type of the widget: - *Target device* - this datasource type is used in RPC. Basically, you need to specify target device for RPC widget - *Alarm source* - this datasource type is used in Alarm widgets. This datasource requires source entity to display related alarms and corresponding alarm fields. - *Entity* - this datasource type is used in both time-series and latest values widgets. Basically, you need to specify target entity and timeseries key or attribute name. - *Function* - this datasource type is used in both time-series and latest values widgets for debug purposes. Basically, you are able to specify a javascript function that will emulate data from a device in order to tune visualization. - Reference: See https://thingsboard.io/docs/user-guide/ui/widget-library/ for features and ideas. --- ### Analytics Apps #### A1. Data Cruncher - Description: - App to choose one or more data source and perform logical operations on the data stream. - Data Cruncher is an interactive data analysis tool on the MADS Platform for exploring and mining data using pre-programed (or self-programmable) code snippet. There are 3 main components to the Data Cruncher app: 1. *Message* - any incoming event. It can be an incoming data from devices, device life-cycle event, REST API event, RPC request, etc. 2. *Rule Node* - a function (achieved through a code snippet) that is executed on an incoming message. There are many different Node types that can filter, transform or execute some action on incoming Message. 3. *Rule Chain* - nodes are connected with each other with relations, so the outbound message from rule node is sent to next connected rule nodes. - Industry specific data analysis can be complex and needs to be executed in a workflow. And the results often needs to be further analyzed or computed on. In most cases, the size of the data makes it quite hard and impractical to download results and perform further analysis on a local machine. Instead, Data Cruncher app brings the analytical tool to your data, integrating them within the MADS Platform. When you need to run a few very simple shell commands to explore data or perform a more complex analysis using Python or R, Data Cruncher will enable you to do it directly on the Platform using pre-programed (or self-programmable) code snippets called Rule Nodes, thus avoiding the time and resource consuming process of downloading data to the local machine. - Features: - Allows you to perform analyses directly where your data is. - Allows you to save outputs of analyses to your projects on the Platform. - Uses Python (JupyterLab or RStudio) as the computational environment, so an user can edit the code or add new code if they wish to. - Is tightly integrated with the MADS Platform. --- #### A2. Trend Predictor --- #### A3. Usage Analyser --- #### A4. AI Recommender --- ### Security Apps Info #### S1. MADS Security #### S2. Login Tracker #### S3. Network Analyser #### S4. Secure Exporter #### S5. App Locker #### S6. Password Manager ### General Apps Info #### G1. Calendar --- #### G2. Calculator - Description: App to perform basic, advanced, or scientific calculations and share results. - Features: - In the Calculator app, you can perform basic arithmetic calculations with the standard calculator. Or use the scientific calculator for exponential, logarithmic, and trigonometric functions. - Basic & Scientific Calculator - History Tape: Save, Copy & Share Calculations - Copy a calculation result: Touch and hold the calculation result in the display, tap Copy, then paste the result somewhere else, such as a note or message. - Delete the last digit: If you make a mistake when you enter a number, swipe left or right on the display at the top. - Clear the display: Tap the Clear [C] key to delete the last entry, or tap the All Clear [AC] key to delete all entries. - Calculator Themes - Light theme - Dark theme --- #### G3. Chat App --- ## Architecture ## Random Thoughts <!-- Random thought below --> ### Features of Basic Apps - Device Mangement - Firmware upate over air - Dashboards - Alarms and Notifications - Ticket generation - Tool management - Visualizations - Digital Twin - Entity Management - Role Based Access Control - Widgeting Library ## More - Rule Engine - Security - Analytics - Plugins - ITUS Secure Edge ### Important design related links - [how facebook apps are handled][link_1] - [Streaming data for and from different services][link_2] - [link_1]: https://developers.facebook.com/docs/apps/ [link_2]: https://www.confluent.io/blog/data-dichotomy-rethinking-the-way-we-treat-data-and-services/ [dig_draw_io]: https://www.draw.io/ **NEED TO DISCUSS WITH JADON** Ayoush currently 76:26:34 On 01/21/2020 12:58PM - 1:45PM and 1:28PM - 2:11PM has overlap. On 01/22/2020 8:01AM - 10:03AM and 8:10PM - 11:11AM has overlap. On 01/31/2020 6:35AM - 11:07AM and 10:48PM - 2:30PM has overlap. ——