# Operational View For Neo4j
## Justification

The log files are really important for developers when they want to resolve some bugs or when they want to understand the system that they are working on. Therefore, it would be helpful for developers to include a look-up table in this AD that matches different log levels with their triggering events and purposes.

Furthermore, guidance regarding how to properly configure Neo4j should also be provided since the official documentation regarding configuration is not straightforward to understand. Because of the requirements above, it is reasonable to include an operational view in the AD.

## Scope
The scope of this view is limited to Neo4j community edition version 4.4
* Features not included in this community edition are excluded from this view. The target audience for this view is developers.

### Logging Overview
Four different logging levels are provided:
* info
* warn
* debug
* error

Logging levels indicate the nature of the messages being logged. The format of each log and the location to which the logs are stored are configurable.

**Possible Formats for Log:** JSON or plain text.

**How logs are stored:** Log entries at info, error and warn level are stored in one file, and log entries at debug level are stored in another file.

**Each log entry has the following fields**
* time
* level
* category
* message
* stacktrace (if an exception is raised).

**One interesting characteristic of logging in Neo4j:** If the operation triggered by the user request is complex and has different phases, more information is logged to indicate if a certain phase of the operation is successful; for example, before stopping a database, Neo4j needs to apply all the queued transactions, and whether all the queued transactions have been successfully applied is also logged.

### Logging Levels and Explanation ([Logging Module Source Code](https://github.com/neo4j/neo4j/tree/4.4/community/logging/src/main/java/org/neo4j/logging/log4j))

| Logging Level | Purpose | Example Triggering Events | Example Messages Logged at this level |
|---------------|------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Info | Report the status of Neo4j's operations | The starting and completion of Neo4j's operations. | "Starting web server" <br/> <br/> "Web server started." |
| Warn | Report errors that are not severe but need users' attention | Users fail to configure the system properly, and such failure has created errors. <br/> <br/> Users ask the system to do illegal operations. | "Old community lock manager is configured to be used. Ignoring and fallback to default lock manager." <br/> <br/> "No transaction logs were detected, but recovery was forced by user." |
| Debug | Report more technical information regarding Neo4j's operations | Sub-operations within a Neo4j's operation, and that sub-operations usually involve technical details not relevant to the users who do not have a technical background | "Shutting down thread pool" <br/> <br/> "Thread pool shut down" |
| Error | Report severe errors that need to be addressed immediately | Errors that might damage data integrity or prevent the Neo4j server from running. | " Failed to roll back transaction.{exception}" <br/> <br/> "Transaction logs are missing and recovery is not possible." |

### The Definition of the terms used in the table above

**Operation**: operation is an abstraction that can refer to one method call or a series of method calls that serves the same or similar functionalities.

## Configuration Management Model
### Note
There are hundreds of parameters that can be configured by the users of Neo4j. Therefore, only the parameters that are useful for the developers of Neo4j are included into this model. You can view all the parameters by clicking this [link](https://neo4j.com/docs/operations-manual/current/reference/configuration-settings/)

### Configuration Groups

| Configuration Group Name | Example Parameters in the Group | Purpose of This Configuration Group |
|-------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Memory parameters | dbms.memory.heap.initial_size=512m <br/> <br/> dbms.memory.heap.max_size=512m <br/> <br/> dbms.memory.transaction.global_max_size=256m <br/> <br/> dbms.memory.transaction.max_size=16m <br/> <br/> dbms.tx_state.memory_allocation=ON_HEAP | 1. Specify how much memory will be used during runtime for Neo4j related activities.<br/> 2. Determine the amount of memory that all the running transaction can consume, the amount of memory that a single transaction can consume, and transaction state location (for example on heap) etc. |
| Network connection parameters | dbms.connector.bolt.enabled=true<br/> <br/> dbms.connector.bolt.tls_level=DISABLED<br/> <br/> dbms.connector.bolt.listen_address=:7687<br/> <br/> dbms.connector.bolt.advertised_address=:7687<br/> <br/> dbms.connector.http.enabled=true<br/> <br/> dbms.connector.https.enabled=false | 1. Specify what types of network connection is allowed and the listen addresses of these connections |
| SSL policy parameters | dbms.ssl.policy.https.enabled=true<br/> <br/> dbms.ssl.policy.https.base_directory=certificates/https<br/> <br/> dbms.ssl.policy.https.private_key=private.key<br/> <br/> dbms.ssl.policy.https.public_certificate=public.crt<br/> <br/> dbms.ssl.policy.https.client_auth=NONE<br/> <br/> dbms.ssl.policy.bolt.enabled=true<br/> <br/> dbms.ssl.policy.cluster.enabled=true<br/> <br/> dbms.ssl.policy.backup.enabled=true<br/> <br/> dbms.ssl.policy.backup.base_directory=certificates/backup | 1. Configure the secure sockets layer for Bolt connection, HTTPs connection, backup databases and cluster. <br/> <br/> 2. Specify many security related properties for network connections. These safety related properties include but not limited to if authentication is needed or if a private key is needed. |
| Logging parameters | dbms.logs.http.enabled=true<br/> <br/> dbms.logs.http.rotation.keep_number=5<br/> <br/> dbms.logs.http.rotation.size=20m<br/> <br/> dbms.logs.gc.enabled=true | 1. Configure the HTTP logs and garbage collection logs. Specify these logs' size, amount and threshold for rotations.<br/> <br/> 2. Specify if the HTTP logs and the garbage collection logs are enabled. |
| JVM parameters | dbms.jvm.additional=-XX:+UseG1GC<br/> <br/> dbms.jvm.additional=-XX:-OmitStackTraceInFastThrow<br/> <br/> dbms.jvm.additional=-Djdk.nio.maxCachedBufferSize=262144<br/> <br/> dbms.jvm.additional=-XX:+DisableExplicitGC | 1. Configure the properties of JVM. These properties include but not limited to if stacktrace is omitted in fast throw, which type of garbage collector is used and if explicit garbage collection is enabled. |
| Transaction parameters | dbms.transaction.timeout=10s<br/> <br/> dbms.lock.acquisition.timeout=10s | 1. Specify how long a transaction can be executed and how long a transaction can hold a lock. |

### Configuration Dependencies
For any dependency relationship A->B, if B changes, A must change.

#### Dependency Graph 1

#### Explanation For Dependency Graph 1:
(Intergroup dependencies) If a connection is disabled in the *Network Connection Parameters Configuration Group*, then the parameters related to such connection in *SSL Policy Parameters Configuration Group* also need to be disabled. For instance, if dbms.connector.https.enabled = false, then the corresponding parameters in the *SSL Policy Parameters Configuration Group* also need to be disabled. To disable these parameters, users can just comment out these parameters.
#### Dependency Graph 2

#### Explanation For Dependency Graph 2:
(Intergroup dependencies) If a user decreases the size of the heap memory allowed for Neo4j activities by changing the corresponding parameters in the *Memory Parameters Configuration Group*, then the user also needs to lower the size of cached JDK buffers by changing the Djdk.nio.maxCachedBufferSize parameter *in the JVM Parameters Configuration Group*.

#### Dependency Graph 3

#### Explanation For Dependency Graph 3:
(Intragroup dependencies) If a user decreases the size of the heap memory allowed for Neo4j activities, then the user also needs to decrease the amount of memory used by transactions. All the relevant parameters for this dependency are in the *Memory Parameters Configuration Group*

### Configuration Sets
**High performance:** allocate as much amount of memory as possible to Neo4j and enable all possible network connections to Neo4j. This configuration can be applied when the developers expect a high volume of requests to Neo4j.

**Low performance:** allocate as little amount of memory as possible to Neo4j and only enable necessary network connections to Neo4j. This configuration can be applied when the hardware is overheated.

**Safety first:** only allow bolt connection and disable all other network connections. At the same time, Neo4j requires authentication and a private key for bolt connection. This configuration can be applied when the developers suspect that the safety of the network is compromised.
### Configuration Change Strategy
All the parameters are located in the neo4j.conf file. Theoretically, users can configure these parameters however they would like by editing the neo4j.conf file, and Neo4j will just load the file when it starts. The default location of the file is illustrated by this [documentation](https://neo4j.com/docs/operations-manual/current/configuration/file-locations/#table-file-locations)

Nevertheless, it is encouraged to set these configuration parameters according to the condition of the hardware hosting Neo4j. Also, it is encouraged to change one group of parameters at a time, so that it is easier to isolate which parameter change causes system failure or suboptimal performance during runtime.

It is also possible to configure these parameters while Neo4j is running. Users need to type the corresponding commands into the cypher shell. For instance, user can type the following command into the cypher shell to set the transaction parameter

dbms.transaction.timeout=10s

However, it is not recommended to change some parameters while Neo4j is running. For instance, it is not recommended to change the memory parameters while Neo4j is still running since that change might lead to unexpected behavior or system failure.

<!-- ## Performance and Scalability Perspective
Since the community edition of Neo4j does not implement any useful metrics, the developers could only infer the performance of Neo4j using the log files, and this inference of performance only indicates if Neo4j has encountered any errors so far. Therefore, it is not feasible for us to extensively discuss the Performance and Scalability perspective in the operational view.

[According to the official documentation of Neo4j, only the enterprise edition has implemented metrics.](https://neo4j.com/docs/operations-manual/current/monitoring/metrics/) -->