Why SystemD?

systemd is capable of starting services in parallel, which speeds up the boot process. It is also capable of handling hot-plug devices; such as starting up services on demand at the occurrence of an event (like upstart).

The process of managing service daemons in System V is a little bit involved. This is so because scripts get detached from the daemons they spawn upon runtime. Some daemons spawn 3rd party processes that get orphaned and re-parented by PID 1. Some processes even re-fork and rename themselves which makes process management difficult.

systemd makes daemon/service management easier by placing their processes in a control group (cgroup).

Here, spawned processes are made members of their parent cgroup (named after the service). These processes are not capable of leaving their cgroup until their parent terminates them. This prevents the creation of zombie processes. It also eases the management (running kill -9 as an example) of related processes since they belong to one cgroup.

systemd also provides a tool called “systemctl”. man 1 systemctl says systemctl may be used to introspect and control the state of the “systemd” system and service manager.

This tool can be used to start and stop services including other things as we will soon see. System V init implementations manage system state by the use of shell scripts. systemd on the flip-side takes a different approach towards writing configuration files. It does this by expressing commands in a declarative format. This makes the job of administering Linux systems easier by making uniform all aspects of system administration.

Dependency Management

System V init implementations don’t have dependency management issues because things are kept simple at the expense of flexibility. systemd developers devised a method for handling dependencies in such a way that the failure of one dependency doesn’t put the entire system in limbo.
There is a handful of dependency types that gives systemd the ability to flexibly manage dependencies which we will cover later.

SystemD Deep Dive

Systemd operates and manages things called units. Units have configuration files called unit files

Units are a group of standardized representations of the system resources. The system's copy of unit files are generally kept in the /lib/systemd/system directory. When a software installs a unit files on the system, it is placed there by default.

The files stored in this place are what systemd operates on. If you wish to change or modify a functionality of a unit, do not change the files stored in this directory. Instead, make a copy and store it in /etc/systemd/system directory. The system will take the unit files in this directory in precedence over /lib/systemd/system

If you wish to just change one directive in the unit file (and not replace the entire unit file), you can create a folder with the unit file's name, but ending with a .d and make a file ending with .conf. i.e. to change one directive in dummy.service, create a folder called dummy.service.d and place a .conf file inside.

systemd process also uses /run/systemd/system for dynamic creation of unit files created at runtime. This directory can be used to change the system's unit behavior for the duration of the session. All changes made in this directory will be lost when the system is rebooted.

In summary, there are three directories that systemd uses (in order of system precedence):

• /etc/systemd/system
• /run/systemd/system
• /lib/systemd/system

Targets (Runlevels)

• poweroff.target maps to runlevel 0 - This target is used for system shutdown.
• rescue.target maps to runlevel 1 - This target is used to enter single-user mode.
• multi-user.target maps to runlevels 2-4 This target is used to enter the standard multi-user mode without a graphical interface.
• graphical.target maps to runlevel 5 - This target is used for multi-user mode with networking and a display manager.
• reboot.target maps to runlevel 6 - This target is used to reboot the system.

Types of units

The suffix of a unit file tells us what kind of service it provides. Below shows the collection of types of units

Suffix	Function
.service	This describes how to manage a service or an application on the system. This includes starting and stopping the service, when it should restart and the dependencies
.socket	This describes a network or an IPC socket
.device	Units that end with this suffix needs management by udev or sysfs
.mount	This defines a mount point on the system. They are names after the mount point path. Entries in /etc/fstab have units created automatically
.automount	This defines units that mount automatically. They must have a matching .mount unit
.swap	This describes the swap space on the system
.target	This provides synchronization points for other units when booting up or changing states. They can also be used to change the state of the system
.path	This defines a path that can be used for path-based activation
.timer	This defines a timer that will be managed by systemd
.snapshot	This unit is created when the command systemctl snapshot is executed
.slice	These unit files are associated with Linux Control Group nodes, allowing resources to be restricted or assigned to any process associated with that slice
.scope	These units are created automatically by systemd from information received from bus interfaces

Different unit types work hand in hand to perform some functionality

Link to the list of Special Units in SystemD

General Syntax

General Characteristics of Unit Files
Section names are well-defined and case-sensitive. So, the section [Unit] will not be interpreted correctly if it is spelled like [UNIT]. If you need to add non-standard sections to be parsed by applications other than systemd, you can add a X- prefix to the section name. Within these sections, unit behavior and metadata is defined through the use of simple directives using a key-value format with assignment indicated by an equal sign, like this:

[Section]
Directive1=value
Directive2=value
. . .

The default_value can be eliminated in an override file by referencing Directive1 without a value, like this:

Directive1=

Template and Instance Unit Names

Template unit files can be identified because they contain an @ symbol after the base unit name and before the unit type suffix. A template unit file name may look like this:

example@.service
When an instance is created from a template, an instance identifier is placed between the @ symbol and the period signifying the start of the unit type. For example, the above template unit file could be used to create an instance unit that looks like this:

[e-mail]
An instance file is usually created as a symbolic link to the template file, with the link name including the instance identifier. In this way, multiple links with unique identifiers can point back to a single template file. When managing an instance unit, systemd will look for a file with the exact instance name you specify on the command line to use. If it cannot find one, it will look for an associated template file.

Template Specifiers

The power of template unit files is mainly seen through its ability to dynamically substitute appropriate information within the unit definition according to the operating environment. This is done by setting the directives in the template file as normal, but replacing certain values or parts of values with variable specifiers.

The following are some of the more common specifiers will be replaced when an instance unit is interpreted with the relevant information:

%n: Anywhere where this appears in a template file, the full resulting unit name will be inserted.
%N: This is the same as the above, but any escaping, such as those present in file path patterns, will be reversed.
%p: This references the unit name prefix. This is the portion of the unit name that comes before the @ symbol.
%P: This is the same as above, but with any escaping reversed.
%i: This references the instance name, which is the identifier following the @ in the instance unit. This is one of the most commonly used specifiers because it will be guaranteed to be dynamic. The use of this identifier encourages the use of configuration significant identifiers. For example, the port that the service will be run at can be used as the instance identifier and the template can use this specifier to set up the port specification.
%I: This specifier is the same as the above, but with any escaping reversed.
%f: This will be replaced with the unescaped instance name or the prefix name, prepended with a /.
%c: This will indicate the control group of the unit, with the standard parent hierarchy of /sys/fs/cgroup/ssytemd/ removed.
%u: The name of the user configured to run the unit.
%U: The same as above, but as a numeric UID instead of name.
%H: The host name of the system that is running the unit.
%%: This is used to insert a literal percentage sign.

By using the above identifiers in a template file, systemd will fill in the correct values when interpreting the template to create an instance unit.

Unit Section Directives [ Dependency Types ]

Directives in unit files and are found with the Unit section of the unit:

• Description: This directive can be used to describe the name and basic functionality of the unit. It is returned by various systemd tools, so it is good to set this to something short, specific, and informative.

• Documentation: This directive provides a location for a list of URIs for documentation. These can be either internally available man pages or web accessible URLs. The systemctl status command will expose this information, allowing for easy discoverability.

• Requires: A unit file with a ‘Requires’ dependency means units within this directive must be successfully activated when an activation signal is, otherwise the current unit will fail. Dependencies with this directive are strict.

• Wants: This is a less strict version of Requires. The Wants dependencies are activated upon activation of the containing unit file. This unit will be activated regardless of if the Wants dependencies fail.

• Conflicts: Units outlined within this dependency are automatically stopped upon activation of the containing unit file.

• Requisite: Units within this dependency must be already active before the unit is activated, otherwise systemd fails on the activation of the unit with the dependency.

• Before: This dependency directive explicitly declares ordering. This is important in situations when you’d want to start a unit before another. For this instance, the current unit is activated before the unit under the Before directive.

• After: This dependency is also used for ordering dependency activation. It’s the opposite of Before. The unit under this directive is/are activated before activation of the current unit.

• Conditional dependencies: This is used to evaluate a truthful or false value of units with it. The current unit won’t be activated if the conditional directive evaluates to false.

• BindsTo: This directive is similar to Requires=, but also causes the current unit to stop when the associated unit terminates.

• Assert: Similar to the directives that start with Condition, these directives check for different aspects of the running environment to decide whether the unit should activate. However, unlike the Condition directives, a negative result causes a failure with this directive.

Install Section Directives

• WantedBy: The WantedBy= directive is the most common way to specify how a unit should be enabled. This directive allows you to specify a dependency relationship in a similar way to the Wants= directive does in the [Unit] section. The difference is that this directive is included in the ancillary unit allowing the primary unit listed to remain relatively clean. When a unit with this directive is enabled, a directory will be created within /etc/systemd/system named after the specified unit with .wants appended to the end. Within this, a symbolic link to the current unit will be created, creating the dependency. For instance, if the current unit has WantedBy=multi-user.target, a directory called multi-user.target.wants will be created within /etc/systemd/system (if not already available) and a symbolic link to the current unit will be placed within. Disabling this unit removes the link and removes the dependency relationship.

• RequiredBy: This directive is very similar to the WantedBy= directive, but instead specifies a required dependency that will cause the activation to fail if not met. When enabled, a unit with this directive will create a directory ending with .requires.
  Alias: This directive allows the unit to be enabled under another name as well. Among other uses, this allows multiple providers of a function to be available, so that related units can look for any provider of the common aliased name.

• Also: This directive allows units to be enabled or disabled as a set. Supporting units that should always be available when this unit is active can be listed here. They will be managed as a group for installation tasks.

• DefaultInstance: For template units which can produce unit instances with unpredictable names, this can be used as a fallback value for the name if an appropriate name is not provided.

Few unit type specific Directives

The device, target, snapshot, and scope unit types have no unit-specific directives

The [Service] Section

• Type: directive can be one of the following:
  • simple: The main process of the service is specified in the start line. This is the default if the Type= and Busname= directives are not set, but the ExecStart= is set. Any communication should be handled outside of the unit through a second unit of the appropriate type (like through a .socket unit if this unit must communicate using sockets).
  • forking: This service type is used when the service forks a child process, exiting the parent process almost immediately. This tells systemd that the process is still running even though the parent exited.
  • oneshot: This type indicates that the process will be short-lived and that systemd should wait for the process to exit before continuing on with other units. This is the default Type= and ExecStart= are not set. It is used for one-off tasks.
  • dbus: This indicates that unit will take a name on the D-Bus bus. When this happens, systemd will continue to process the next unit.
  • notify: This indicates that the service will issue a notification when it has finished starting up. The systemd process will wait for this to happen before proceeding to other units.
  • idle: This indicates that the service will not be run until all jobs are dispatched.

Some handy commands

systemctl get-default
systemctl isolate runlevel5.target
systemctl isolate multi-user.target
systemctl set-default graphical.target

systemctl status/start/stop/restart <servicename>
systemctl reload <servicename>
systemctl reload-or-restart <servicename>
systemctl enable/disable <servicename>
systemctl mask/unmask <servicename>

systemctl list-units –all
systemctl list-units –state=active –type=service

systemctl list-dependencies <servicename>

systemctl show <servicename>
systemctl show -p <propery_name> <servicename>

systemctl cat <servicename>
systemctl edit <servicename>
systemctl edit –full <servicename>

To run on remote server add -H
systemctl status <servicename> -H root@remotehost