# HTB academy - Linux Fundamental
###### tags: `HTB`
## Philosophy
| Principle | Decription |
| -------- | -------- |
| Everything is a file | All configuration files for the various services running on the Linux operating system are stored in one or more text files. |
| Small, single-purpose programs | Linux offers many different tools that we will work with, which can be combined to work together. |
| Ability to chain programs together to perform complex tasks | The integration and combination of different tools enable us to carry out many large and complex tasks, such as processing or filtering specific data results. |
| Avoid captive user interfaces | Linux is designed to work mainly with the shell (or terminal), which gives the user greater control over the operating system. |
| Configuration data stored in a text file | An example of such a file is the /etc/passwd file, which stores all users registered on the system. |
## Architecture
| Layer | Description |
|-------|-------------|
| System Utility | Makes available to the user all of the operating system's functionality. |
| Shell | A command-line interface (CLI), also known as a shell that a user can enter commands into to execute the kernel's functions. |
| Kernel | The core of the Linux operating system whose function is to virtualize and control common computer hardware resources like CPU, allocated memory, accessed data, and others. The kernel gives each process its own virtual resources and prevents/mitigates conflicts between different processes. |
| Hardware | Peripheral devices such as the system's RAM, hard drive, CPU, and others. |
## File System
| Path | Description |
|------|-------------|
|/| The top-level directory is the root filesystem and contains all of the files required to boot the operating system before other filesystems are mounted as well as the files required to boot the other filesystems. After boot, all of the other filesystems are mounted at standard mount points as subdirectories of the root.|
|/bin |Contains essential command binaries.|
|/boot | Consists of the static bootloader, kernel executable, and files required to boot the Linux OS.|
|/dev |Contains device files to facilitate access to every hardware device attached to the system.|
|/etc |Local system configuration files. Configuration files for installed applications may be saved here as well.|
|/home | Each user on the system has a subdirectory here for storage.|
|/lib |Shared library files that are required for system boot.|
|/media| External removable media devices such as USB drives are mounted here.|
|/mnt |Temporary mount point for regular filesystems.|
|/opt |Optional files such as third-party tools can be saved here.|
|/root | The home directory for the root user.|
|/sbin | This directory contains executables used for system administration (binary system files).|
|/tmp |The operating system and many programs use this directory to store temporary files. This directory is generally cleared upon system boot and may be deleted at other times without any warning.|
|/usr |Contains executables, libraries, man files, etc.|
|/var |This directory contains variable data files such as log files, email in-boxes, web application related files, cron files, and more.|
## command for System Infomation
|Command | Description|
|-----------|--------------|
|whoami | Displays current username.|
|id | Returns users identity.|
|hostname | Sets or prints the name of current host system.|
|uname | Prints operating system name.|
|pwd | Returns working directory name.|
|ifconfig | The ifconfig utility is used to assign or to view an address to a |network interface and/or configure network interface parameters.|
|ip | Ip is a utility to show or manipulate routing, network devices, |interfaces |and tunnels.|
|netstat | Shows network status.|
|ss | Another utility to investigate sockets.|
|ps | Shows process status.|
|who | Displays who is logged in.|
|env | Prints environment or sets and executes command.|
|lsblk | Lists block devices.|
|lsusb | Lists USB devices
|lsof | Lists opened files.|
|lspci | Lists PCI devices.|
## PKG Management
## Service and Process Management
In general, there are two types of services:
* internal, the relevant services that are required at system startup, which for example, perform hardware-related tasks
* services that are installed by the user, which usually include all server services. Such services run in the background without any user interaction. These are also called daemons and are identified by the letter 'd' at the end of the program name, for example, sshd or systemd.
Most Linux distributions have now switched to systemd from system V. This daemon is an Init process started first and thus has the process ID (PID) 1. This daemon monitors and takes care of the orderly starting and stopping of other services.
All processes have an assigned PID that can be viewed under /proc/ with the corresponding number. Such a process can have a parent process ID (PPID), known as the child process.
### Systemctl
The main command used to introspect and control systemd is **systemctl**. Some of its uses are examining the system state and managing the system and services. See systemctl(1) for more details.
### Background Proccess
Sometimes it will be necessary to put the scan or process we just started in the background to continue using the current session to interact with the system or start other processes. As we have already seen, we can do this with the shortcut [Ctrl] + Z. As mentioned above, we send the SIGTSTP signal to the kernel, which suspends the process.
### Foreground Proccess
After that, we can use the jobs command to list all background processes. Backgrounded processes do not require user interaction, and we can use the same shell session without waiting until the process finishes first. Once the scan or process finishes its work, we will get notified by the terminal that the process is finished.
### Execute Multiple Commands
There are three possibilities to run several commands, one after the other. These are separated by:
* Semicolon (;)
* Double ampersand characters (&&)
* Pipes (|)
## Web Services
Another essential component is the communication with the web servers. There are many different ways to set up web servers on Linux operating systems. One of the most used and widespread web servers, besides IIS and Nginx, is **Apache**. For an Apache web server, we can use appropriate modules, which can encrypt the communication between browser and web server (mod_ssl), use as a proxy server (mod_proxy), or perform complex manipulations of HTTP header data (mod_headers) and URLs (mod_rewrite).
Apache offers the possibility to create web pages dynamically using server-side scripting languages. Commonly used scripting languages are PHP, Perl, or Ruby. Other languages are Python, JavaScript, Lua, and .NET, which can be used for this.
### Apache
### wget
An alternative to curl is the tool wget. With this tool, we can download files from FTP or HTTP servers directly from the terminal and serves as a good download manager. If we use wget in the same way, the difference to curl is that the website content is downloaded and stored locally.
### cURL
cURL is a tool that allows us to transfer files from the shell over protocols like HTTP, HTTPS, FTP, SFTP, FTPS, or SCP. This tool gives us the possibility to control and test websites remotely. Besides the remote servers' content, we can also view individual requests to look at the client's and server's communication. Usually, cURL is already installed on most Linux systems.
### npm, Node Package Manager
npm is a package manager for the JavaScript programming language.
My understanding is npm is knida like git, apt or dkpg, as they are all package manager tools.
npm is the default package manager for the JavaScript runtime environment Node.js. It consists of a command line client, also called npm, and an online database of public and paid-for private packages, called the npm registry. The registry is accessed via the client, and the available packages can be browsed and searched via the npm website. The package manager and the registry are managed by npm, Inc.
### 3 way to start http-server
Below shows 3 very quick and different ways that you can start up a HTTP Web Server from the terminal on your local machine:
* PHP
* Python
* NodeJs
## Navigation & Files
### which
One of the common tools is **which**. This tool **returns the path to the file or link that should be executed**. This allows us to determine if specific programs, like cURL, netcat, wget, python, gcc, are available on the operating system. Let us use it to search for Python in our interactive instance.
``` bash=
HungJu@htb[/htb]$ which python
/usr/bin/python
```
If the program we search for does not exist, no results will be displayed.
### find
Another handy tool is find. Besides the function to find files and folders, this tool also contains the function to filter the results. We can use filter parameters like the size of the file or the date. We can also specify if we only search for files or folders.
``` bash=
HungJu@htb[/htb]$ find <location> <options>
```
|Option | Description|
|-------|--------------|
|-type f | Hereby, we define the type of the searched object. In this case, 'f' stands for 'file'.
|-name \*.conf | With '-name', we indicate the name of the file we are looking for. The asterisk (\*) stands for 'all' files with the '.conf' extension.|
|-user root | This option filters all files whose owner is the root user.|
|-size +20k | We can then filter all the located files and specify that we only want to see the files that are larger than 20 KiB.|
|-newermt 2020-03-03 | With this option, we set the date. Only files newer than the specified date will be presented.|
|-exec ls -al {} \; | This option executes the specified command, using the curly brackets as placeholders for each result. The backslash escapes the next character from being interpreted by the shell because otherwise, the semicolon would terminate the command and not reach the redirection.|
|2>/dev/null | This is a STDERR redirection to the 'null device', which we will come back to in the next section. This redirection ensures that no errors are displayed in the terminal. This redirection must not be an option of the 'find' command.
### locate
It will take much time to search through the whole system for our files and directories to perform many different searches. The command locate offers us a quicker way to search through the system. In contrast to the find command, **locate works with a local database that contains all information about existing files and folders**. We can update this database with the following command.
You will find that this search produces results **much faster** than using find.
```bash=
HungJu@htb[/htb]$ locate *.conf
/etc/GeoIP.conf
/etc/NetworkManager/NetworkManager.conf
/etc/UPower/UPower.conf
/etc/adduser.conf
<SNIP>
```
However, this **tool does not have as many filter options** that we can use. So it is always worth considering whether we can use the locate command or instead use the find command. It always depends on what we are looking for.
## Filter Contents
To read files, we do not necessarily have to use an editor for that. There are two tools called more and less, which are very identical.
### more
```bash=
HungJu@htb[/htb]$ more /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
<SNIP>
--More--
```
### less
```bash=
HungJu@htb[/htb]$ less /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
<SNIP>
:
```
### head
Sometimes we will only be interested in specific issues either at the beginning of the file or the end. If we only want to get the first lines of the file, we can use the tool head. **By default, head prints the first ten lines** of the given file or input, if not specified otherwise.
```bash=
HungJu@htb[/htb]$ head /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
```
### tail
Opposite to head, which returns the last ten lines.
```bash=
HungJu@htb[/htb]$ tail /etc/passwd
miredo:x:115:65534::/var/run/miredo:/usr/sbin/nologin
usbmux:x:116:46:usbmux daemon,,,:/var/lib/usbmux:/usr/sbin/nologin
rtkit:x:117:119:RealtimeKit,,,:/proc:/usr/sbin/nologin
nm-openvpn:x:118:120:NetworkManager OpenVPN,,,:/var/lib/openvpn/chroot:/usr/sbin/nologin
nm-openconnect:x:119:121:NetworkManager OpenConnect plugin,,,:/var/lib/NetworkManager:/usr/sbin/nologin
pulse:x:120:122:PulseAudio daemon,,,:/var/run/pulse:/usr/sbin/nologin
beef-xss:x:121:124::/var/lib/beef-xss:/usr/sbin/nologin
lightdm:x:122:125:Light Display Manager:/var/lib/lightdm:/bin/false
do-agent:x:998:998::/home/do-agent:/bin/false
user6:x:1000:1000:,,,:/home/user6:/bin/bash
```
### sort
Depending on which results and files are dealt with, they are rarely sorted. Often it is necessary **to sort the desired results alphabetically or numerically to get a better overview**. For this, we can use a tool called sort.
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | sort
_apt:x:104:65534::/nonexistent:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
cry0l1t3:x:1001:1001::/home/cry0l1t3:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
dnsmasq:x:107:65534:dnsmasq,,,:/var/lib/misc:/usr/sbin/nologin
dovecot:x:114:117:Dovecot mail server,,,:/usr/lib/dovecot:/usr/sbin/nologin
dovenull:x:115:118:Dovecot login user,,,:/nonexistent:/usr/sbin/nologin
ftp:x:113:65534::/srv/ftp:/usr/sbin/nologin
games:x:5:60:games:/usr/games:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
htb-student:x:1002:1002::/home/htb-student:/bin/bash
<SNIP>
```
### grep
More often, we will only search for specific results that contain patterns we have defined. One of the most used tools for this is grep, which offers many different features.
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | grep "/bin/bash"
root:x:0:0:root:/root:/bin/bash
mrb3n:x:1000:1000:mrb3n:/home/mrb3n:/bin/bash
cry0l1t3:x:1001:1001::/home/cry0l1t3:/bin/bash
htb-student:x:1002:1002::/home/htb-student:/bin/bash
```
### cut
Specific results with different characters may be separated as delimiters. Here it is handy to know how to remove specific delimiters and show the words on a line in a specified position. One of the tools that can be used for this is cut. Therefore we use the option "-d" and set the delimiter to the colon character (:) and define with the option "-f" the position in the line we want to output.
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | grep -v "false\|nologin" | cut -d":" -f1
root
sync
mrb3n
cry0l1t3
htb-student
```
### tr
Another possibility **to replace certain characters** from a line with characters defined by us is the tool tr. As the first option, we define which character we want to replace, and as a second option, we define the character we want to replace it with. In the next example, we replace the colon character with space.
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | grep -v "false\|nologin" | tr ":" " "
root x 0 0 root /root /bin/bash
sync x 4 65534 sync /bin /bin/sync
mrb3n x 1000 1000 mrb3n /home/mrb3n /bin/bash
cry0l1t3 x 1001 1001 /home/cry0l1t3 /bin/bash
htb-student x 1002 1002 /home/htb-student /bin/bash
```
### column
Since such results can often have an unclear representation, the tool column is well suited to **display such results in tabular form using the "-t"**.
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | grep -v "false\|nologin" | tr ":" " " | column -t
root x 0 0 root /root /bin/bash
sync x 4 65534 sync /bin /bin/sync
mrb3n x 1000 1000 mrb3n /home/mrb3n /bin/bash
cry0l1t3 x 1001 1001 /home/cry0l1t3 /bin/bash
htb-student x 1002 1002 /home/htb-student /bin/bash
```
### awk
Awk **is a scripting language** used for manipulating data and generating reports. **Mostly used for pattern scanning** and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then performs the associated actions.
As we may have noticed, the user "postgres" has one row too many. To keep it as simple as possible to sort out such results, the (g)awk programming is beneficial, which allows us to display the first (\$1) and last (\$NF) result of the line.
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | grep -v "false\|nologin" | tr ":" " " | awk '{print $1, $NF}'
root /bin/bash
sync /bin/sync
mrb3n /bin/bash
cry0l1t3 /bin/bash
htb-student /bin/bash
```
### sed
This is the **stream editor** called sed. One of the **most common uses of this is substituting text**. Here, sed looks for patterns we have defined in the form of **regular expressions** (regex) and replaces them with another pattern that we have also defined. Let us stick to the last results and say we want to replace the word "bin" with "HTB."
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | grep -v "false\|nologin" | tr ":" " " | awk '{print $1, $NF}' | sed 's/bin/HTB/g'
root /HTB/bash
sync /HTB/sync
mrb3n /HTB/bash
cry0l1t3 /HTB/bash
htb-student /HTB/bash
```
### wc
To avoid counting the lines or characters manually, we can use the tool wc. With the "-l" option, we specify that only the lines are counted.
```bash=
HungJu@htb[/htb]$ cat /etc/passwd | grep -v "false\|nologin" | tr ":" " " | awk '{print $1, $NF}' | wc -l
5
```
## Linux Security
All computer systems have an inherent risk of intrusion. Some present more of a risk than others, such as an internet-facing web server hosting multiple complex web applications. Linux systems are also less prone to viruses that affect Windows operating systems and do not present as large an attack surface as Active Directory domain-joined hosts. Regardless, it is essential to have certain fundamentals in place to secure any Linux system.
One of the Linux operating systems' most important security measures is **keeping the OS and installed packages up to date**.
If **firewall rules** are not appropriately set at the network level, we can use the Linux firewall and/or iptables to restrict traffic into/out of the host.
If **SSH** is open on the server, the configuration should be set up to disallow password login and disallow the root user from logging in via SSH. It is also important to avoid logging into and administering the system as the root user whenever possible and adequately managing access control. Users' access should be determined based on the principle of least privilege. For example, if a user needs to run a command as root, then that command should be specified in the sudoers configuration instead of giving them full sudo rights. Another common protection mechanism that can be used is fail2ban. This tool counts the number of failed login attempts, and if a user has reached the maximum number, the host that tried to connect will be handled as configured.
It is also important to periodically audit the system to ensure that issues do not exist that could facilitate **privilege escalation**, such as an out-of-date kernel, user permission issues, world-writable files, and misconfigured cron jobs, or misconfigured services. Many administrators forget about the possibility that some kernel versions have to be updated manually.
Sign in with Wallet
Connect another wallet