# Creating Splunk Alerts
This article demonstrates how to create detection alerts in Splunk. We will focus on:
- Understanding components and functions of Alerts.
- Configuring Alert's settings.
- Creating sample Alerts based on real-world SOC use cases.

### What are Alerts in Splunk ?
Alerts in Splunk are specialized saved searches that monitor for specific events, such as error thresholds or security threats, in real time or on a schedule. When defined conditions are met, they trigger actions like sending emails, running scripts, or creating incidents to notify users and initiate responses. Besides, Alert is a knowledge object in Splunk. For more information about knowledge object,see this [doc](https://docs.splunk.com/Splexicon:Knowledgeobject).

### Key Aspects of Splunk Alerts:
#### Alert Types:
- Scheduled: Runs the search at defined intervals (e.g., every hour, daily).
- Real-time: Monitors data continuously, allowing immediate detection of issues, which is often used in SOC scenarios.
#### Trigger Conditions:
Set up to trigger every time a result is found or when specific thresholds are met (e.g., error count > 5 in 1 minute)
#### Actions:
When triggered, alerts can send emails, run scripts, or populate the Triggered Alerts page.

#### Throttling:
Limits the number of alerts triggered over a period to prevent alert fatigue

## I.Steps

First, you need to create a Search query identifying specific logs that you want to monitor.Here are some Examples:

index=windows EventCode=4625
| stats count by user
| where count > 5

index=sysmon EventCode=1
| search Image="*powershell.exe"

#### Step 1:
To save your query as an Alert,run it,click on "Save As" and select "Alert"

#### Step 2:
In the "Save As Alert" dialog,config the following settings

##### Title:
Give your Alert a descriptive name(e.g.,PowerShell detected,Failed Login Attempts - Windows Servers,...)

##### Description:
Optionally,you can add description to provide more context about your Alert.

##### Permissions:
There are two options
- Private: Makes the alert private to the alert creator
- Shared in App: Display the alert for all users of the app.

See this [doc](https://help.splunk.com/en/splunk-enterprise/manage-knowledge-objects/knowledge-management-manual/10.2/get-started-with-knowledge-objects/manage-knowledge-object-permissions) for more knowledge object permissions.
##### Alert type:
Splunk has two Alert types

- Scheduled: Use a scheduled alert to search for events on a regular basis and monitor whether they meet specific conditions. A scheduled alert is useful if immediate or real-time monitoring is not a priority.
- Real-time: Real-time alerts search for events continuously. They can be useful in situations where immediate monitoring and responses are important. You can use real-time alerts that trigger once per result or only if certain conditions are met within a specific rolling time window.Rolling time window known as a "rolling window alert" is a continuous, moving interval applied to real-time alerts or searches. It calculates metrics or triggers alerts based on data that constantly updates—adding new data while discarding old data as time passes. This is used to detect patterns over a specific recent duration(e.g.5 failed logins within 10 minutes)
##### Expires:
Determines how long the alert results remain accessible in the system after they are generated.
##### Trigger Conditions:
Your Alert does not have to trigger everytime the search results appear,trigger conditions are used to monitor patterns in event data or prioritize specific events
##### Trigger:
There are two options
- Once: The Alert will trigger once the threhold is reached.
- For each result: The Alert will trigger for every matching result.
##### Throttle:
Suppresses redundant notifications by pausing alerts for a specified period when specific field values reoccur,prevents alert fatigue and allows you to limit notifications.Throttle can be applied to both scheduled and real-time alerts.Especially for real-time searches, it is highly recommended to apply throttle to avoid excessive noise.
##### Trigger Actions:
This setting helps you respond to trigger Alerts.You can enable more than one actions
- Email notification action: Sending email notifications when alerts trigger
- Use a webhook alert action: Displaying a message in a chat room or updating another web resource
- Output results to a CSV lookup: Writing the results of the triggered alert or scheduled report to a CSV lookup file
- Log events: Logging and indexing searchable alert events
- Monitor triggered alerts: Adding an alert to a list of recently triggered alerts for monitoring
- Send alerts and dashboards to Splunk Mobile users: Sending an alert to Splunk Mobile users

After configuring the settings above,clink "Save" to create your Alert.

#### Step 3:
To edit your Alerts after creating them,go to "Searches, reports, and alerts" in the "Settings" section
Click on the "Edit" drop down arrow
- Choose "Edit Alert" to edit the settings mentioned in step 2
- Choose "Edit Permissions" to edit your Alert's permissions which define who can read and modify it

## II.Examples

### 1. SSH brute-force Alert
In this section,we will use this query to identify signs of SSH brute-force attacks. In reality,attackers often perform brute-force attacks by repeatedly attempting SSH authentication.This alert helps SOC analysts identify potential unauthorized access attempts and take appropriate response actions by monitoring for a high number of failed login attempts within a short time frame

index="linux_auth_log" source="/var/log/auth.log" ("Failed password" OR "invalid user")  
| rex "(?<user>[\w]+) from (?<src_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})" 
| bin _time span=5m 
| stats min(_time) as firstTime max(_time) as lastTime count by user, src_ip
| where count > 10                                                                                                   
| convert ctime(firstTime)
| convert ctime(lastTime)

Explanation:
- Use `rex` command to extract users and source IP from raw logs
- Use `bin` command with `_time` and `span=5m` parameters to group events into 5-minute buckets to show activity over time
- Use `stats` command with `by` clause to create a data table
- Use `convert ctime` command to convert `first/lastTime` field to a human-readable string

With the settings:
Setting | Value|
|----------------------|-----------------------|
|Cron Schedule |0 * * * * |
|Earliest Time | -70m@m |
|Latest Time | -10m@m |
|Schedule Window | auto |
|Trigger conditions| Number of results is greater than 0|
|Throttle: Supress triggering for| 60 minutes|
|Trigger Actions | Add to Triggered Alerts|

See the result

### 2.Windows PowerShell Script Block With Malicious String
The following Alert detects the execution of PowerShell scripts containing Base64 encoded content. It filters events which have fields that contain specific strings.

index= winsysmon EventCode=1 Image="*powershell.exe" CommandLine = "*EncodedCommand*"
| table _time host user Image CommandLine ParentImage

Explanation:

- `table` is a command used to format search results into a clean, tabular format

the settings are the same as the previous example.See the result

>[!NOTE] Note:
>This detection may generate false positives if users intend to use encoded commands.
>This Alert will not trigger if attackers instead of using fully spelled out `-EncodedCommand` parameter,they use `-eNcO` or `-e^N^c^O` which the regex in this Alert won't catch.

Visit [Splunk documentation](https://help.splunk.com/en/splunk-enterprise/search/spl-search-reference/9.0/quick-reference/commands-by-category) for more information about Splunk commands and [Splunk Security Content](https://research.splunk.com/) for real-world Alert queries.