# Module 20: Network Security Data :::success Here is a cheat sheet generated from the course content using ChatGPT. It recaps the main concepts of each module, definitions and examples. At the end of (almost) each part, you will find a link to online related flashcards. ::: ### **20.1 Types of Security Data** --- #### **20.1.1 Alert Data** **Definition** Alert data is generated by intrusion prevention systems (IPS) or intrusion detection systems (IDS) when traffic matches a signature of a known exploit or violates a defined rule. **Example** A Snort IDS generates alerts for traffic that contains malicious patterns. For instance: **Snort Rule**: ``` alert ip any any -> any any (msg:"GPL ATTACK_RESPONSE id check returned root"; content:"uid=0|28|root|29|"; fast_pattern:only; classtype:bad-unknown; sid:2100498; rev:8;) ``` This rule triggers an alert when traffic contains `uid=0(root)` in the payload. **Testing Snort** Visiting the `testmyids` website generates a harmless alert to verify Snort functionality. The triggered alert contains: - **Message**: GPL ATTACK_RESPONSE id check returned root - **Rule ID**: 2100498 **Visualization** Tools like Sguil and Squert, part of the Security Onion suite, display and manage alerts. These tools enable analysts to view packet data and rule triggers, as seen in the **Sguil Console** interface. --- #### **20.1.2 Session and Transaction Data** **Session Data** Session data records metadata about network communication between two endpoints. It includes: - **Five Tuples**: Source and destination IPs, ports, and protocol. - **Metrics**: Duration, bytes, and packets exchanged. - **Tools**: Zeek logs session data, providing detailed metrics. **Example Zeek Fields**: 1. `ts`: Session start timestamp. 2. `uid`: Unique session ID. 3. `id.orig_h`: Source IP. 4. `id.resp_h`: Destination IP. 5. `proto`: Protocol used. 6. `service`: Application protocol (e.g., HTTP). **Transaction Data** Transaction data refers to the actual exchanged messages during a session, such as: - HTTP request: `GET /home/index.html HTTP/1.1` - HTTP response: `HTTP/1.1 200 OK` **Storage** Logged in access logs or captured by tools like Zeek, transaction data offers insight into user actions and server responses. --- #### **20.1.3 Full Packet Captures** **Definition** Full packet captures include every detail of network communication: metadata and the actual content (e.g., emails, web pages, files). **Use Cases** - Extract content for malware analysis or user behavior inspection. - Verify security policy violations. **Tools** - **Wireshark**: A popular tool for viewing full packet captures. - **Cisco Prime Infrastructure**: Supports detailed packet capture analysis. **Example** Captured traffic includes: - Full HTTP requests and responses. - Files or email contents for forensic analysis. --- #### **20.1.4 Statistical Data** **Definition** Statistical data is derived by analyzing session and transaction data to detect patterns, predict behaviors, and identify anomalies. **Applications** - **Network Behavior Analysis (NBA)**: Detects anomalies by comparing traffic to a baseline of normal behavior. - **Network Behavior Anomaly Detection (NBAD)**: Utilizes advanced analytics, such as machine learning, to identify threats. **Example Tool** - **Cisco Cognitive Threat Analytics**: - Uses machine learning to establish traffic baselines. - Detects anomalies and potential threats, such as: - Command-and-control (C&C) communications. - Data exfiltration. - Suspicious applications. **Visualization** Behavioral analysis, anomaly detection, and machine learning work together to flag potential threats in the network. ![image](https://hackmd.io/_uploads/B1u1Z0KN1e.png) **Figure Details**: Arrows between **internal users**, **behavioral analysis**, **anomaly detection**, and **machine learning** illustrate the architecture of advanced statistical analysis in detecting potential threats. :::danger **Check Your Understanding** :ballot_box_with_check: You can find the answers to the quiz by clicking [here](https://itexamanswers.net/25-1-5-check-your-understanding-identify-types-of-network-monitoring-data-answers.html). ::: >[!Warning]Recap >Alert data consists of messages that are generated by intrusion prevention systems (IPSs) or intrusion detection systems (IDSs) in response to traffic that violates a rule or matches the signature of a known exploit. Within the Security Onion suite of NSM tools, alerts are generated by Snort and are made readable and searchable by the Sguil, Squert, and Kibana applications. > >Session data is a record of a conversation between two network endpoints, which are often a client and a server. The server could be inside the enterprise network or at a location accessed over the internet. Session data will include identifying information such as the five tuples of source and destination IP addresses, source and destination port numbers, and the IP code for the protocol in use. Data about the session typically includes a session ID, the amount of data transferred by source and destination, and information related to the duration of the session. > >Full packet captures are the most detailed network data that is generally collected. Full packet captures require the greatest storage. Full packet captures contain the actual contents of data conversations, such as the text of email messages, the HTML in webpages, and the files that enter or leave the network. Extracted content can be recovered from full packet captures and be analyzed for malware or user behavior that violates business and security policies. > >Statistical data is created through the analysis of various forms of network data. Statistical characteristics of normal network behavior can be compared to current network traffic in an effort to detect anomalies. Statistics can be used to characterize normal amounts of variation in network traffic patterns in order to identify network conditions that are significantly outside of those ranges. Statistically significant differences should raise alarms and prompt investigation. ### 20.2 End Device Logs --- #### 20.2.1 Host Logs Host-based Intrusion Detection Systems (HIDS) operate on individual devices to detect and sometimes prevent intrusions. They generate logs stored locally, but these can be centralized for better visibility using log management servers. Open-source tools like OSSEC, Microsoft Windows Event Viewer, and Tripwire for Linux are commonly used. Windows Event Viewer provides visibility into five key types of logs: - **Application logs:** Events logged by software applications. - **System logs:** Information on drivers, processes, and hardware. - **Setup logs:** Data about software installations. - **Security logs:** Tracks events like login attempts and file access. - **Command-line logs:** Helps detect suspicious command-line activity. Windows events are classified into five event types: | **Event Type** | **Description** | |------------------------|-----------------------------------------------------------------------------------------------| | **Error** | A significant problem, such as a service failing to load during startup. | | **Warning** | Indicates potential issues, like low disk space, but no immediate system failure. | | **Information** | Describes successful operations, like a network driver loading successfully. | | **Success Audit** | Logs successful security-related attempts, such as a user successfully logging in. | | **Failure Audit** | Logs failed security attempts, such as an unauthorized user failing to access a network drive. | --- #### 20.2.2 Syslog Syslog provides a standardized format for event logging and is a client-server protocol. ![image](https://hackmd.io/_uploads/ByOi4chVkx.png) Syslog messages consist of: 1. **PRI (Priority):** Encodes Facility and Severity. The PRI value is calculated as: $$\text{Priority} = (\text{Facility} \times 8) + \text{Severity}$$ Severity levels are listed here: | **Value** | **Severity** | |-----------|---------------------------------------------------------------------------------------------------| | **0** | **Emergency:** System is unusable. | | **1** | **Alert:** Immediate action required. | | **2** | **Critical:** Needs correction to avoid failure. | | **3** | **Error:** A failure occurred but is not urgent. | | **4** | **Warning:** May lead to issues in the future if not addressed. | | **5** | **Notice:** Informational messages about unusual but not problematic activities. | | **6** | **Informational:** Reports normal operations. | | **7** | **Debug:** Developer-focused details. | 2. **HEADER:** Includes timestamp and hostname/IP. 3. **MSG:** Contains the actual message text, which is useful for understanding the event. --- #### 20.2.3 Server Logs Server logs are invaluable for tracking network activity. Common types include: - **Apache Access Logs:** Provide details on web server activity, as shown in this example: ``` 203.0.113.127 – dsmith [10/Oct/2016:10:26:57 - 0500] "GET /logo_sm.gif HTTP/1.0" 200 2254 "http://www.example.com/links.html" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0" ``` - **IIS Access Logs:** A Microsoft log example: ``` 6/14/2016, 16:22:43, 203.0.113.24, -, W3SVC2, WEB3, 198.51.100.10, 80, GET, /home.htm, -, 200, 0, 15321, 159, 15, HTTP/1.1, Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0), -, http://www.example.com ``` DNS proxy logs are another critical resource for monitoring malicious activity and detecting connections to malware CnC servers. --- #### 20.2.4 SIEM and Log Collection Security Information and Event Management (SIEM) systems provide: 1. **Log collection:** Gathers logs from diverse sources. 2. **Normalization:** Converts logs into a standardized format. 3. **Correlation:** Links related events for faster threat detection. 4. **Aggregation:** Combines duplicate data to reduce clutter. 5. **Reporting:** Real-time and long-term reports for monitoring and compliance. Tools like Splunk and Security Onion (which integrates ELK—Elasticsearch, Logstash, Kibana) are popular SIEM solutions. These platforms enable comprehensive monitoring, as depicted in the Splunk Threat Dashboard. Security platforms like Cisco SecureX and Palo Alto Cortex XDR extend SIEM functionalities by integrating security tools into a unified system, enhancing alert accuracy and streamlining responses. :::danger **Check Your Understanding** :ballot_box_with_check: You can find the answers to the quiz by clicking [here](https://itexamanswers.net/25-2-5-check-your-understanding-identify-windows-event-security-levels-answers.html). ::: >[!Warning]Recap >Host-based intrusion detection systems (HIDS) run on individual hosts. HIDS not only detects intrusions, but in the form of host-based firewalls, can also prevent intrusion. This software creates logs and stores them on the host. Many host-based protections have a way to submit logs to centralized log management servers providing a way for logs to be searched from a central location using NSM tools. OSSEC, a popular open-source HIDS, includes a robust log collection and analysis functionality. > >Syslog incudes specifications for message formats, a client-server application structure, and network protocol. Many different types of network devices can be configured to use the syslog standard to log events to centralized syslog servers. > >Server logs are an essential source of data for network security monitoring. Network application servers such as email and web servers keep access and error logs. DNS proxy server logs document all the DNS queries and responses that occur on the network. DNS proxy logs are useful for identifying hosts that may have visited dangerous websites and for identifying DNS data exfiltration and connections to malware command-and-control servers. > >SIEM combines the essential functions of security event management (SEM) and security information management (SIM) tools to provide a comprehensive view of the enterprise network using log collection, normalization, correlation, aggregation, reporting, and compliance. ### 20.3 Network Logs --- #### **20.3.1 Tcpdump** **Overview:** - Command-line packet analyzer. - Captures detailed protocol and content data. - Outputs either real-time or file-based captures. **Key Features:** - Real-time packet display. - Supports writing packet captures to files. - Protocol-agnostic structure varies based on requested fields. **Related Tools:** - **Wireshark**: GUI-based, built on Tcpdump functionality. --- #### **20.3.2 NetFlow** **Purpose:** - Developed by Cisco for network troubleshooting, monitoring, and security. - Records metadata about packet flows (not packet contents). **Components of IP Flow:** 1. IP Source Address 2. IP Destination Address 3. Source Port 4. Destination Port 5. Layer 3 Protocol Type 6. Class of Service 7. Router or Switch Interface **Tools:** - **nfdump:** CLI-based NetFlow analyzer. - **FlowViewer:** GUI-based tool for NetFlow data. **Use Cases:** - **Timeline of Compromise:** Identifying attack patterns. - **Behavior Tracking:** Host or attacker behavior analysis. **Visualization:** - Example NetFlow Record: - Source: 10.1.1.2:80 → Destination: 13.1.1.2:8974 - Duration: 00.010 seconds - Flags: .AP.SF --- #### **20.3.3 Application Visibility and Control (AVC)** **Description:** - Recognizes, analyzes, and controls over 1000 applications. - Powered by **Cisco’s NBAR2** engine. **Applications Monitored:** - Cloud services (e.g., Cisco WebEx, YouTube). - Communication tools (e.g., Skype, VoIP). - Peer-to-Peer (P2P) and gaming apps. **Key Features:** 1. Application Recognition: Identifies apps using Layer 3-7 data. 2. Metrics Collection: Tracks bandwidth, latency, jitter, etc. 3. Management & Reporting: Dashboards with **Cisco Prime**. 4. Application Control: Enforces QoS policies to prioritize critical apps. **Visualization:** - Port-based Monitoring: Limited granularity. - Application Monitoring: Tracks specific activities like streaming or teleconferencing. --- #### **20.3.4 Content Filter Logs** **Overview:** - **Cisco ESA/WSA:** Devices for email and web content filtering. **Capabilities:** - Logging email delivery, antivirus, antispam, and blocklist decisions. - Web proxy logging: Monitors inbound/outbound HTTP transactions. - Data Loss Prevention (DLP): Scans outgoing traffic for sensitive information. **Logging Formats:** - Text files stored locally or pushed via syslog/FTP/SCP. - W3C-compatible configurations. **Example Logs:** 1. **Email Logs:** Antivirus or spam activity. 2. **Web Logs:** URL filtering, malware scans, reputation filtering. **Visualization:** - Reports have drill-down dashboards (Overview → Detailed Reports → Targeted Search). --- #### **20.3.5 Logging from Cisco Devices** **Syslog:** - Default format for Cisco devices. - Logs submitted via SNMP or syslog protocols. **Syslog Components:** 1. Timestamp: E.g., `*Mar19 11:22:07.289 EDT:` 2. Cisco Facility: E.g., `ASA` 3. Severity: Levels 0-7 (e.g., `-3-` for error). 4. Message ID and Text: Unique identifier and context. **Formats:** - Cisco ASA Syslog: `%ASA-3-201008: Disallowing new connections.` - Cisco IOS Syslog: `%SYS-5-CONFIG_I: Configured from console by con0.` ![image](https://hackmd.io/_uploads/rJgOBCtNJg.png) --- #### **20.3.6 Proxy Logs** **Functionality:** - Acts as intermediaries for client-server communication. - Logs all HTTP/DNS requests and responses. **Applications:** - Identifying malicious destinations. - Monitoring data leaving the network for DLP. **Examples:** - **Squid Proxy Log:** - Log: `1265939281.764 19478 172.16.167.228 TCP_MISS/200 864 GET http://www.example.com//images/home.png` - Components: Timestamp, Duration, Client IP, Result (e.g., TCP_MISS), Request URL. - **Cisco Umbrella DNS Logs:** - Example: `"2015-01-16 17:48:41", "ActiveDirectoryUserName", "Allowed", "domain-visited.com"` --- #### **20.3.7 Next-Generation Firewalls (NGFWs)** **Capabilities:** - Extends security beyond IP/port to application layers. - Example: Cisco Firepower NGFW. **Features:** 1. **Connection Event:** Logs session metadata. 2. **Intrusion Event:** Tracks malicious activity. 3. **Host/Endpoint Event:** Logs device behavior and changes. 4. **Network Discovery Event:** Monitors network changes via discovery policies. 5. **NetFlow Event:** Uses NetFlow records to generate network events. **Services Provided:** - Intrusion Prevention (Subscription). - Advanced Malware Protection (AMP). - URL Filtering (Subscription). - Application Control. - Network Profiling. **Note:** Always tailor logging configurations to your organizational needs and use dashboards to streamline monitoring and reporting. :::danger **Check Your Understanding #1** :ballot_box_with_check: You can find the answers to the quiz by clicking [here](https://itexamanswers.net/25-3-8-check-your-understanding-identify-the-security-technology-from-the-data-description-answers.html). ::: :::danger **Check Your Understanding #2** :ballot_box_with_check: You can find the answers to the quiz by clicking [here](https://itexamanswers.net/25-3-9-check-your-understanding-identify-the-nextgen-firewall-event-types-answers.html). ::: >[!Warning]Recap >The tcpdump command line tool is a very popular packet analyzer. It can display packet captures in real time or write packet captures to a file. Captures include detailed packet protocol and content data that can be viewed with Wireshark. > >NetFlow efficiently provides an important set of services in a session-based format for IP applications, including network traffic accounting, usage-based network billing, network planning, security, Denial-of-Service monitoring capabilities, and network monitoring. NetFlow provides valuable information about network users and applications, peak usage times, and traffic routing. > >Cisco Application Visibility and Control uses Cisco next-generation network-based application recognition version 2 (NBAR2), also known as Next-Generation NBAR, to discover and classify the applications in use on the network. The NBAR2 application recognition engine supports over 1000 network applications. > >Devices such as the Cisco Email Security Appliance (ESA) and the Cisco Web Security Appliance (WSA), provide a wide range of functionalities for security monitoring by utilizing content filtering. Logging is available for many of these functionalities. > >Proxy servers are devices that act as intermediaries for network clients. An enterprise may configure a web proxy to handle web requests on the behalf of clients. Instead of requests for web resources being sent directly to the server from the client, the request is sent to a proxy server first. The proxy server requests the resources and returns them to the client. The proxy server generates logs of all requests and responses that can be analyzed to determine which hosts are making the requests, whether the destinations are safe or potentially malicious, and to also gain insights into the kind of resources that have been downloaded. > >NextGen Firewall devices extend network security beyond IP addresses and Layer 4 port numbers to the application layer and beyond. NexGen Firewalls are advanced devices that provide much more functionality than previous generations of network security devices. NextGen Firewall reporting dashboards with interactive features allow quick point-and-click reports on very specific information without the need for SIEM or other event correlators.