1f. Hunt for outliers and unknown threats

At this point, detection engineers are hunting for baseline outliers and potential threats. There are two advantages to doing this so early in UDEF. First, it results in better understanding of the data, explains why outliers may exist, and if the outliers are normal and expected or from pre-existing issues or threats. Secondly, it serves as a precursor to the process of hunting for known threats, which formally occurs later in the framework.

During this step in the process, the idea is to approach it from a generic data and architectural technology perspective, with the goal being that an abstracted, generalized approach allows focus at the threat and data level as opposed to reacting based on resource or risk. You may notice some overlap with the 1e/step 5, where we can leverage those initial findings to pivot and delve into the anomalies and outliers.

Below are some approaches to help hunt for outliers and unknown threats.

Perform statistical analysis via searches across the data

Search the data manually and pivot on unusual findings or results. Annotate things to follow up on or create the aforementioned rules or visualizations on them.

As an example, we can search for hosts connecting to the most unique remote systems. We can pivot from there to see how often the connections occur across the environment as well as per host. Finally, we can look at these connections over time.

Fig. 12 ES|QL search to find hosts with the most unique connections by direction

Fig. 13 Pivot to show total connection counts by remote address and direction

Fig. 14 Pivot to show total connection counts by remote address and direction per host

Fig. 15 Pivot to show trend analysis of ingress and egress network traffic

Based on the results, the next steps could include pivoting to focus on specific remote addresses, local hosts, or even significantly deeper by breaking down by users, processes, touched files, etc.

Machine Learning jobs and rules

Write anomaly-based, machine learning jobs or statistical analysis rules generic enough to capture recurring occurrences of specific behaviors, which aren’t necessarily threat focused.

Fig. 16 Kibana machine learning job creation page

Dashboards and visualizations

Build out dashboards and visualizations of the data to take advantage of identifying visual anomalies.

Fig. 17 Kibana built-in dashboard for security detection & response

Compare the resulting outliers with baselines from the previous steps

An example of this could be taking the known systems within an environment assembled in 1a, identifying intra-LAN network traffic from step 1f, and comparing it with expectations determined from 1b and 1c. It could reveal hosts talking to each other that shouldn’t be, which could be indicative of malicious activity such as lateral movement, or benign expected connectivity that was unintentionally unaccounted for.

Focus on common known vectors and behaviors

Attackers and threats have tendencies, such as popular tools or targeting specific resources. Focus on these or where stovepipes of such behaviors may occur (such as powershell execution). Derive likely scenarios from 1e and generically scope it to see if anything sticks out. Later on in 3f, focused research and threat hunting will overlay the specifics of the environmental defense posture and specific threat landscape.