Anomaly Detection (AD) is a ThreatIQ component that detects suspicious behavior based on “deviations” from historical models of activity. The justification for using Anomaly Detection for inferring suspicious behavior is based on the observation that many malicious actions leave a footprint that significantly changes the typical behavior of an entity. For example, a malware may alter the observed traffic patterns when trying to propagate to other workstations or when communicating with C&C servers. When combined with input from other systems, significant evidence may be accumulated in order to raise security alerts for zero-day attacks or in order to provide a level of defense for customers not protected by other security measures.
The AD component analyzes log data generated from network and security devices and gradually builds time-series and seasonal historical models which express expected behavior. Unexpectedly high values (or very low in some cases) in one or more monitored metrics are statistically tested for significance and alerts are triggered when they exceed a minimum level of confidence. The aforementioned deviations are either referring to recent behavior (using time-series smoothing-like algorithms) or to departures from known historical activity which is fed as input in a custom time-weighted decision model.
The AD component uses an advanced proprietary model in order to surpass limitations in the available information and increase the sensitivity and specificity of the system. In order to reduce false negatives (i.e. not detecting an attack) without raising false positives, an extended set of carefully-designed features (measured and computed metrics) has been designed, complemented by data enrichment with external metadata (Origin/Destination GeoIP, AS numbers and IANA-registered services).
Anomaly Detection differs from other Security Mechanisms (like Firewalls, signature-based IPS etc.) since it does not incorporate hand-crafted, rule-based defense but instead it approaches security incidents as unexpected events – in a statistical or information theoretic sense. These unexpected events often correlate with disruptions, attacks or unwanted activity in general. Although often being subject to false positives, the power of anomaly detection lies in detecting novel attacks.
One of the most significant advantages of Anomaly Detection as a security mechanism is transparency : interpreting and validating an anomaly detection event is natural and straightforward. Upon receiving an alert, one may easily investigate the root causes by inspecting the “raw” logs of activity for a monitored entity. The simplicity and transparency of the underlying model allows it also to be easily tuned and be combined with other models in more elaborate decision making systems.
The Anomaly detection component analyzes internal assets, networks and more specific types of activity for those entities:
The specific metrics regarding network events (TCP/UDP/ICMP network sessions, device logins, VPN requests, emails) and security events (IDS alerts, spam, firewall blocks, failed logins) related to the above entities are described in the “MOREAL Metrics” documentation.