Distributed Data Lakes
Distributed Data Lakes
The foundation for a successful autonomous SIEM, SOC, and Threat Hunting program is the availability of quality logs and sensor-enriched data. Regulatory requirements drive minimum log storage and SIEM ingestion standards; however, feature-enriched metadata such as geo-location and Intel matched context is key to generating actionable alerts in your SIEM.
Examples include logs from Networking Devices, DNS (internal and external), OS, Security Devices (IPS, Secure Web Gateway, DLP), SaaS applications (O365, Salesforce), Cloud (AWS Cloud Trail, Azure tenant), Email, and Sensor Devices (Bro, Suricata). Combining this security information across your environment will enrich the data and assist in event correlation with more visibility.
"Collecting the logs is the first step. All log data is not equal and varies depending on the hosting provider whether it be on-prem, Cloud, or SaaS"
Collecting the logs is the first step. All log data is not equal and varies depending on the hosting provider, whether it be on-prem, Cloud, or SaaS. If your organization has a converged IT Operations environment or one that is driven by Agile DevSecOps, storing ALL operational logs including security all in one location becomes challenging. On-prem data center logs typically are available via a traditional syslog forwarder whereas Cloud and SaaS logs are available via AWS S3 through an AWS Lambda pull or via an API call from Azure Event HUB or Salesforce API. Some logs are only available via a proxy and must be sent to something like Microsoft Cloud App Security (MCAS) and then pulled from the MCAS Event Hub to your SIEM. Care must be taken to ensure log immutability regardless where it is stored. Responsibility of that immutability must be clearly defined in the SOC2.
Tip: It used to be recommended that you generate as much detail as you need on the local system, but filter and only send the most valuable and critical events to your centralized log management system. Use CIS Benchmarks to ensure local logging is set sufficiently and monitor the log volume in your SIEM to ensure it’s receiving the right type of data.
Security Analytics is an approach to cybersecurity-focused on attaching metadata to events within your SIEM, such as geo-location and intelligence.
No business can predict the future, especially where security threats are concerned, but by deploying security analytics tools that can analyze security events it is possible to detect a threat before it has a chance to impact your infrastructure and bottom line. If your security tools fail at prevention, security analytics can also help reduce the dwell time of the attacker and alert the SOC that continued vigilance is needed.
Depending on the types of tools installed, security analytics solutions can incorporate large and diverse data sets in your SIEM via detection algorithms. Security analytics data can be collected in several ways, including from:
• Network flow traffic
• Endpoint and user behavior data
• Cloud resources and services
• Business applications
• Non-IT contextual data
• Identity and access management data
• External threat intelligence sources
• Physical security systems data
Security analytics has a variety of use cases, from improving data visibility and threat detection to network traffic analysis and user behavior monitoring. Some of the most common security analytics use cases include:
• Employee monitoring
• Analyzing user behavior to detect potentially suspicious patterns
• Analyzing network traffic to pinpoint trends indicating potential attacks
• Identifying improper user account usage, such as shared accounts
• Detecting data exfiltration by attackers
• Detecting insider threats
• Identifying compromised accounts
• Investigating incidents
• Threat hunting
• Demonstrating compliance during audits
• Provide Blue and Purple Team tools to identify attackers (and Pen Testers)
• Information sharing to colleagues in your ISAC
As an example, information sharing is critical in the Financial Services Industry and executed through the Financial Services – Information Sharing and Analysis Center (FS-ISAC). Animmediate use case for next-generation SIEM security analytics tools in FS-ISAC are responding to requests for information and sharing active attacker indicators of compromise. Having a security analytics tool that can provide real-time visibility helps answer the question of “Is this system with an IP address in Iran attacking you on port 22?” to share community attack data. It helps understand whether the attacks are targeted or are general commodity attacks.
Above all, the primary goal of security analytics is to turn raw data from many disparate sources into actionable insights to identify events that require an immediate response through the correlation of activities and alerts. In doing so, security analytics tools add a critical filter to the volumes of data generated by users, applications, networks, and other security solutions in place.
Threat hunting is the pursuit of abnormal activity on systems, applications, users, and the network that may be indicators of compromise, intrusion, or exfiltration of data. Though the concept of threat hunting isn’t new, for many organizations the very idea of active threat hunting is. Many organizations choose to outsource this task to SIEM providers to augment their SIEM/ SOC offering such as FireEye Managed Defense.
To do this effectively, your team needs next-generation security analytics tools that give them highly granular visibility into the goings‐on in the operating systems of every endpoint and server, the network, and applications in use across the enterprise.
In The Cloud
While there have been cloud-based log monitoring and SIEM tools in the past, the big cloud infrastructure providers have entered the game this past year. AWS announced Security Hub at 2018 Re:Invent. Microsoft announced Sentinel at the 2019 RSA Conference. Google also announced the new Chronicle Backstory product at the 2019 RSA Conference. FireEye, one of the premier prevention, detection and response companies, announced next-generation SIEM and cloud monitoring capabilities in their HELIX security operations platform at their 2018 Cyber Defense Summit along with providing their customers with free access to Security Orchestration & Automation and Response (SOAR) capabilities through playbook responses. Automated playbook response is much more desired over delayed human response. Why wait to block a malicious IP address with your weekly firewall block process when you can do it automatically. It’s much more preferred to send an API request to your firewall via a playbook response. These are the crowning story on next generation cloud-based SIEMs while cloud enablers including inexpensive data ingress on the network, cheap cloud data storage, and cloud elasticity and scalability continue to drive SIEM customers to solutions that are better “in the cloud”.
The next generation SIEM provides advanced detection capabilities to enable your organization to correlate events in multiple dimensions - by identity, vulnerability, asset, time, patterns and other events - across all security log generations such as firewalls, web servers, system access logs, and other security services to determine if a system has been successfully attacked, is currently being probed for attack, or detect advanced threats before they cause damage. Where possible, your next generation SIEM should take advantage of being in the cloud and have the capability to auto-remediate problems as they are detected in real-time.