Big Data Analytics is the Future of the Intelligence-driven Security Operations Center

Jeremy Kelley, Head of Solutions Innovation, HPE Security Products, Hewlett Packard Enterprise [NYSE:HPE]

In today’s digital economy we constantly hear about the potential for big data to transform the way we make decisions and conduct business, but the reality is that without the proper technologies, people, and processes in place, organizations can struggle to leverage the deluge of data for meaningful insights and value. This holds true for Security Operations Centers (SOCs) where security professionals are tasked with the challenge of leveraging the vast amount of data to monitor, detect and respond to potential threats in real-time.

The security industry as a whole is not catching enough of the threats and not catching them fast enough, as it takes an organization an average of 146 days to detect a breach. Security teams need advanced big data analytics to better detect and respond to threats.

While the influx of data spanning multiple sources such as IT, operational technology, Internet of Things (IoT) and physical can be overwhelming if not used properly, it provides the visibility and scope needed to proactively hunt adversaries and effectively mitigate risk. The future intelligence-driven security operations center will put security professionals on the offensive in stopping adversaries, and will be achieved through effective data integration, data exploration, and properly trained security analysts.

  ​To achieve the next-generation intelligent SOC, organizations need to first leverage security solutions that enable data integration and ingestion from multiple sources  

Data Integration Key to the Evolution of the SOC

Traditionally, SOCs have leveraged big data by trying to get as much information as possible to find known cyberattack patterns. For example, using the data to write correlation rules to determine when a host downloaded something from a malicious site and then communicated with another malicious site. While this can be effective in detecting known threats, the SOC of the future will feature a new model that transforms the data collection and monitoring process, allowing security professionals to proactively seek out and mitigate both known and unknown threats instead of responding reactively. This will allow organizations to detect and respond faster to potential breaches.

To achieve the next-generation intelligent SOC, organizations need to first leverage security solutions that enable data integration and ingestion from multiple sources, and provide one comprehensive picture of the overall landscape. This means having a centralized event broker that can easily integrate, collect, and process all of the data from IT, IoT, OT, and physical to provide the context and visibility needed to use the data at scale for meaningful insights. Without it, organizations are left with an overwhelming amount of data points from disparate sources, which can actually be counterproductive and detrimental in detecting threats as it adds more noise and alerts to sort through.

Meaningful Detection through Data Exploration

Once organizations collect all of the data and have a clear picture of the overall landscape, they need to slice and dice the information to better inform their investigations. They can do this by leveraging the event broker to seamlessly connect with third-party analytics tools such as Hadoop to really amplify data exploration.

This can help with both known and unknown threats by creating a baseline to track against. For example, exploring a user’s behavior to see how she typically interacts with a particular server, and then seeing what the server has done subsequently. This intelligent information can inform investigations for better detection of unknown threats. Instead of waiting long enough for the breach to get loud enough or correlate to a previously set rule, analysts can be much more proactive by identifying patterns and potential threats.

This is where "hunt teams" come into play, as they can use big data analytics tools to proactively track and catch unknown threats.

Hunting for Trouble

In some ways, the analysts on the typical hunt team behave exactly as one might expect. Hunched over screens and clicking through complex dashboards searching for patterns. It's the kinds of patterns they're searching for that is unique. Instead of tracking every DNS record and every badge entry and looking for outliers, they're using big data to develop pictures of general security activity. So when a picture shows a small cluster of activity that doesn't fit with the overall flow the hunt team can open an investigation. With big data tools they can drill down to finer-grained details, making it easier to spot anomalies that may indicate a breach.

A good Security Information and Event Management (SIEM) working in tandem with rich big data analytics tools gives hunt teams the means to spot the leads that are actually worth investigating. In addition, organizations must invest in training their hunt teams and other security analysts to properly leverage the data and spot potential attack patterns.

Turning the Unknown into the Known

Over time, hunt teams’ work to find and detect unknown breaches and other attack avenues that can yield intelligence about where to prepare new defenses. For example, if there is an anomaly in the system for tracking badge entries, further investigation may yield hits to the accounting system from unknown IPs that normally wouldn't have access. The team may conclude that an impostor has gained access to a badge and has been spending an inordinate amount of time reprogramming servers to provide back-door access to miscreants hoping to steal either funds or valuable corporate information.

Fixing the breach could be as simple as adjusting badge permissions so that only those who actually have to work in the server room are granted access via their badge.

The Future of Intelligence-Driven Security

For CIOs and Chief Information Security Officers (CISOs), the modern Security Operations Center has to be more than just be a place that tracks and files security events. The SOC has to be the headquarters for informed investigations into both known and unknown threats facing the enterprise, whether it’s targeted at the network, an application, or a connected device. That means adopting big data security analytics tools that integrate the data collection process allowing for end-to-end visibility and data exploration, as well as properly training security staff to quickly identify anomalies and potential threats. This advanced analytics approach will help transform the security landscape, empowering organizations with faster detection and response.

Read Also

Tapping the Potential of IT to Achieve Academic Excellence

Tapping the Potential of IT to Achieve Academic Excellence

Dan Moore, CIO, Southeastern Oklahoma State University