Despite broad adoption of security information and event management (SIEM) software, threat coverage remains below an organization’s expectations, creating a false sense of security.

That’s according to new research by CardinalOps. It highlights failures within the enterprise SIEM among the Fortune 1000. Nine out of 10 customers surveyed represent multibillion-dollar, multinational corporations.

The SIEM system is typically the centerpiece of the SOC. It detects and responds to attacks that circumvent an organization’s threat prevention layer.

Businesses invest more than $3 billion annually in SIEM software, so they expect this investment to result in comprehensive threat coverage. However, an analysis of live SIEM deployments reveals threat coverage remains far below expectations, and what SIEM software and detection tools can provide. That’s among CardinalOps customers in multiple industry verticals, including health care and financial services.

Even worse, businesses are often unaware of the gap between the theoretical security they assume they have and the actual security they do have.

SIEM deployment has rules associated with only 16% of the techniques listed in the MITRE ATT&CK framework. It’s an industry-standard catalog of tactics, techniques and procedures used by attackers.

What’s Behind Low Coverage

Yair Manor is CardinalOps‘ co-founder and CTO.

CardinalOps’ Yair Manor

“There are several reasons contributing to the low coverage,” he said. “Visibility is one obvious problem. Since it is hard for organizations to know what their effective coverage is, it never becomes a concrete problem that needs to be addressed. Additionally, the complexity of managing and operating the SIEM often creates a glass ceiling, limiting the coverage that can be achieved.”

Also, security engineers don’t know how to address the latest use cases and threats, Manor said.

The problem is not with adapting existing rules and policies to MITRE ATT&CK.

“It is emphatically not the case that organizations have a multitude of rules and policies that provide broad coverage and are simply not mapped to MITRE ATT&CK,” Manor said. “The problem is the lack of rules and policies, not the adaptation. Thus, MITRE ATT&CK is not creating the problem, but rather is a tool for quantifying and measuring it.”

It’s necessary to use a pre-defined framework such as MITRE ATT&CK to measure the coverage of the SIEM and other products, he said.

“No single framework is a silver bullet,” Manor said. “However, MITRE ATT&CK is the best tool today for these purposes.”

Broken SIEM Rules

CardinalOps‘ research shows that an average of 25% of SIEM rules are broken and will never execute. That’s primarily due to fields that are not extracted correctly or log sources that are not sending the required data.

However, organizations are completely unaware that these rules aren’t functioning, according to CardinalOps. Additionally, only 15% of SIEM rules lead to 95% of the tickets handled by the SOC. That means a small percentage of noisy rules overwhelm SOC analysts with distracting false positive (FP) alerts.

“Buying security technologies seems to be a much easier task than utilizing them and operationalizing them for many organizations,” said Anton Chuvakin of Google Cloud‘s security solution strategy.

There’s more guidance on buying security than on how to make use of it in a particular environment, he said.