Broken Glam: How healthy is your SIEM?

Posted by Ilan Goldschmidt on December 21, 2021

Threats are constant. Organizations are trying to always stay ahead of new methods of attack, APT groups, and other known vulnerabilities. A key component of any SOC is a well-functioning SIEM. However, the SIEM is only as good as its detection rules! Many times rules are broken, leaving you vulnerable to outside threats.

Why should you care about the health of your rules?

Why should you care about rules in the first place? Well, because many of them are probably ineffective in achieving their goal of detecting threats. In fact, we conducted an analysis of multiple large-scale SIEM deployments across large organizations and found that 25% of the rules were broken, on average . These rules will never fire, primarily due to fields that are not extracted correctly or log sources that are not sending the required data.

This leads to a false sense of security because organizations are typically unaware that their rules are not functioning.

Why are there so many broken rules?

Our research further shows that most of the broken rules were created and implemented by someone outside of the organization. The usual sources of rules are:

  1. SIEM built-in rules
  2. SIEM content packs
  3. Integrators, professional services, and MSSPs

We have discovered that some service providers dump their entire content library on all their customers creating generic rules which are noisy at best since they were enabled pre-tuning, or they're broken!

Start analyzing these rules before moving on to your custom ones. 

Don't be shy - make sure to inform your service provider on any broken rules you find since they should be held accountable for the content they provide. 

Start with the Source

Rules will always rely on a log source, and we discovered that the number one reason for broken rules is a malfunctioned log source. Check that the log source exists, enabled, and sends logs regularly to the SIEM. We have seen too many stale or disabled log sources causing rules to break and never trigger.

Dig Deeper

There is still more though! Security Engineering is doing something really important for your organization, and unfortunately, it isn't easy. You need to think of what the rule is searching for in the payload. Look for the following:

  1. Parsing issues - the fields which are being searched for are not parsed and (depends on the SIEM) might require field extraction.
  2. Incompatible field values - operators must operate on the same field types otherwise they will most likely never be true.
  3. Query helps health - rules often rely on entities such as QRadar reference sets, Splunk look-ups, etc. These entities can be empty or missing. 

While these will probably find you 80% of your broken rules, there is more SIEM specific analysis which will find you the rest. A great topic for another blog!

I'm done! What now?

Scanning and fixing rules is not a one-time thing. New rules, as they should, will keep finding their way into your SIEM. New rules are added and need to be constantly checked and old rules can break due to environmental changes.

This of course, depends on how dynamic your environment is but a good best practice will be to repeat this process on a quarterly basis. This will ensure you are addressing threats and optimizing your threat coverage.

Pay attention and try to categorize rules which got broken over time. If these are usually new rules - consider implementing controls pre-deployment. If these are old rules, consider adding controls when changes are made in your SIEM.

Finally, we know this process takes time, effort, and skills. However, it is imperative that you keep at it to maintain and improve your threat coverage. 

CardinalOps can help by continuously analyzing your rule set for broken or missing rules. View a 2-minute demo.