Announcement: open-sourcing ee-outliers

Today, we are excited to announce we are open-sourcing ee-outliers, our in-house developed framework to detect outliers in events stored in Elasticsearch!

The framework was developed for the purpose of detecting anomalies in security events, however it could just as well be used for the detection of outliers in other types of data. We have been developing ee-outliers in-house for the past year as part of Eagle Eye, our own security monitoring technology built on top of the ELK stack. As we are getting some great results from the outlier detection capabilities of ee-outliers, we have decided to share this work with the community as a way of giving back, as we are convinced this could be useful to a broad range of users: from individuals wanting to analyse their personal data to Security Teams building their own security monitoring capabilities.

The framework makes use of statistical models that are easily defined by the user in a configuration file. Below, you can find an example of a use case that is capable of detecting beaconing TLS connections. Similar use cases can be added by duplicating the use case, changing the query filter and the aggregator fields, and done!

68747470733a2f2f666f72657665722e6461616e72616d616e2e636f6d2f73637265656e73686f74732f436f6e66696775726174696f6e253230757365253230636173652e706e673f7261773d74727565.png
Configured use case to detect beaconing TLS connections

In case the models detect an outlier, the relevant Elasticsearch events are enriched with additional outlier fields. These fields can then be dashboarded and visualized using the tools of your choice (Kibana or Grafana for example). Below, you can find the resulting tagged events based on the SSL beaconing use case above, dashboarded in Kibana.

68747470733a2f2f666f72657665722e6461616e72616d616e2e636f6d2f73637265656e73686f74732f426561636f6e696e67253230646574656374696f6e2e706e673f7261773d74727565
Detecting beaconing TLS connections using ee-outliers
68747470733a2f2f666f72657665722e6461616e72616d616e2e636f6d2f73637265656e73686f74732f456e7269636865642532306f75746c6965722532306576656e74253230322e706e673f7261773d74727565
Detected outlier events are enriched with new fields in Elasticsearch

The possibilities of the type of anomalies you can spot using ee-outliers is virtually limitless. A few examples of types of outliers we have detected ourselves using ee-outliers during threat hunting activities include:

  • Detect beaconing (DNS, TLS, HTTP, etc.)
  • Detect geographical improbable activity
  • Detect obfuscated & suspicious command execution
  • Detect fileless malware execution
  • Detect malicious authentication events
  • Detect processes with suspicious outbound connectivity
  • Detect malicious persistence mechanisms (scheduled tasks, auto-runs, etc.)

We welcome all contributions to the project, as well as feature suggestions and feedback. We will soon start sharing example threat hunting use cases, updates & ways in which we and other security teams can leverage ee-outliers on this blog, so keep an eye out for new content!

The project is hosted on Github, and can be found here:
https://github.com/NVISO-BE/ee-outliers

We look forward to your feedback and to see the ways in which you and your teams use ee-outliers to improve your own security monitoring and threat hunting activities!

About the author

-tRU0XaV_400x400Daan Raman is in charge of NVISO Labs, the research arm of NVISO. Together with the team, he drives initiatives around innovation to ensure we stay on top of our game; innovating the things we do, the technology we use and the way we work form an essential part of this. Daan doesn’t like to write about himself in third-person. You can contact him at draman@nviso.be or find Daan online on Twitter and LinkedIn.

 

3 thoughts on “Announcement: open-sourcing ee-outliers

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s