Arnaud Loos

Elasticsearch 7.4 - Snapshot and Restore

2019-10-03T12:00:00-04:00

The recent 7.4 release of the ELK stack now includes the ability to snapshot (backup) and restore indices from within Kibana as well as manage repositories and policies. In this walkthrough we’ll configure Elasticsearch to snapshot to an Amazon S3 bucket. Note that Azure and Google Cloud are supported as well.

In short we’ll be creating a new S3 bucket, creating an IAM account with permissions to just the new bucket, installing the Elasticsearch S3 Repository Plugin, creating a repository, and creating an associated Policy to specify which indexes to backup and how often. I’ll then test restoring a single index with a new name.

Configure AWS

Let’s start by logging into AWS and navigating to S3.

Create a new S3 bucket. I’m calling mine “filebeat-backup”.
Accept all the other defaults and create the bucket.

Now navigate to IAM in the AWS console.
Users > Add User.
I’m naming my user es-backup and giving the account Progmatic Access.
On the Set Permissions screen select Attach existing policies directly.
Click the Create policy button.
A new window opens. Click JSON tab.

Copy over the plugin’s recommended policy and be sure to change the bucket name (arn:aws:s3:::filebeat-backup) twice.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation",
        "s3:ListBucketMultipartUploads",
        "s3:ListBucketVersions"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::filebeat-backup"
      ]
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::filebeat-backup/*"
      ]
    }
  ]
}

Give the new policy a name.
Close this window and go back to the window to set permissions. Hit the refresh button and search for your new policy name. Check the box next to it and hit Next.
No tags. Create User.

Be sure to copy the access key and secret key from the next screen and then hit Close.

Configure Elasticsearch Nodes

You’ll have to perform the following on each of your Elasticsearch nodes.

SSH to the first node and execute the following to install the S3 plugin.
sudo bin/elasticsearch-plugin install repository-s3

You should see the following:

[root@1375dd8ff618 bin]# elasticsearch-plugin install repository-s3
-> Downloading repository-s3 from elastic
[=================================================] 100%?? 
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,resolve
* java.util.PropertyPermission es.allow_insecure_settings read,write
See http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Continue with installation? [y/N]y
-> Installed repository-s3

We now need to add our AWS IAM credentials to the Elasticsearch keystore with the following commands.

bin/elasticsearch-keystore add s3.client.default.access_key  
bin/elasticsearch-keystore add s3.client.default.secret_key  

You should see the following:

[root@1375dd8ff618 bin]# elasticsearch-keystore add s3.client.default.access_key
Enter value for s3.client.default.access_key: 
[root@1375dd8ff618 bin]# elasticsearch-keystore add s3.client.default.secret_key
Enter value for s3.client.default.secret_key: 

If you paste in a value you won’t see it appear on the screen, don’t worry it’s there, just hit Enter after you paste it.

Now reboot this node and make sure the cluster health is back to green before moving on to the next node.

Once all the nodes have rebooted and the Cluster state is green open Kibana Dev Tools and run:
POST _nodes/reload_secure_settings

Configure the Repository and Policy

Note that I just intend to backup my filebeat-* indices with the policy below. If you want to backup all your indices it’s even simpler as that’s the default.

In Kibana navigate to Management > Snapshot and Restore.
Select the Repository tab and press Register a Repository.
You should see AWS S3 listed next to “Shared file system” and “Read-only URL”.

Give the repository a name (I chose “filebeat-repo”) and select AWS S3.
Fill in the AWS Bucket name (For me that’s “filebeat-backup”).
It will use a client name of default by default which is fine since that’s the key you set in the keystore earlier.

Save the repo and hit the Verify button. It should show all of your nodes as Connected.

Now in Kibana go to the Policies tab under Snapshot and Restore.
Click Create a Policy.

Name: daily-filebeat
Snapshot name: <daily-filebeat-{now/d}>
Repository: filebeat-repo
Schedule: default (9:30 PM)

On the snapshot settings page de-select All Indices and typed in filebeat-*. You can choose to leave All Indices selected.
Be sure you’re selecting Index Patterns and not individual Indices when you do this.

Create the policy.

On the Policies tab click the play button next to your new policy in order to run it now.

Once complete, if you go back to the AWS console and check the bucket Overview tab you should see new files.

Restoring

Let’s test restoring a single Index to a new Index name.

Go to the Snapshots tab and to the far right click the down arrow next to the trashcan to start a restore.

I’m selecting just a single Index and underneath I’m selecting the option to rename indices. I then supply the original name pattern and my new name. Note that I used restored-filebeat but should have used restored-filebeat-.

Start the restore and watch the progress.

You’ll see when the restore is complete.

Now go check that the new Index name exists.

I then created a Kibana Index pattern for restore-* and after adjusting my timeframe verified that all the logs were present.

Siemplify SOAR to Elastic Common Schema Mapping

2019-08-26T12:00:00-04:00

This is a sample mapping of connector, entity, and system fields required by Siemplify to source events in Elasticsearch mapped to the Elastic Common Schema.

For this to be valid you must be storing your logs in Elasticsearch using ECS as well.

ECS is essentially a field naming convention adopted by Elasticsearch for use within its products. As the engineers at Elastic develop new dashboards (like for the new SIEM App in 7.2), and new machine learning jobs (a SIEM App improvement in 7.3), they will do so using ECS defined fields. All of the 7.x Beats agents already use ECS to name their fields (and provide a good reference to those just starting with ECS). What this means for the end-user is that if you map your incoming logs to use ECS field names before the document is stored in Elasticsearch, then many of the dashboards and machine learning jobs will work without any further effort on your part. If you decide instead to keep your current field names then you’ll have to re-map all the existing visualizations and ML jobs to your custom fields.

So now that you’re storing log fields in Elasticsearch using ECS we need to map those fields to the Siemplify schema. What follows is my suggested mapping, subject to change.

You’ll notice that some Siemplify fields reference the same ECS field, such as DestinationDnsDomain and DestinationNtDomain. In these cases it may be necessary to use the Siemplify transform function EXTRACT_BY_REGEX in order to capture a subset of the ECS field to use.

I also have a spreadsheet available for download.

Update Aug 27: Having just discovered that ECS version 1.1 was recently released I’ve made some modifications to the process and file hash field mappings.

Siemplify Connector Setting	ECS Field
Product Field Name	event.module
Event Field Name	event.category
Alert Name Field	event.action
Timestamp Field	@timestamp

Siemplify Entity Mapping	ECS Extracted Field	ECS Alternative Field	ECS Alternative Field
SourceUserName	user.name
SourceHostName	source.hostname	client.hostname
DestinationUserName	user.name
DestinationHostName	destination.hostname	server.hostname
SourceAddress	source.ip	client.ip
DestinationAddress	destination.ip	server.ip
SourceProcessName	process.pid	process.name
DestinationProcessName	process.pid	process.name
DestinationURL	url.full	url.original
FileName	file.name
FileHash	hash.md5	hash.sha1	hash.sha256
EmailSubject	email.subject
USB
Deployment
DestinationMacAddress	destination.mac	server.mac
SourceMacAddress	source.mac	client.mac
CreditCard
PhoneNumber
CVE
ThreatActor
ThreatCampaign
ThreatSignature
GenericEntity

Siemplify System Field	ECS Extracted Field	ECS Alternative Field	ECS Alternative Field
StartTime	event.start	@timestamp
EndTime	event.end
Message	message	event.original	log.original
Name	event.action
CategoryOutcome	event.outcome
DestinationDnsDomain	destination.domain	server.domain
DestinationNtDomain	destination.domain	server.domain
DestinationPort	destination.port	server.port
SourceDnsDomain	source.domain	client.domain
SourceNtDomain	source.domain	client.domain	user.domain

Elasticsearch snapshots to Windows share

2019-07-29T12:00:00-04:00

I recently had a need to snapshot the data from an Elasticsearch cluster to a Windows share on the network. I encountered some issues so I’ll post the steps that worked for me below. I’m running Elasticsearch on Ubuntu 18.04.

The Elasticsearch instructions for a shared file system repository seem to indicate that you can use a UNC path directly but I didn’t find this to be the case. I ended up mounting the Windows share locally with fstab and then referencing the local mount point. Note that you have to mount the share and modify elasticsearch.yml on each Elasticsearch node in your cluster.

First we need to make sure the necessary packages are installed.
apt install cifs-utils

Now we need to find the User ID of our Elasticsearch user. You will use this in the fstab entry.
id -u elasticsearch. For me this ID is 112.

Let’s create out local mount point.
sudo mkdir /mnt/elastic

Before we modify fstab our Windows share should be accessible. My share name is elastic and I created a new Windows user named esbackup that has write permissions to the share and file system.

Now on the Elasticsearch host we modify fstab.
sudo nano /etc/fstab

Add the following line.
//<windows_server_IP/<share_name> /mnt/elastic cifs user=esbackup,password=password,uid=112,vers=3.0 0 0

Replace the 112 above with the UID of your elasticsearch user.

Now mount the share.
sudo mount -a

Test the mount by navigating to the share and creating a test file.

Add path.repo in elasticsearch.yml.
path.repo: ["/mnt/elastic"]

Restart elasticsearch service (on each node).
systemctl restart elasticsearch

Now login to Kibana and navigate to Dev Tools. We’re going to create a snapshot repository at our new mount point.

PUT /_snapshot/my_backup_repo
{
  "type": "fs",
  "settings": {
    "location": "/mnt/elastic",
    "compress": true
  }
}

Hopefully acknowledged: true is returned.

Now check to make sure the repository is registered.
GET /_snapshot/_all
You should see essentially the same JSON block that you entered a moment ago returned back to you showing the repository name and location.

Now write your first snapshot.
PUT /_snapshot/my_backup_repo/%3Csnapshot-%7Bnow%2Fd%7D%3E?wait_for_completion=true

If you’re following along in the elasticsearch.log you’ll see the following line.
[INFO ][o.e.s.SnapshotsService ] [snap1] snapshot [my_backup_repo:snapshot-2019.07.26/Bw8UwYFoRcmGvbrTPs7wvw] started

You can check the status of the snapshot with the following command. Note that the way I’m naming the snapshot causes the date to be appended.
GET /_snapshot/my_backup_repo/snapshot-2019.07.26

If you check the Windows folder you should see lots of new files.

Siemplify SOAR Overview

2019-07-23T12:00:00-04:00

I’ve been fortunate to have spent the last few months working with Siemplify and watching it evolve from version 4 to the recently released version 5. Siemplify is a SOAR solution which stands for Security Orchestration, Automation, and Response. It enables case management and the running of Playbooks on incoming alerts, with the goal of reducing the amount of time an analyst must spend on repetitive tasks which can be easily automated. After an enrichment phase the analyst can better make quick, informed decisions as to how to proceed, and can be assured that actions performed are as expected without the possibility of forgetting or missing a step.

Below I’ll do a high-level introduction of Siemplify’s main tabs, and in later posts I’ll dive deeper into playbook creation and case management.

We’ll start with the Analyst Homepage. On this screen each analyst can track their assigned tasks and cases, as well as setup shortcuts to their frequently accessed files and links. A scrolling announcements side-bar allows the SOC Manager to keep the whole team up-to-date with current events.

Siemplify requires a connector to an event source in order to ingest alerts. Numerous connectors are available to various SIEM platforms, as well as Email mailboxes, ServiceNow, and for many of the products listed in their Marketplace.

Below I’m connecting to Elasticsearch 7.2. Along with server and user connection details we also do some preliminary field mapping of product, event, and alert name fields. These can be pulled from the testing tab after a test connection has been established.

Once a connector has been established to an alert source, some additional field mappings must take place. Siemplify has its own internal schema and requires that the fields from the alert be mapped to its entity fields. Example entity fields would be SourceUserName, DestinationHostName, DestinationDomain, FileHash, URL, etc. Along with this field mapping you must select a Visual Model which best represents the relationship of the entities. This Visual Model determines how the various entities are connected together. For example, in most Visual Models SourceHostName and SourceAddress would be linked because the machine’s IP address is linked to its name.

Now that we’re properly parsing our alerts it’s time to start working with them. In order to do so we must first visit the Siemplify Marketplace (from within the application) and install all the Integrations we want to use. Some will require very little configuration while most will require an API key and credentials.

Navigating to the Cases tab will show us all the cases that are open within Siemplify, regardless of the analyst assigned. From here we can take ownership of a case and begin to work it.

The Overview tab will show us one or more alerts grouped within the case. These alerts correspond to events listed on the Events tab. We can see a timeline display of the events as well as any Insights that have been called-out from running a playbook. The Playbook pane on the bottom shows us the playbooks attached to the alert as well as their status.

Entities mapped earlier are shown in the right-hand column along with additional case details. Clicking on an entity will take you to it’s details page where you can see statistics about where this entity has been seen before.

The playbooks that run against alerts are designed on the Playbooks tab. Each playbook must contain one trigger and at least one action. Actions are scoped to entities. So you may start with the Active Directory action to “Enrich Entities” and scope that to just “Internal User” entities. Siemplify is able to make this determination because during setup you told it about all the Domain names you own. So when it sees sallie@corp.local and you have corp.local defined as an internal Domain, it knows Sallie is internal. The same method can be used for nework actions such as Palo Alto’s “Add IP to Block Group” action. We scope this to “External IPs” and Siemplify will make that determination based on the network ranges you have defined.

When a playbook action runs, such as Active Directory’s “Enrich Entity”, it results in Siemplify gaining additional data about an entity (enrichment). With this method we gain additional data points such as LastLogonTime. Enriching through your AV solution may allow you to associate a machine name to the IP from the alert, as well as list the last scan results. Based on the result of the last scan we can add a branching conditional to our playbook to execute one sequence of actions if the scan reports clean, and a different set of actions if there’s an infection. If the machine is infected we may want to contain it until the analyst can review the case. This is how you build out a playbook. Ideally your procedure for handling each alert is already defined and Siemplify is merely automating the execution of actions you’ve previously been doing manually.

You can automate as much or as little as you want on an alert. Many environments will want to run an enrichment playbook to ensure as much data as possible is attached to the case when the analyst begins their investigation. Some alert categories may need some automated decisions to occur to filter down their numbers before an analyst sees them, and others can be fully automated away.

As you can see below, building the playbook means dragging the desired action from the left-hand column and placing it in the proper step in the sequence. Then it’s just a matter of ensuring you have the right information coming from the action outputs to feed into the next action inputs.

Version 5 of Siemplify includes a new expression builder component. With the expression builder you can look at the JSON results of a previous action in the playbook and target its fields directly.

The image below is the example output from the McAfeeNSM “Get Alert Data” action. This allows me to select any placeholder field from the list when designing my playbook, allowing me to see what the data in that field will look like for a real alert. This is very useful if for example I want to select a specific field from the previous action’s output as input to my current action.

The final components to note are the Dashboards page and the new War Room.

Dashboards and Reports allow you to keep track of your SOC metrics and guage your analyst’s closure rate. Siemplify includes both of these capabilities but I won’t explore this much other than to show the example dashboard below. With the next release all the dashboarding and reporting will be changing for the better so stay tuned.

The War Room is a new feature allowing the SOC to “Raise an Incident” and coordinate activities with outside parties. The key here is the ability to invite anyone, anywhere into the War Room where they can stay apprised of what’s happening as well as contribute their updates. The people you invite don’t need to have Siemplify accounts or licenses, but are still able to share their updates with the group. The War Room administrator can set check-in deadlines for status report updates and update all departments through one interface. It’s completely up to you under what circumstances you choose to start a new incident.

I hope you enjoyed this overview. In future posts I’ll dive deeper into specific playbooks to really see how automation can allow analysts to focus on the most important cases. We’ll also explore ways of gracefully handling those less-critical alerts that would otherwise go unhandled.

High-level PCAP Analysis

2019-06-30T12:00:00-04:00

PCAP files play a critical role in network troubleshooting and security. When an issue arrises and Developers are pointing at the SysAdmin who are pointing at the Network Admins, a PCAP capture will give you the unbiased answer. Depending on the amount of traffic and length of the capture however, these files can be extremely daunting to look at when first opened. Where do you start in a capture with 200,000 entries?

Often you don’t need to dig into individual packets to get the answer you’re looking for. Even if you do end up there, you still need a place to start. In this post I’ll be looking at three different ways to get a high-level overview of the traffic in your PCAP. The goal is to give you a sense of who the high traffic endpoints are, where they reside, and who they’re communicating with.

GrassMarlin

The first tool we’ll be looking at is GrassMarlin, an open source tool released by the NSA. It’s primary purpose is to help passively map Industrial Control Systems and SCADA networks, and that’s what it’s fingerprint database is centered around, but it works just fine for our purpose as well.

It’s a Java application and can be installed on Windows or Linux. I won’t go over the installation as I just selected all the defaults. The full documentation is available here.

I’m going to assume you have a PCAP file you want to analyze. I just downloaded a random one off the Internet.

Once the application is running, click the “Import Files” button in the toolbar or from the file menu. Click “Add Files” to add it to Pending Imports. Highlight it and select “Import Selected”.

The first view you will see is the Logical Graph. Here each node that communicated in the PCAP is represented. The red circles are groupings. By default the Nodes are grouped by network, but this can be changed.

I like to modify the view a little.
Right-click the Logical Graph and select “Weight Edges by Byte Count”. The darker the line, the more traffic was transmitted.
Right-click the Logical Graph and select “Use Curved Edges”

Other than the Logical Graph, you also have tabs for Physical Graph and Sniffles. Physical Graph requires some additional configuration. Basically you export your running-config from your Cisco devices and import them into GrassMarlin. Then, under this tab, it’s able to show connections grouped by Switch and Router.
The Sniffles tab is extracted from 802.15.4 data contained in the PCAP. This is meant to group devices on a Mesh network. The necessary metadata may not be present in a default PCAP.

Other than the Graph windows you also have a “Message Log” that will display updates and errors at the bottom left, and above this is a tree view that mirrors what you see in the graph.

After having analyzed this view, right-click anywhere in the Logical Graph and change “Group By” to Host Name. This will essentially remove the red coloring of groups on the map. Now right-click the graph again and select “Run Layout Now”. Unless you change the grouping first the second command does nothing.

This is the view that best allows me to visually interpret the full contents of the PCAP. I can clearly see a node on the bottom left communicating with lots of peers, as well as a dark line almost through the center showing a high amount of traffic passing between two nodes. Then there’s also the two outliers off to the right. All of these data points can allow me to hone in on the important traffic more easily. Maybe this is traffic that needs to be analyzed further, or maybe these are sources of noise preventing me from finding the interesting data, in which case excluding them from further analysis will probably trim the number of remaining packets by more than 50%.

NetworkMiner

The next tool we’ll look at, NetworkMiner, can only be installed on Windows. I’m also using the free version as opposed to the more full-featured commercial version. An installation isn’t even required, simply unzip and run the executable.

We’ll be using NetworkMiner for it’s sorting and categorization capabilities. By default when you first open your PCAP you’re presented with the hosts sorted by IP address.

Just under the tabs you’ll see “Sort Hosts On:”. Select “Sent Packets (descending)” to see your chattiest endpoints. There’s obviously many other sorting options under here as well.

You’ll also notice other tabs across the top. Here NetworkMiner has followed and categorized conversations between endpoints and attempted to extract and reconstruct the contents. These tabs allow you to see what was transferred and what category or tab those communications fall under. Most likely some of these tabs are empty, but quickly glancing at the numbers next to the tab headers allows you to see what the bulk of the traffic spent time transferring.

Extracting credentials is a nice feature also.

Wireshark

Last but certainly not least is our favorite PCAP analysis tool of all, Wireshark.

For high-level analysis we’ll focus on the Statistics menu at the top.

Statistics > Summary
Look here under “Time” to see the duration of the capture.

Statistics > Protocol Hierarchy
This will show what type of network traffic you’re dealing with. Most likely this will be IPv4 TCP traffic but it doesn’t hurt to check.

Statistics > Endpoints > TCP
Here you can sort by port and packet count. Useful for honing in on the ports and services doing most of the talking.

Statistics > IO Graph
This will show you the distribution of packets over time. Is the distribution consistent or are there bursts of traffic you may want to focus on?

Statistics > IP Statistics > Source and Dest IP Addresses > Create Stat
Another way of sorting the endpoints by packet count. Who sent the most traffic and who received the most.

Remember that the goal of this exercise is to enable you to focus on the traffic that matters most to you. By taking the noisiest sources and either including or excluding them from your filter you’ll be best able to find the IPs and conversations that matter. Once you’re done filtering and have a list of IPs of interest, I like to go back and undo my existing filters and now filter by IP. I may have excluded a noisy endpoint early on and wouldn’t see if my suspect IP ever communicated with those excluded endpoints.

On first pass filter often and find those suspect IPs. On the second pass focus on those suspect IPs and everyone they communicated with.

Open Source SIRP with Elasticsearch and TheHive - Part 6 - Case Management

2019-05-24T12:00:00-04:00

We now have a working pipeline starting with an alert being triggered at our endpoint, through escalating that alert into TheHive. Once we have an alert we can begin the process of case creation, task assignment, IoC enrichment, and ultimately case closure. Let’s walk through this process in more detail.

Creating Cases from Alerts

You should begin by navigating to Users in the Admin menu to create unique accounts for all Analysts.

Click on the Alerts button at the top of the page. Here you’ll see a list of unassigned alerts waiting to be picked up by any available Analyst. Select an alert by clicking the Preview and Import icon off to the right. Here you’ll be able to see the alert details, along with the list of extracted observables. A Similar Cases section will appear at the bottom if any observable in this case has been seen before. If the two alerts share a link then you can opt to add this alert to an existing case instead of generating a new one.

There is an Import alert as drop down in the bottom right, next to the Import button. This allows you to assign a case template that will be used for case creation. Templates contain lists of predetermined tasks that should be preformed on the alert, more on that below. For now we can create an empty case.

Finally, choose Yes, Import to turn the alert into a case that will be assigned to you.

Case Management

TheHive allows Analysts to work together to complete tasks and close cases. Tasks and cases both support assignment to clearly differentiate who is responsible for what, while allowing everyone to follow along through the live stream.

Access your cases by clicking on TheHive logo in the top left corner.

Cases have 3 tabs in the main window (Details, Tasks, and Observables) as well as a live stream on the right-hand side showing task and status updates from all analysts.

The Details page shows metadata related to the case such as tags, date, severity, related cases, a description, and TLP and PAP designations.

TLP is the Traffic Light Protocol which uses 4 color codes to indicate boundaries of how far outside the original group or recipient the information may be shared.

An example provided by TheHive website is: “For example, a file added as observable can be submitted to VirusTotal if the associated TLP is WHITE or GREEN. If it’s AMBER, its hash is computed and submitted to VT but not the file. If it’s RED, no VT lookup is done.”

PAP is the Permissible Actions Protocol which mimics the TLP but indicates to the analyst how they may use the IoC in investigating the alert. It dictates actions that may be taken with each IoC, such as active vs passive response.

The Tasks tab shows analyst defined tasks or those defined in an attached case template. Tasks should be used to track the actions taken to answer investigative questions. Tasks that you accept or which are auto-assigned to you show up in My Tasks at the top of the page. Tasks which have not yet been assigned are under Waiting Tasks.

The Observables tab shows all extracted observables and their type. The observable will have few if any tags until enrichment occurs. Click the observable value under Value/Filename (i.e. click on the IP or HTTP link) to open its details page. Here you’ll get additional metadata as well as links to other cases where the IoC is also present, and an Analysis section to run your Analyzers for enrichment.

After running the Analyzers return to the Observables tab. Your IoC should now include tags summarizing the results.

For more detailed results return to the IoC details page where the Analyzers are listed that you ran. Under Last Analysis click the date to view a more detailed report of the scan results.

Case Templates

Under the Admin menu at the top select Case Templates.

Here you’re able to give initial designations (Severity, TLP, PAP) for the IoCs in this alert category. You also provide a description and a task list to outline the investigative steps for this alert. This provides a consistent approach to handling events since the Task List becomes you’re investigative playbook. These should also reflect the actions defined in your SOPs.

You’ll also see sections for Metrics and Custom fields. Items you select here must first be defined in the respective Case Metrics and Case Custom Fields sections under the Admin menu.

A case metric is merely a variable you define to increment. Metrics can also be displayed in graphs on the Dashboard.

A case custom field allows you to add additional fields for an Analyst to fill-in with the response as either a string drop-down, number, boolean, or date.

Responders

Responders are part of Cortex and are installed along with the Cortex-Analyzers.

Responders allow you to automate initiating an action. The default Responder provided as an example is the Mailer Responder which allows you to e-mail the case information and IoCs.

Enable the Mailer responder by adding the Responder section to the end of /etc/cortex/application.conf. This requires you to have downloaded the Cortex-Analyzers repository.

responder {
  path = ["/etc/cortex/Cortex-Analyzers/responders"]
  # Sane defaults. Do not change unless you know what you are doing.
  fork-join-executor {
    # Min number of threads available for analysis.
    parallelism-min = 2
    # Parallelism (threads) ... ceil(available processors * factor).
    parallelism-factor = 2.0
    # Max number of threads available for analysis.
    parallelism-max = 4
  }
}

Now login as the orgAdmin and configure and enable the Mailer responder.

The default repository for Responders is here.

Two additional Responders are also provided.
Cisco Umbrella Blacklister
Add domain from observables in cases to Umbrella blacklist.
https://umbrella.cisco.com/

Crowdstrike Falcon
Submit observables from alerts and cases to the Crowdstrike Falcon Custom IOC Service.
https://www.crowdstrike.com/endpoint-security-products/

If you find yourself with an abundance of free time you can always write your own Responder.

For instance if someone wanted to write a Responder to interface with StackStorm that would open a world of possibilities.

Case Closure

When you’re ready to close out a case click the Close button in the title bar of the case.

This will bring up the case resolution screen.

Dashboards & Search

Built-in dashboards for Case statistics, Observable statistics, Job statistics, and Alert statistics.

You can also build your own custom dashboards through a drag-and-drop interface.

Search across Cases, Tasks, Task logs, Observables, Alerts, Jobs, and Audit logs.

Synapse, API, & TheHive4py

TheHive Project needs the help of the community to build additional Responders, integrations, and improvements, but makes it easy to do so with the following:

Most actions in TheHive are accessible over the REST API if you’re looking to connect or send from an external application.

If programming for TheHive or an application in the ecosystem, TheHive4py is the Python API client you’ll want to use.

Another project from TheHive creators is Synapse. Synapse is a “meta alert feeder for creating alerts in TheHive”. It’s a framework that can be expanded upon to ingest alerts from various sources. Currently it supports Microsoft Exchange, O365, and IBM QRadar. There are others in TheHive community building “connectors” to other SIEM products.

It’s been enjoyable working through TheHive stack and seeing firsthand how this could successfully be used within an organization to implement an SOC with as few as one or two Analysts. I look forward to building on this project even more by now attempting to integrate StackStorm to develop a full-fledged SOAR solution.

Phishing Email Pipeline with imap2thehive

2019-05-13T12:00:00-04:00

Today I’ll show you how to use imap2thehive to pull emails from a mailbox, extract as many unique observables as possible, and generate a case in TheHive. It won’t be a long post as the author of imap2thehive has done an excellent job with his script and some small configuration changes are all that are required.

I’m assuming you already have a running instance of TheHive. If not, start with my post on TheHive installation.

Create a folder named imap2thehive and grab the files we need.

mkdir imap2thehive
cd imap2thehive/
wget https://github.com/xme/dockers/blob/master/imap2thehive/requirements.txt
wget https://github.com/xme/dockers/blob/master/imap2thehive/imap2thehive.whitelists
wget https://github.com/xme/dockers/blob/master/imap2thehive/imap2thehive.py
wget https://github.com/xme/dockers/blob/master/imap2thehive/imap2thehive.conf

Install the requirements.
pip install -r requirements.txt

Now go to your instance of TheHive and create a new user for these alerts. Generate an API key for the user. Note that this is the user that will be listed as the case Assignee.

Modify imap2thehive configuration settings.
nano imap2thehive.conf

Modify the connection settings for your mailbox. I’m connecting to GMail.

[imap]
host: imap.gmail.com
port: 993
user: <username>@gmail.com
password: <Password goes here>
folder: inbox
expunge: false
spam: (X-Spam-Flag: YES)

Note the renamed folder:

Modify the connection settings to find TheHive.

[thehive]
url: http://x.x.x.x:9000
apikey: <Paste API key here>
observables: true
whitelists: imap2thehive.whitelists

Under [alert] change the tlp: from 3 to 2. Having such a high alert prevents some analyzers from running in Cortex. Do the same under [case].

Create a new case template in TheHive for these emails or use an existing template. Replace template: thehive_template with the name of your template.

Also under [case] you’ll want to list all the mime types you want to ingest as observables for further analysis. GMail is perhaps different than other mail systems in this regard. I’m using files: application/octet-stream since trial-and-error told me that’s what I wanted.

To discover this I modified imap2thehive.py. After line 238 I added a line with print(filename), and after the mimetype = line I added a new line with print(mimetype). this will show you the mime type info in the console as the program runs.

    else:
        # Extract MIME parts
        filename = part.get_filename()
        print(filename)
        mimetype = part.get_content_type()
        print(mimetype)
        if filename and mimetype:

Now you’re ready to run the script. I’d have just a single unread email with some IPs and URLs in the message body waiting in the mailbox you’re testing against.

python3 imap2thehive.py --config imap2thehive.conf

[WARNING]: Both case template and tasks are defined. Template (email_template) will be used.
[INFO]: Processing <username>@gmail.com@imap.gmail.com:993/inbox
[INFO]: Connected to IMAP server.
[INFO]: 1 unread messages to process
[INFO]: From: Arnaud <xxxxxx@arnaudloos.com> Subject: Test Email
None
multipart/mixed
None
multipart/alternative
Evilpdf.pdf
application/octet-stream
[INFO]: Found attachment: Evilpdf.pdf (application/octet-stream)
Maliciousdoc.docx
application/octet-stream
[INFO]: Found attachment: Maliciousdoc.docx (application/octet-stream)
MaliciousExcel.xls
application/octet-stream
[INFO]: Found attachment: MaliciousExcel.xls (application/octet-stream)
[DEBUG]: Found observable url: https://maliciousurl.com/files/content/page.hta
[DEBUG]: Found observable ip: 103.14.229.253
[DEBUG]: Found observable ip: 103.25.58.34
[DEBUG]: Found observable ip: 1.186.77.13
[DEBUG]: Found observable ip: 73.164.105.200
[DEBUG]: Found observable domain: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[DEBUG]: Found observable domain: maliciousurl.com
[DEBUG]: Found observable mail: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: https://maliciousurl.com/files/content/page.hta
[INFO]: Ignoring duplicate observable: 103.14.229.253
[INFO]: Ignoring duplicate observable: 103.25.58.34
[INFO]: Ignoring duplicate observable: 1.186.77.13
[INFO]: Ignoring duplicate observable: 73.164.105.200
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: maliciousurl.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: https://maliciousurl.com/files/content/page.hta
[INFO]: Ignoring duplicate observable: https://maliciousurl.com/files/content/page.hta
[INFO]: Ignoring duplicate observable: 103.14.229.253
[INFO]: Ignoring duplicate observable: 103.25.58.34
[INFO]: Ignoring duplicate observable: 1.186.77.13
[INFO]: Ignoring duplicate observable: 73.164.105.200
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: maliciousurl.com
[INFO]: Ignoring duplicate observable: maliciousurl.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: https://maliciousurl.com/files/content/page.hta
[INFO]: Ignoring duplicate observable: 103.14.229.253
[INFO]: Ignoring duplicate observable: 103.25.58.34
[INFO]: Ignoring duplicate observable: 1.186.77.13
[INFO]: Ignoring duplicate observable: 73.164.105.200
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: maliciousurl.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: https://maliciousurl.com/files/content/page.hta
[INFO]: Ignoring duplicate observable: 103.14.229.253
[INFO]: Ignoring duplicate observable: 103.25.58.34
[INFO]: Ignoring duplicate observable: 1.186.77.13
[INFO]: Ignoring duplicate observable: 73.164.105.200
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: maliciousurl.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: https://maliciousurl.com/files/content/page.hta
[INFO]: Ignoring duplicate observable: https://maliciousurl.com/files/content/page.hta
[INFO]: Ignoring duplicate observable: 103.14.229.253
[INFO]: Ignoring duplicate observable: 103.25.58.34
[INFO]: Ignoring duplicate observable: 1.186.77.13
[INFO]: Ignoring duplicate observable: 73.164.105.200
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: arnaudloos.com
[INFO]: Ignoring duplicate observable: maliciousurl.com
[INFO]: Ignoring duplicate observable: maliciousurl.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Ignoring duplicate observable: xxxxxx@arnaudloos.com
[INFO]: Removed duplicate observables: 72 -> 8
[DEBUG]: Searching for \S*(ALERT|VTMIS)\S* in 'Fwd: test 18'
[INFO]: Created case 29
[INFO]: Added observable /tmp/Evilpdf.pdf to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable /tmp/Maliciousdoc.docx to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable /tmp/MaliciousExcel.xls to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable url: https://maliciousurl.com/files/content/page.hta to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable ip: 103.14.229.253 to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable ip: 103.25.58.34 to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable ip: 1.186.77.13 to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable ip: 73.164.105.200 to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable domain: arnaudloos.com to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable domain: maliciousurl.com to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Added observable mail: xxxxxx@arnaudloos.com to case ID AWoEXp2TI5SmE1Wz2WvB
[INFO]: Message 33 successfully processed and flagged as read

Notice how the script elegently trims the number of observables down to just the unique entries.

You should now see a newly created Case in TheHive dashboard.

WSUS Troubleshooting Steps

2019-05-06T12:00:00-04:00

Below is the guide I use when troubleshooting a broken WSUS installation. This can manifest as a server console error, the ever popular “it’s just not reporting in”, or through the event log. I’ll walk you through the components of WSUS and how to check and make sure each one is functioning properly.

First a note on what’s available on your platform.
If you’re on Windows 7/8/Server 2008R2/2012R2 then wuauclt works for you.
The only real command you need to know is wuauclt /resetauthorization /detectnow.
wauauclt /reportnow doesn’t do what you think and isn’t very useful.

If you’re on Windows 10/Server 2016 then wuauclt has been depricated and you have UsoClient.
usoclient.exe startscan to detect missing patches
usoclient.exe refreshsettings to refresh settings if any changes were made
usoclient.exe startdownload to download patches
usoclient.exe startinstall to install patches

You also have the option of using Powershell to initiate a scan request.

(New-Object -ComObject Microsoft.Update.AutoUpdate).DetectNow()

Speaking of Powershell, Microsoft makes a number of cmdlets available for managing WSUS Server. To see if these are available to you run

Get-Command -Module UpdateServices

These cmdlets are primarily geared towards deployment and management of the server, patches, and computers, not troubleshooting.

On the client there is a WindowsUpdate module but its only function is Get-WindowsUpdateLog which is used for generating a readable WindowsUpdate.log file in Windows 10.

There exists a third-party PS module for managing Windows Update on the client.

Install-Module PSWindowsUpdate  
Import-Module PSWindowsUpdate  
Add-WUServiceManager -ServiceID 7971f918-a847-4430-9279-4a52d1efe18d  
Get-Command -Module PSWindowsUpdate  
Get-WUList –MicrosoftUpdate  
Get-WUInstall –MicrosoftUpdate –AcceptAll –AutoReboot           

Database

WSUS has the option of utilizing an internal WID (Windows Internal DB) or SQL database. I’m going to assume it’s installed with WID at the default path of C:\Windows\WID\.
Database related errors (other than fragmentation) usually occur at the time of installation, often during the post-installation task sequence.
While it is possible to connect to the WID and drop the SUSDB, errors at this stage are usually best fixed through a complete uninstall and reinstall.
For reference purposes I will state that you can use either SQLCMD or SQL Mgmt Studio to connect to the DB.
The connection string is np:\\.\pipe\MICROSOFT##WID\tsql\query

select name from sys.sysdatabases
drop table susdb
select name from sys.sysdatabases

Uninstallation

Uninstall using Server Manager.
Make sure the WSUS content folder is gone as well as the IIS WSUS Administration site.
Uninstalling WSUS doesn’t always remove the WID. Remove it in Powershell with Uninstall-WindowsFeature -Name windows-internal-database
Reboot and delete c:\windows\WID\SUSDB.mdf and SUSDB_log.ldf files.

Installation

Along with selecting the WSUS Role, Windows installs ASP.NET 4.6, RSAT Tools, IIS, and the Windows Process activation service.

Install from an administrative Powershell prompt

Install-WindowsFeature -Name Updateservices,UpdateServices-WidDB,UpdateServices-services -IncludeManagementTools

Afterwards, before launching the GUI, run:

C:\Program Files\Update Services\Tools\WsusUtil.exe PostInstall CONTENT_DIR=<Directory_Path_Here, i.e. D:\WSUS>
Change the CONTENT_DIR to the correct path.

Check the client

There’s quite a few potential causes to a client not receiving updates. I’ll go through a range of troubleshooting steps and at the end give two scripts to “reset” the client if nothing else works.

Check free hard drive space.

Check the Windowsupdate log.
On Windows 10 open an administrative powershell prompt and run Get-Windowsupdatelog. The log will be on the desktop. Wait for the command to finish running.
On Windows 7 check C:\Windows\WindowsUpdate.log
This should indicate whether the issue is client based or a problem communicating to a remote endpoint.

The simplest first step in troubleshooting the client is to run the Windows Update Troubleshooter.
On Windows 10 go to the Microsoft iFixIt for WSUS site.
This will prompt you to download the Windows Update Troubleshooter (in cab file format). Save this and run.
Choosing Windows 7 on this page prompts you to right-click Network (in your system tray) and “Troubleshoot problems”. You can probably skip this step.

On Windows 10 try an online update check.
https://support.microsoft.com/en-us/help/4027667/windows-update-windows-10

Check to make sure the service is running.
Get-Service -Name wuauserv

Next check and make sure client is receiving WSUS settings from Group Policy.
Run gpresult /scope computer to see if your WSUS settings policy is being applied to the machine.
Query client registry for WSUS settings with Get-ItemProperty HKLM:\Software\Policies\Microsoft\Windows\WindowsUpdate

Telnet can be used to check port connectivity issues. If you attempt a telnet connection to an open and accessible port telnet will open a new blank command window. If the port is inaccessible the command will simply fail. Try and establish a connection to the WSUS server and port. If the connection fails check firewall settings.
telnet wsus.server.com 8530

Let’s use Powershell to test the connectivity.
Test-NetConnection -ComputerName <WSUS_Server> -Port 8530 -InformationLevel Detailed

Don’t forget the obvious step of checking the Event log.
Application Event log as well as App and Service Logs > Microsoft > Windows > WindowsUpdateClient

Sometimes anti-virus or other endpoint security agents can interfere with network communications. Consider disabling or uninstalling these for testing.

Run sfc /scannow from an administrative command prompt to check for file corruption that can affect the client.

If you have both working and non-working clients in WSUS check c:\program files\update services\WebServices\ClientWebServices for a web.config file and compare a working file to a non-working file for differences.

If this client comes from an OS image and Sysprep wasn’t run then the issue may be that multiple clients are using the same SUSClientID key.
Check HKLM\Software\Microsoft\Windows\CurrentVersion\WIndowsUpdate and delete the current SUSClientID.
Run wuauclt /resetauthorization /detectnow from an elevated command prompt.

The sequence below will clear out the local cache of the Windows Update client

attrib -h -r -s %windir%\system32\catroot2
attrib -h -r -s %windir%\system32\catroot2\*.*
net stop wuauserv
net stop CryptSvc
net stop BITS
ren %windir%\system32\catroot2 catroot2.old
ren %windir%\SoftwareDistribution SoftwareDistribution.old
ren "%ALLUSERSPROFILE%\application data\Microsoft\Network\downloader" downloader.old
net start BITS
net start CryptSvc
net start wuauserv

The sequence below will re-register a machine with the WSUS Server:

net stop wuauserv  
net stop bits  
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate" /v AccountDomainSid /f  
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate" /v PingID /f  
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate" /v SusClientId /f
reg delete "HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\WindowsUpdate" /v SusClientIDValidation /f
rd /s /q "C:\WINDOWS\SoftwareDistribution"
net start bits
net start wuauserv
wuauclt /resetauthorization /detectnow (or usoclient.exe RefreshSettings)
PowerShell.exe (New-Object -ComObject Microsoft.Update.AutoUpdate).DetectNow()

Check the Server

Check free hard drive space, for the boot volume and repository volume.

Check the Windows Update service
Get-Service -name WsusService

Check the IIS service
Get-Service -name W3SVC

Is server listening? Check open ports.
netstat -an | findstr 853*

Can you browse to http://server:8530/ClientWebServices/client.asmx?
You should see a blue and tan Client Service info page.
If not, we can try resetting the port. Open an elevated command prompt and run wsusutil usecustomwebsite false. This will change the port WSUS uses from 8530 to 80, so make sure nothing is running on port 80.
Now run wsusutil usecustomwebsite true followed then by iisreset /restart. This changes the port back to 8530 and “resets” the configuration.

Check firewall rules.
Check server side logging: c:\program files\updateservices\logfiles\SoftwareDistribution
Run sfc /scannow to check for file corruption.

WSUSUtil.exe

wsusutil.exe reset
- Checks that every update metadata row in the database has corresponding update files stored in the file system. If update files are missing or have been corrupted, WSUS downloads the update files again.
wsusutil.exe checkhealth
- Check Application eventlog for entries with source “Windows Server Update Services”
wsusutil.exe usecustomwebsite
- Changes the port WSUS uses. Also “resets” the port during the change.

IIS

If you go to http://WSUS_SERVER:8503 you don’t see anything. This is normal.
To test IIS go to http://WSUS_SERVER:8503/selfupdate/iuident.cab and you should be prompted to save the file.

Check for http connection errors in c:\windows\system32\logfiles\httperr\

Check IP bindings.
Check to make sure the Application Pool is running.

Check IIS logs for connection errors c:\inetpub\logs\logfiles
The default IIS installation by the WSUS installer doesn’t install logging component however.
You need to enable this under Manage Computer > Roles > Web Server > Web Server > Health & Diag > HTTP Logging

And there you have it. If following this guide didn’t fix the issue hopefully you at least came across the malfunctioning component and have a better idea where to focus your Google searches.

Enable X-Pack Security for Elasticsearch

2019-04-29T12:00:00-04:00

At some point, after probably dozens of test Elasticsearch instances, you’ll want to actually deploy a cluster into production. If you’re now responsible for a production cluster you’ll need to protect against credential harvesting and random curl DELETE queries that can cause all your indexes to disappear. Thus the motivation for purchasing X-Pack.

Throughout this post we’ll generate certificates for elasticsearch (using a root CA and certificates for each node signed with this root CA), as well as enable authentication, change the built-in account passwords, secure ES node-to-node communication (port 9300 traffic), force HTTPS queries to ES (port 9200 traffic), modify Kibana and Logstash to talk to ES, and then secure the Kibana front-end.

I’m making a few assumptions before we start. X-Pack should already be installed by default. You should have also applied your license or enabled the 30 day trial license. You might as well go ahead and enable monitoring as well. I’m also assuming you have no Machine Learning jobs running.

Also note that there are many sources you can use for Authentication. I’ll be using the “Native Realm” which just means Elasticsearch will store accounts and password inside a local index, it’s also the default.

In this scenario I have 3 cluster members. One is just a master node and two are master and data nodes.

Generating certificates

SSH into one of your Elasticsearch hosts. Create a file to be used as a template and enter the information for each Elasticsearch host in your cluster. In my elasticsearch.yml I specify my hosts by IP address. If you use a host name I believe you want the FQDN listed in the dns: section as an additional entry: - "elastic1.mydomain.com".

sudo nano ~/cert-gen.yml

instances:
  - name: "elastic1" 
    dns:
      - "elastic1"
    ip:
      - "192.168.1.11"
  - name: "elastic2"
    dns:
      - "elastic2"
    ip:
      - "192.168.1.12"
  - name: "elastic3"
    dns:
      - "elastic3"
    ip:
      - "192.168.1.13"

Now generate both the CA certificate as well as the node certificates.

/usr/share/elasticsearch/bin/elasticsearch-certutil cert --in cert-gen.yml --keep-ca-key

I opted not to use a password for the certificates, you may wish to.

If you want to use a commercial or organization-specific CA, you can use the elasticsearch-certutil csr command to generate certificate signing requests (CSR) for the nodes in your cluster. Find more info here.

You should now have a certificate-bundle.zip file.

Configuring Node-to-Node Encryption

Let’s unzip the certificate bundle.

apt install unzip
unzip certificate-bundle.zip

You should have a ca folder, as well as a folder for each host.

First thing we’ll do is go into the ca folder and convert out p12 certificate to PEM format. This will be necessary for the Kibana and Logstash servers.

cd /ca
openssl pkcs12 -in ca.p12 -clcerts -nokeys -chain -out ca.pem

Copy the relevant node certificates to each Elasticsearch node, and copy the ca.pem certificate to your Kibana and Logstash servers.
I’ll scp the files to my user’s home directory (where that user has permission to write files) and then on each host I’ll create a certs directory in /etc/elasticsearch/ and copy the cert there. For each Elasticsearch host you only need the single host p12 file, not the CA file.

scp elastic2.p12 user@elastic2:/home/user

On each host, SSH in, create /etc/elasticsearch/certs, and copy over the certificate.

Stop Logstash so no more data is being sent to Elasticsearch.

POST _flush/synced

Keep running this command until there are no more failures.

We’re now going to shutdown the entire cluster. Elasticsearch will not start re-allocating shards until after the index.unassigned.node_left.delayed_timeout value has expired which is one minute by default. Hopefully you’re able to shutdown all your hosts in under a minute. If not, look here for instructions on disabling allocation of replicas.

sudo systemctl stop elasticsearch.service

On each host edit elasticsearch.yml and add the following lines, being sure that the certificate path and name are correct.

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/certs/elastic1.p12 
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/certs/elastic1.p12

Once you’ve modified all the nodes we can start bringing them back up. I’ll start with my master-only node since it has no data. Since I have my minimum master nodes set to 2 in elasticsearch.yml it should start up and wait since the minimum threshold has not been met.

Start the first host.

sudo systemctl start elasticsearch.service

View the logs.

tail -f /var/log/elasticsearch/cluster-name.log

You should see a message saying the host is waiting on the minimum master nodes.

Bring up the second node. If you’re watching the log file on the first node you should see the second node come up and a master elected. Now bring up the third node.

On the master host the log file should show the cluster going from red to yellow, and eventually yellow to green.

Before we can login to Kibana and look around we’ll need to reset the internal account passwords.

Set Built-in Account Passwords

On one of your Elasticsearch hosts run:
/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

Your cluster needs to be running and healthy or the command will throw an error.

You’ll be prompted to change the passwords for the following users: elastic, kibana, logstash_system, beats_system, apm_system, and remote_monitoring_user. You should probably have these written down beforehand.

Once complete this information is stored in a .security index in Elasticsearch.

You cannot run the elasticsearch-setup-passwords command a second time. Instead, you can update passwords from the Management > Users UI in Kibana, use the security user API, or delete the entire .security index.

Now point your web browser to http://<ES_Host>:9200. You should be prompted to log in to Elasticsearch. To log in, you can use the built-in elastic user and the password you just specified. You should see returned a status OK JSON message.

Now go into your kibana.yml file and uncomment the following lines:

elasticsearch.user: kibana
elasticsearch.password: <your password here>

Instead of writing the user password in kibana.yml you can add it to a kibana-keystore also. See the Elastic website for details.

Now log into Kibana by visiting the website. You should be prompted to enter credentials. Use elastic for the user and supply the password.

Under Management in the left-hand menu you’ll now see a new Security section with Users and Roles.

Take a minute to create an account for yourself and assign it the superadmin role. Remember that the principle of least privilege applies here as well. While it’s important to have a superadmin account you should also create another account for yourself with fewer privileges for daily use.

I’ll also create a role for my Logstash writers named logstash_hostname by selecting “New Role”.
For cluster privileges, add manage_index_templates and monitor.
For Indices privileges, add write, delete, and create_index.
Now create a user with the same name and assign it this new role. This is what we will use in our logstash output {} to connecto to Elasticsearch.

We’ve now enabled authentication and transport layer encryption for internal node communications.

Configuring HTTP Security

We’ll continue on by enabling HTTPS as well for every client that wants to make requests to the cluster. Note that I could have enabled the following at the same time that I enabled transport layer security but doing this in 2 steps aids in troubleshooting should something go wrong and allows you to run the cluster for a few days with only transport security enabled if you wish to do so.

This procedure mimics a rolling upgrade. We’ll only be shutting down a single host at a time so the cluster stays up. Any clients (Logstash, Kibana, etc.) will break until we configure them with the new settings. You should stop their services now.

This won’t have any effect on inter-node communication since we already secured that. That means after each host goes down and comes back up we’ll wait until the cluster status is green again before proceeding to the next host.

SSH to the first host and stop the Elasticsearch service. Keep in mind that Kibana is connecting to a specific host. It’ll be beneficial to save this host for last.

Edit the config file and add the following lines.

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: /etc/elasticsearch/certs/elastic1.p12
xpack.security.http.ssl.truststore.path: /etc/elasticsearch/certs/elastic1.p12 

Start the Elasticsearch service. Run tail -f /var/log/elasticsearch/cluster-name.log to check for any issues. Login to Kibana, go to Monitoring, and check the cluster status. Wait until it’s green before proceeding to the next host.

Kibana will continue to work until you change the settings on the host it connects to. If you save this host for last you can check the cluster status at each step until the last.

Kibana

Now we’ll configure Kibana to both connect to Elasticsearch securely as well as require HTTPS for the front-end.

For the front-end certificate I needed it to be trusted by my browsers so I used my Enterprise certificate authority to generate a new certificate. This will allow all the clients in my Domain to trust it automatically. You can either do this or purchase just this single certificate from a trusted third-party. I have a few recommendations in my software list.

Hopefully earlier you copied over the ca.pem file that we generated and moved that to /etc/kibana/certs/. If not, scroll up and do that now.

We’re now going to change the ownership and permissions on the certs directory and it’s files.

cd /etc/kibana/
sudo chown -R root:kibana certs/
sudo chmod -R 750 certs/

Kibana and Logstash both require execute permissions on their certs directories.

Modify Kibana to connect to ES via HTTPS.

elasticsearch.hosts: ["https://<your_elasticsearch_host>:9200"]
elasticsearch.ssl.certificateAuthorities: /etc/kibana/certs/ca.pem

If you have issues you can disable certificate validation in the configuration, but since all the node certificates were signed by this ca.pem everything should validate cleanly.

I copied over a .key and .crt file from my Enterprise CA to use for the front-end and set the following.

server.ssl.enabled: true
server.ssl.key: /etc/kibana/certs/server.key
server.ssl.certificate: /etc/kibana/certs/server.crt

Re-start the service and connect to your Kibana front-end over HTTPS.

Logstash

This process will closely mimic the Kibana setup.

Start by changing the certs directory ownership and permissions

cd /etc/logstash
sudo chown -R root:logstash certs/
sudo chmod -R 750 certs/

Now with our certificate in place we can modify the logstash output section of our logstash configuration.

I created a new logstash_hostname user from the Kibana UI to assign to Logstash. Follow the directions here to do the same or use the logstash_writer built-in account.

output {
  elasticsearch {
    host => https://<elasticsearch_host>:9200
    user => logstash_hostname
    password => <password-here>
    ssl => true
    cacert => /etc/logstash/certs/ca.pem
  }
}

Now restart logstash sudo systemctl restart logstash

Check the logs to make sure logstash is able to connect to Elasticsearch. sudo tail -f /var/log/logstash/logstash-plain.log

And that’s it, everyone’s communicating over encrypted channels. Well, almost everyone. Monitoring traffic, if enabled, is still unencrypted. The recommendation here is to send monitoring metrics to a separate cluster, which means generating a CA cert for that cluster and configuring all of our servers to send data to a different IP. We’ll save that for another post.

Cuckoo Sandbox Installation

2019-04-11T12:00:00-04:00

Cuckoo Sandbox is an open source malware analysis system used to launch files in an isolated environment and observe their behavior. Pass it a URL, executable, office document, pdf, or any file, and it will get launched in an isolated virtual machine where cuckoo can observe it’s process execution, API calls, network access, and all filesystem activity. You’ll then get a report and a threat score based on the observed behavior. Once the analysis is complete the VM restores to a known good snapshot and waits for the next execution.

Once Cuckoo is running you can pass it samples in three ways. Drag and Drop through the web interface, through the command line with cuckoo --submit, or through the API.

I’m going to install Cuckoo 2.0.6 on Ubuntu Desktop 16.04. I need a GUI to run Virtualbox and running this on 18.04 is problematic due to a change in Openssl 1.1.0. I’m sure I could have just as easily gone with a server OS and GUI. I’m running as the user cuckoo.

The official installation instructions are here and many of the steps in this tutorial were copied from this excellent guide.

For my VM I’m using a licensed copy of Windows XP. You’ll need the XP ISO and a license key or a trial version. Cuckoo is supposed to work equally well with Windows 7 but I’ve not tested that.

Note that this is not an efficient or secure installation. I’m unsure if all the packages being installed are necessary, for instance you’ll be installing sqlite, mongodb, and postgres which is really not recommended. I’ve taken these steps from other guides and haven’t bothered to do a full clean-up. Also note ironically that if you want to enable searching in Cuckoo you need to install yet another database, Elasticsearch.

Installation

Start with some pre-requisites

sudo apt-get install git mongodb libffi-dev build-essential python-django python python-dev python-pip python-pil python-sqlalchemy python-bson python-dpkt python-jinja2 python-magic python-pymongo python-gridfs python-libvirt python-bottle python-pefile python-chardet tcpdump apparmor-utils libjpeg-dev python-virtualenv python3-virtualenv virtualenv swig postgresql libpq-dev libguac-client-rdp0 libguac-client-vnc0 libguac-client-ssh0 guacd autoconf libtool libjansson-dev libmagic-dev libssl-dev -y

Set TCPDump for non-root user
sudo aa-disable /usr/sbin/tcpdump
sudo setcap cap_net_raw,cap_net_admin=eip /usr/sbin/tcpdump

Test by running getcap /usr/sbin/tcpdump and expect to get back /usr/sbin/tcpdump = cap_net_admin,cap_net_raw+eip

Install Yara

wget https://github.com/plusvic/yara/archive/v3.4.0.tar.gz -O yara-3.4.0.tar.gz
tar -zxf yara-3.4.0.tar.gz
cd yara-3.4.0
./bootstrap.sh
./configure -with-crypto -enable-cuckoo -enable-magic
make
sudo make install

Test and make sure you get back a version number
yara -v

And install yara-python

cd yara-python
python setup.py build
sudo python setup.py install

Install ssdeep

cd ~/
wget http://sourceforge.net/projects/ssdeep/files/ssdeep-2.13/ssdeep-2.13.tar.gz/download -O ssdeep-2.13.tar.gz
tar -zxf ssdeep-2.13.tar.gz
cd ssdeep-2.13
./configure
make
sudo make install

Test and make sure you get back a version number
ssdeep -V

Install some python dependencies

pip install pydeep
pip install openpyxl
pip install ujson
pip install pycrypto
pip install distorm3
pip install pytz
pip install jsonschema

Install Volatility

git clone https://github.com/volatilityfoundation/volatility.git
cd volatility
python setup.py build
python setup.py install

Make sure you’re installing VirtualBox 5.1 or earlier, not 5.2 or later.
apt-cache policy virtualbox

Install Virtualbox
sudo apt-get install virtualbox

Add my cuckoo user to the vboxusers group
sudo usermod -a -G vboxusers cuckoo

Copy your Windows XP ISO (or Win7) to /home/cuckoo/

As the cuckoo user create the VM.

vboxmanage createvm --name windowsxp --ostype WindowsXP --register
vboxmanage modifyvm windowsxp --memory 1000 --acpi on --boot1 dvd --nic1 nat
cd ~/VirtualBox/windowsxp
vboxmanage createhd --filename windowsxp.vdi --size 12000
vboxmanage storagectl windowsxp --name 'IDE Controller' --add ide --controller PIIX4
vboxmanage storageattach windowsxp --storagectl 'IDE Controller' --port 0 --device 0 --type hdd --medium windowsxp.vdi
vboxmanage storageattach windowsxp --storagectl 'IDE Controller' --port 0 --device 1 --type dvddrive --medium /home/cuckoo/WXPVOL_EN.iso
vboxmanage hostonlyif create
vboxmanage modifyvm windowsxp --nic1 hostonly
vboxmanage modifyvm windowsxp --hostonlyadapter1 vboxnet0

Locally, from the GUI console of the OS (not an ssh session), open the Terminal application and run vboxmanage startvm windowsxp.

Your VM should start and boot from the ISO allowing you to install the Operating System.

While the OS is installing we’ll switch gears for a moment and setup the host machine to talk to the guest and forward traffic.

You’ll want to change enp0s25 in the first rule to match the public interface name of the network card in the host with Internet access. Get this by running ip addr.
Run the following commands on the Linux host machine.

sudo iptables -t nat -A POSTROUTING -o enp0s25 -s 192.168.56.0/24 -j MASQUERADE
sudo iptables -P FORWARD DROP
sudo iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT
sudo iptables -A FORWARD -s 192.168.56.0/24 -j ACCEPT
sudo iptables -A FORWARD -s 192.168.56.0/24 -d 192.168.56.0/24 -j ACCEPT
sudo iptables -A FORWARD -j LOG

Note that these rules aren’t currently persistent, meaning they’ll be erased on reboot. To make them permanent install the iptables-persistent package. When you modify the rules in the future use sudo netfilter-persistent save to make the change permanent.

Now we’ll enable forwarding in the kernel.

echo 1 | sudo tee -a /proc/sys/net/ipv4/ip_forward
sudo sysctl -w net.ipv4.ip_forward=1

Complete the guest OS install and continue with the steps below.

Once completed, don’t install any additional system updates and configure the VM with a static IP.

IP: 192.168.56.10
Subnet: 255.255.255.0
Gateway: 192.168.56.1
DNS: 8.8.8.8 

Try and ping the host PC ping 192.168.56.1. You should get a response.

Install the Virtualbox guest additions in the guest OS and enable host to guest drag and drop from the VM settings.
Settings > General > Advanced > Drag'n'Drop: Host to Guest

Download the following packages and drag them into the guest OS to copy them to the Desktop. Now install them.

The Python 2.7 installer for your guest OS.
The Python Imaging Library to take snapshots of the files executing.
Cuckoo Agent and place the file in the startup folder of the guest VM so it executes on startup. A command window should open showing the agent is listening.

This would also be a good time to install Adobe Acrobat Reader and perhaps a trial version of Microsoft Office.

The way in which the VM is snapshotted and the state it’s in is very important for cuckoo.
Do the following exactly as I describe. Visit the Cuckoo troubleshooting doc for more information.

While the VM is running

vboxmanage snapshot windowsxp take snapshot1 --pause
vboxmanage controlvm windowsxp poweroff
vboxmanage snapshot windowsxp restorecurrent

And now, as the cuckoo user, install Cuckoo.

cd ~\
git clone https://github.com/cuckoosandbox/cuckoo
cd cuckoo
python stuff/monitor.py
python setup.py build
sudo python setup.py sdist
sudo python setup.py build install

Now run Cuckoo
cuckoo -d

  eeee e   e eeee e   e  eeeee eeeee
  8  8 8   8 8  8 8   8  8  88 8  88
  8e   8e  8 8e   8eee8e 8   8 8   8
  88   88  8 88   88   8 8   8 8   8
  88e8 88ee8 88e8 88   8 8eee8 8eee8

 Cuckoo Sandbox 2.0.6
 www.cuckoosandbox.org
 Copyright (c) 2010-2018

=======================================================================
    Welcome to Cuckoo Sandbox, this appears to be your first run!
    We will now set you up with our default configuration.
    You will be able to see and modify the Cuckoo configuration,
    Yara rules, Cuckoo Signatures, and much more to your likings
    by exploring the /home/cuckoo/.cuckoo directory.

    Among other configurable items of most interest is the
    new location for your Cuckoo configuration:
              /home/cuckoo/.cuckoo/conf
=======================================================================

Cuckoo has finished setting up the default configuration.
Please modify the default settings where required and
start Cuckoo again (by running `cuckoo` or `cuckoo -d`).

Modify the Cuckoo configuration

nano /home/cuckoo/.cuckoo/conf/cuckoo.conf
under [resultserver] verify that "ip=192.168.56.1" is set.

nano /home/cuckoo/.cuckoo/conf/virtualbox.conf
Since we’re using Virtualbox as our provider we’ll also modify some VM settings in this file.

[cuckoo1]
label = windowsxp
ip = 192.168.56.10
snapshot = snapshot1

This part is optional: sudo nano reporting.conf

    [mongodb]
    enabled = yes

Now start it all up again
cuckoo -d

2019-04-04 12:32:53,616 [cuckoo] WARNING: It appears that you haven't loaded any Cuckoo Signatures. Signatures are highly recommended and improve & enrich the information extracted during an analysis. They also make up for the analysis score that you see in the Web Interface - so, pretty important!
2019-04-04 12:32:53,616 [cuckoo] WARNING: You'll be able to fetch all the latest Cuckoo Signatures, Yara rules, and more goodies by running the following command:
2019-04-04 12:32:53,616 [cuckoo] INFO: $ cuckoo community

Run cuckoo community. I know I could have just told you to do that before but I wanted you to be aware that cuckoo has the ability to download and refresh signatures and rules.

Verify that the cuckoo directory is owned by the cuckoo user.
If not run sudo chown -R cuckoo:cuckoo ~/.cuckoo

Remember that your VM was created by the cuckoo user so don’t expect to find it if you run sudo cuckoo -d

Now run cuckoo -d again.

2019-04-04 17:05:57,609 [cuckoo.core.resultserver] DEBUG: ResultServer running on 192.168.56.1:2042.
2019-04-04 17:05:57,611 [cuckoo.core.scheduler] INFO: Using "virtualbox" as machine manager
2019-04-04 17:05:58,288 [cuckoo.machinery.virtualbox] DEBUG: Restoring virtual machine windowsxp to snapshot1
2019-04-04 17:05:58,727 [cuckoo.core.scheduler] INFO: Loaded 1 machine/s
2019-04-04 17:05:58,767 [cuckoo.core.scheduler] INFO: Waiting for analysis tasks.

Cuckoo is online and awaiting a file submission. Let’s submit a sample from the command line, you can submit any file on the system.

cuckoo submit evilfile.exe

2019-04-04 17:09:25,364 [cuckoo.core.scheduler] DEBUG: Processing task #6
2019-04-04 17:09:25,385 [cuckoo.core.scheduler] INFO: Starting analysis of FILE "evilfile.exe" (task #6, options "")
2019-04-04 17:09:25,509 [cuckoo.core.scheduler] INFO: Task #6: acquired machine cuckoo1 (label=windowsxp)
2019-04-04 17:09:25,510 [cuckoo.core.plugins] DEBUG: Started auxiliary module: Replay
2019-04-04 17:09:25,544 [cuckoo.auxiliary.sniffer] INFO: Started sniffer with PID 3537 (interface=vboxnet0, host=192.168.56.10)
2019-04-04 17:09:25,545 [cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer
...
2019-04-04 17:09:25,734 [cuckoo.machinery.virtualbox] DEBUG: Starting vm windowsxp
2019-04-04 17:09:26,048 [cuckoo.machinery.virtualbox] DEBUG: Restoring virtual machine windowsxp to snapshot1
2019-04-04 17:09:26,551 [cuckoo.common.abstracts] DEBUG: Waiting 0 cuckooseconds for machine windowsxp to switch to status ('saved',)
2019-04-04 17:09:27,793 [cuckoo.common.abstracts] DEBUG: Waiting 1 cuckooseconds for machine windowsxp to switch to status ('saved',)
...
2019-04-04 17:30:08,011 [cuckoo.core.scheduler] INFO: Task #6: acquired machine cuckoo1 (label=windowsxp)
2019-04-04 17:30:08,012 [cuckoo.core.plugins] DEBUG: Started auxiliary module: Replay
2019-04-04 17:30:08,025 [cuckoo.auxiliary.sniffer] INFO: Started sniffer with PID 7198 (interface=vboxnet0, host=192.168.56.10)
2019-04-04 17:30:08,027 [cuckoo.core.plugins] DEBUG: Started auxiliary module: Sniffer
2019-04-04 17:30:08,211 [cuckoo.machinery.virtualbox] DEBUG: Starting vm windowsxp
2019-04-04 17:30:08,508 [cuckoo.machinery.virtualbox] DEBUG: Restoring virtual machine windowsxp to snapshot1
2019-04-04 17:30:11,536 [cuckoo.core.guest] INFO: Starting analysis on guest (id=cuckoo1, ip=192.168.56.10)
2019-04-04 17:30:12,548 [cuckoo.core.guest] DEBUG: cuckoo1: not ready yet
2019-04-04 17:30:13,553 [cuckoo.core.guest] DEBUG: cuckoo1: not ready yet
2019-04-04 17:30:16,586 [cuckoo.core.guest] DEBUG: cuckoo1: waiting for status 0x0001
2019-04-04 17:30:16,595 [cuckoo.core.guest] DEBUG: cuckoo1: status ready
2019-04-04 17:30:16,699 [cuckoo.core.guest] DEBUG: Uploading analyzer to guest (id=cuckoo1, ip=192.168.56.10, monitor=latest, size=3884139)
2019-04-04 17:30:18,723 [cuckoo.core.guest] DEBUG: cuckoo1: analyzer started with PID 1616
2019-04-04 17:30:18,811 [cuckoo.core.guest] DEBUG: cuckoo1: waiting for completion
2019-04-04 17:30:19,280 [cuckoo.core.resultserver] DEBUG: LogHandler for live analysis.log initialized.
2019-04-04 17:30:19,823 [cuckoo.core.guest] DEBUG: cuckoo1: analysis not completed yet (status=2)
2019-04-04 17:30:20,834 [cuckoo.core.guest] DEBUG: cuckoo1: analysis not completed yet (status=2)
2019-04-04 17:30:23,484 [cuckoo.core.resultserver] DEBUG: New process (pid=1880, ppid=1960, name=PIL-1.1.7.win32-py2.7.exe)
2019-04-04 17:30:23,884 [cuckoo.core.guest] DEBUG: cuckoo1: analysis not completed yet (status=2)
2019-04-04 17:30:24,481 [cuckoo.core.resultserver] DEBUG: File upload request for shots/0001.jpg
2019-04-04 17:30:24,493 [cuckoo.core.resultserver] DEBUG: Uploaded file length: 45281
2019-04-04 17:30:24,894 [cuckoo.core.guest] DEBUG: cuckoo1: analysis not completed yet (status=2)
2019-04-04 17:30:25,599 [cuckoo.core.resultserver] DEBUG: File upload request for shots/0002.jpg
2019-04-04 17:30:25,619 [cuckoo.core.resultserver] DEBUG: Uploaded file length: 62214
2019-04-04 17:30:25,903 [cuckoo.core.guest] DEBUG: cuckoo1: analysis not completed yet (status=2)
2019-04-04 17:30:26,710 [cuckoo.core.resultserver] DEBUG: File upload request for shots/0003.jpg
2019-04-04 17:30:26,727 [cuckoo.core.resultserver] DEBUG: Uploaded file length: 62648
2019-04-04 17:30:26,912 [cuckoo.core.guest] DEBUG: cuckoo1: analysis not completed yet (status=2)
2019-04-04 17:33:19,549 [cuckoo.core.guest] INFO: cuckoo1: end of analysis reached!
2019-04-04 17:33:19,642 [cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Replay
2019-04-04 17:33:19,683 [cuckoo.core.plugins] DEBUG: Stopped auxiliary module: Sniffer
2019-04-04 17:33:19,684 [cuckoo.machinery.virtualbox] DEBUG: Stopping vm windowsxp
2019-04-04 17:33:21,358 [cuckoo.core.scheduler] DEBUG: Released database task #6
2019-04-04 17:33:21,502 [cuckoo.core.plugins] DEBUG: Executed processing module "AnalysisInfo" for task #6
2019-04-04 17:33:21,630 [cuckoo.core.plugins] DEBUG: Executed processing module "BehaviorAnalysis" for task #6
2019-04-04 17:33:21,643 [cuckoo.core.plugins] DEBUG: Executed processing module "Dropped" for task #6
2019-04-04 17:33:21,644 [cuckoo.core.plugins] DEBUG: Executed processing module "DroppedBuffer" for task #6
2019-04-04 17:33:21,645 [cuckoo.core.plugins] DEBUG: Executed processing module "MetaInfo" for task #6
2019-04-04 17:33:21,646 [cuckoo.core.plugins] DEBUG: Executed processing module "ProcessMemory" for task #6
2019-04-04 17:33:21,647 [cuckoo.core.plugins] DEBUG: Executed processing module "Procmon" for task #6
2019-04-04 17:33:21,823 [cuckoo.core.plugins] DEBUG: Executed processing module "Screenshots" for task #6
2019-04-04 17:33:22,714 [cuckoo.core.plugins] DEBUG: Executed processing module "Static" for task #6
2019-04-04 17:33:22,825 [cuckoo.core.plugins] DEBUG: Executed processing module "Strings" for task #6
2019-04-04 17:33:22,899 [cuckoo.core.plugins] DEBUG: Executed processing module "TargetInfo" for task #6
2019-04-04 17:33:22,911 [cuckoo.core.plugins] DEBUG: Executed processing module "NetworkAnalysis" for task #6
2019-04-04 17:33:22,911 [cuckoo.core.plugins] DEBUG: Executed processing module "Extracted" for task #6
2019-04-04 17:33:22,912 [cuckoo.core.plugins] DEBUG: Executed processing module "TLSMasterSecrets" for task #6
2019-04-04 17:33:22,917 [cuckoo.core.plugins] DEBUG: Executed processing module "Debug" for task #6
2019-04-04 17:33:22,921 [cuckoo.core.plugins] DEBUG: Running 540 signatures
2019-04-04 17:33:23,852 [cuckoo.core.plugins] DEBUG: Analysis matched signature: has_pdb
2019-04-04 17:33:23,853 [cuckoo.core.plugins] DEBUG: Analysis matched signature: packer_entropy
2019-04-04 17:33:23,985 [cuckoo.core.plugins] DEBUG: Executed reporting module "JsonDump"
2019-04-04 17:33:24,299 [cuckoo.core.plugins] DEBUG: Executed reporting module "MongoDB"
2019-04-04 17:33:24,299 [cuckoo.core.scheduler] INFO: Task #6: reports generation completed
2019-04-04 17:33:24,319 [cuckoo.core.scheduler] INFO: Task #6: analysis procedure completed

Launching Cuckoo the next time

[cuckoo] CRITICAL: CuckooCriticalError: Unable to bind ResultServer on 192.168.56.1:2042 [Errno 99] Cannot assign re
quested address. This usually happens when you start Cuckoo without bringing up the virtual interface associated with the ResultServer IP ad
dress. Please refer to https://cuckoo.sh/docs/faq/#troubles-problem for more information.                  

Run:
VBoxManage hostonlyif create
VBoxManage hostonlyif ipconfig vboxnet0 --ip 192.168.56.1 --netmask 255.255.255.0
before running cuckoo -d

Cuckoo API

Start the API listener
cuckoo api --host 0.0.0.0

Runs on port 8090 by default.

I’m calling Cuckoo Sandbox from Cortex and I’m currently unsure how to pass an API token from there so for now I’m just disabling authentication.

nano /home/cuckoo/.cuckoo/conf/cuckoo.conf
Comment out the api_token

Launch cuckoo web server

Submit files and view results on the cuckoo webpage.

cuckoo web -H 0.0.0.0

Navigate to http://<IP of server>:8000 to view the Dashboard.

View the results of recently run scans and open the summary page.

View an analysis summary.

View in-depth system activity.

View and capture network activity.

See additional files that are downloaded

You now have your very own sandbox in which to detonate any suspicious file you come across. Consider enabling the Cuckoo Analyzer in Cortex and giving it the IP of your new sandbox server.