Malware threat indicators in AWS using MISP

Malware threat indicators in AWS using MISP

Every zero-day vulnerability is an attack vector that has existed before the day it was announced. When this happens we must vigilantly patch all of our vulnerable services while also ensuring that nothing has been compromised. We share threat indicators to limit the potential impact of attackers; however, when a new malware indicator has been identified in the wild, updating your firewall isn’t always enough.

AWS GuardDuty is a great solution for parsing VPC flow logs and Route53 query logs with public threat feeds. Attacks targeted against specific industries are often underrepresented in public feeds. There are also delays from when the attack is first seen until when the data is pulled into a threat feed.

Amazon Athena is a valuable tool we can use when it comes to searching for threat data in AWS accounts. Athena allows you to query large amounts of data from S3 using a SQL syntax. AWS has helpful guides for how to set up VPC flow logs to be queryable from Athena here.

Searching over large amounts of flow log data quickly is very useful; however, we will want automatic integration with MISP to identify malicious traffic. We can pull out malicious IP addresses from the MISP API. Below is a screenshot of the MISP query builder.

This example shows a search for all of the malicious IP addresses (ip-dst) over the last seven days with the intrusion detection system (IDS) flag set. The IDS flag lets security analysts highlight which attributes of an event are strong indicators of compromise. For example, if a malware package sends a DNS request to the google nameserver 8.8.8.8 it may help identify the malware family, though this by itself does not represent a host is compromised.

Pulling the list of malicious IP addresses can be performed in a scheduled Lambda task running the MISP python API. This example shows how attributes can be pulled and dumped out as a CSV file.

#!/usr/bin/env python

from pymisp import PyMISP
import json

misp = PyMISP('https://misp.localhost/', '<api-key>', True, 'json')

ret = ""
result = misp.search('attributes', type_attribute = 'ip-dst', to_ids = True)
for attribute in result['response']['Attribute']:
 ret += attribute['id'] + ","
 ret += attribute['event_id'] + ","
 ret += attribute['value'] + "n"

print (ret)

This file can then be used to set up a new Athena database table. The example here shows the syntax to create a basic table for malicious IP addresses while retaining the MISP event ID.

CREATE EXTERNAL TABLE IF NOT EXISTS misp_dest_indicators (
 attributeid int,
 eventid int,
 destinationaddress string
)
PARTITIONED BY (dt string)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ' '
LOCATION 's3://your_log_bucket/vpcflowlogs/';

 

Now we have all of the data to parse over our VPC flow logs with our MISP threat indicators. Joining these Athena tables, we can see if any of our MISP indicators show up in our VPC flow logs.

SELECT v.account,
  v.interfaceid,
  v.sourceaddress,
  v.destinationaddress,
  v.action,
  m.attributeid,
  m.eventid
FROM vpc_flow_logs v,
  Misp_dest_indicators m
WHERE v.destinationaddress = m.destinationaddress;

If we want this in a more automated process we can execute this Athena query directly from Lambda. We could then trigger an alert with SNS if we find any matches on our hosts. For example:

import boto3 
session = boto3.Session()
client = session.client('athena', region_name='ap-southeast-2') 

response = client.start_query_execution(
    QueryString='select * from vpc_flow_logs limit 100;',
    QueryExecutionContext={
        'Database': 'vpc_logs'
    },
    ResultConfiguration={
        'OutputLocation': 's3://<bucket>'
    }
)

This solution allows us to search over large amounts of data when a new threat emerges. We also want to make sure these security events don’t happen in the future. AWS has a threat detection service called GuardDuty which will passively search for threats in VPC flow logs and Route 53 query logs. GuardDuty can use custom threat lists from S3 which allows us to provide another dump of MISP threat indicators in a text file. This will then alert any future events where hosts will try to route to any of these hosts to your security team. This will then alert your security team to any future events where hosts try to route to any malicious addresses.