Enrichments

Enrichment Overview

ACE uses the RabbitMQ messaging system to provide enrichment. For more information about RabbitMQ and a better understanding, they offer a great tutorial in numerous different languages (RabbitMQ Tutorials). ACE currently uses C# for RabbitMQ, so all code samples provided will be in C#.

[embed image] This is an overview diagram of how data flows through RabbitMQ. The ACE Web Server forwards the events to the ACE Exchange. Based on the specified Routing Key, the ACE Exchange routes the event to the proper queue, finally being picked up by either the FileWriter or SIEM consumer for analysis.

Adding Enrichments

In order to extend the current enrichment capability of ACE, there are several necessary steps:

1. Creating a Queue
2. Creating a Binding
3. Creating an Enrichment Consumer

The following sections will cover each step in detail and provide a C# code sample

1. Creating a Queue

Example Code

//Creating new queue
channel.QueueDeclare(queue: "queue_name",
                     durable: false,
                     exclusive: false,
                     autoDelete: false,
                     arguments: null);

2. Creating a Binding

Example Code

//Creating new exchange binding for new enrichment
channel.QueueBind(queue: "queue_name",
                          exchange: "ace_exchange",
                          routingKey: "newEnrichment.#");

3. Creating an Enrichment Consumer

Example Code

EventingBasicConsumer newEnrichmentConsumer = new EventingBasicConsumer(channel);
newEnrichmentConsumer.Received += (model, ea) =>
{
    var body = ea.Body;
    var message = Encoding.UTF8.GetString(body);
    var routingKey = ea.RoutingKey;

    // New Enrichment Action
    try
    {
        //Parse JSON
        JObject originalMessage = JObject.Parse(message);
        

        // Perform desired enrichment action


        // Add new data to event 
        originalMessage.Add("Name1", "Value");
        originalMessage.Add("Name2", "Value2");
        
        // Recreate JSON for export
        string enrichedMessage = originalMessage.ToString(Newtonsoft.Json.Formatting.None);

        body = Encoding.UTF8.GetBytes(enrichedMessage);
    }
    catch (Exception e)
    {
        Console.WriteLine("Exception" + e);
    }

    // parse enrichment off front of routing key
    string[] words = routingKey.Split('.');
    words = words.Skip(1).ToArray();
    routingKey = string.Join(".", words);

    // Ack recieving the message from the queue
    channel.BasicAck(deliveryTag: ea.DeliveryTag, multiple: false);
    // Publish new enriched message back to ACE exchange for routing
    channel.BasicPublish(exchange: "ace_exchange",
                         routingKey: routingKey,
                         basicProperties: null,
                         body: body);
};