Receiving daily AWS costs in Slack

Receive daily unblended cost estimates of AWS accounts from an AWS organization, to slack using Cost Explorer API & a lambda function.

Raywon Kari - Published on July 23, 2020 - 6 min read

Now a days, almost every company is embracing Multi Account strategies, to properly organise their workloads into logical groupings. AWS makes this easier with one of their service called AWS Organizations. Doing so, it has its own benefits such as security, access control, insights etc, and one of the most important is the ability to see the cost usage.

For example, we could create one acccount for each environment, such as dev, stage, qa, prod etc, or we could create one account for each team, or a mix of both. Doing so, we can easily find out how much cost we are spending on different AWS accounts, which would help us in optimizing the cost depending on the type of the account.

In this blog post, we will see how we can get to know daily cost estimates of such multi accounts, directly to a slack channel.

Here after, will go through the following aspects:

• Architecture of this project
• Prerequisites
• Necessary AWS components needed
• Deploying the project

To receive daily cost estimates for a set of AWS accounts, we will write a lambda function, which would query the Cost Explorer's API to get the cost details, and thereafter, send the same info to a slack channel. Following is the workflow we will setup:

1. Create a lambda function, with a cron setting, so it runs once everyday.
2. Lambda function will query Cost Explorer's API using getCostAndUsage method.
3. The received cost info, is then processed, and sent as a slack message, using Slack's incoming webhooks.
4. That's it. 🎉

Here is how the architecture looks like:

In this section, Let's quickly take a look at the prerequisites needed.

These are the things we will need:

1. Permissions to deploy and configure lambda functions in the AWS Master Account, such as IAM, Cron, CloudWatch etc.
2. Access to slack's configuration, such as adding an incoming webhook to a channel.
3. Access to AWS Organizations, to fetch necessary AWS account IDs.

Now that, we have seen the prerequisites, let's start building our lambda function.

Firstly, let's take a look at the getCostAndUsage method and see what it needs. Full doc on this method can be found here. This API method, requires a config, which consists of the time period we want to query, dimensions, granularity etc, and optionally a filter. We will use the following config:

config = {
    TimePeriod: {
        End: EndDate, // this is a variable
        Start: StartDate // this is a variable
    },
    Granularity: 'DAILY',
    Metrics: ['UNBLENDED_COST'],
    Filter: {
        "Dimensions": {
            "Key": "LINKED_ACCOUNT",
            "Values": accountList, // this is a variable
        }
    },
    GroupBy: [
        { 
            "Key": "LINKED_ACCOUNT",
            "Type": "DIMENSION"
        }
    ]
}

Let's construct the actual API call. Following is an example usage:

// Init cost explorer
var costexplorer = new AWS.CostExplorer();

costexplorer.getCostAndUsage(config, function(err, data) {
    if (err) {
        console.log(err);
    } else {
        processDataAndSendToSlack(data)
    }
});

These two are the most important things for this project i.e., the config we need, such as timeperiod, metrics and so on, and then retreiving the actual data using the config. The response will be in JSON. As you can see, the response will be available for access in the data object.

Once the data is retrieved, we can easily parse it and use it however we want down the line. In this case, we are sending that info as a slack message. You can modify the config according to your needs, and similarly process the response and construct subsequent steps according to your needs.

In the project, I have separated out the appconfig, into a JSON file of its own. Here is an example of how the config file looks like:

{
    "aws": {
        "apiVersion": "2017-10-25",
        "region": "us-east-1"
    },
    "accounts": {
        "account-name-one": "000000000000",
        "account-name-two": "111111111111",
        ...and so on
    },
    "slack": {
        "webhook": "https://hooks.slack.com/services/XXXXXXXXX/YYYYYYYYYYYYYYYYYYYYYYYYYYY"        
    }
}

If you are trying out this project, make sure to update the values with your account IDs, and the right slack webhook URL. Region is set to us-east-1, that is because, cost explorer provides only one endpoint i.e., https://ce.us-east-1.amazonaws.com .

In order to get a webhook URL, we need to create an incoming webhook App in slack & add and connect that to a channel of our choice. Thereafter, I have uploaded a custom icon as the App's icon, if you do not specify any icon, the default icon will be used. For more info on slack's incoming webhooks, refer the docs here.

For deploying, I am using AWS CDK. Following are the packages we need along with CDK:

• Lambda
• IAM
• Events
• Targets

We can initialise a cdk project and install these packages using:

# Creates an empty project
cdk init --language=typescript

# Install packages
npm install --save @aws-cdk/aws-events @aws-cdk/aws-events-targets @aws-cdk/aws-iam @aws-cdk/aws-lambda          

# after the installation is done, make sure all packages are having the same version.

Now that, we have our base line project ready, let's add the lambda function, and other configuration, as shown below in lib/xxxxx.ts file:

const lambdafunc = new lambda.Function(this, 'DailyCosts', {
    runtime: lambda.Runtime.NODEJS_12_X,
    code: lambda.Code.asset("../src/"),
    handler: "lambda.main",
    timeout: cdk.Duration.seconds(10),
    logRetention: 7,
    initialPolicy: [new iam.PolicyStatement({
    actions: [
        'ce:Get*',
        'ce:Describe*',
    ],
    resources: ['*'],
    })]
});

const rule = new events.Rule(this, 'Rule', {
    schedule: events.Schedule.expression('cron(0 7 * * ? *)')
})

rule.addTarget(new targets.LambdaFunction(lambdafunc));

This kind of feels a lot declarative isn't it, i.e., we can understand what this is doing simply by looking at it.

We are creating a lambda function by specifying a runtime, handler settings, timeout, IAM policy etc, and thereafter adding a rule which is configuring the cron job.

Once we are done, we simply deploy the project using cdk deploy command.

Here are a few examples of the slack messages we might get:

I do not have many accounts, and I don't have much activity, so the daily cost estimate is empty. If we try with regular running accounts, we will see different slack fields in the same message, with the cost info similar to the one from the examples.

Full source code, and other instructions are available in the github repo here.

Feel free to try out the project, and let me know If you have any questions, thoughts or feedback. Also feel free to contact me on any platform you prefer.