What is the CUR and why should I have it?
In my opinion the Amazon Cost and Usage Report (CUR) is a vital tool on the road to FinOps.‘The AWS Cost and Usage report provides information about your use of AWS resources and estimated costs for that usage.’ AWS. This report allows for an in-depth view of how you are utilising Amazon's services. But first, a bit of background on the CUR.
Previously users were able to read the ‘Detailed Billing Report’ which had a lot of similarities to the CUR but had a different way of calculating line items. This has now been totally removed and AWS suggest users use the CUR as their main report.
If you have a Organisation set up with your AWS Accounts you can get access to the CUR from your master billing account which holds the purse strings. It can be created in the 'Billing and Cost Management Dashboard', but I will talk a bit more about this later.
It holds a lot of information, and when I say a lot, I mean a lot. The reports consists of different columns each providing extensive detail depending on the line item. To get AWS’s full description of each column see their documentation. All of these columns are important but to just get you started with your CUR I have the 5 W’s:
What the costs are = line_item_unblended_cost
Who Used it = line_item_usage_account_id
When the did you use it = line_item_usage_start_date
Where in the AWS globe you are using this = product_region
Why you spent the money = line_item_resource_id
Using these 5 W's can get you started with understanding and using your Cost and Usage Report.
Something else you can do to give more detail to your CUR is to ensure you have cost allocation tags defined. These are tags that you define on your billing console which will be brought into your CUR file. Having these active through your accounts will enable you to analyse costs on the levels you decide. For example, you could look at costs per application, per environment or per team, all utilising tags. In this way you are able to personalise your report to get the most out of it. Having the tags set up before you create the CUR will allow this data to be picked up by the report and add an extra level of detail to your data.
How do you create the CUR?
So how do you create your Cost and Usage Report? Well you have a few options at this very moment in time.
(Hint hint its option 3)
From delving in to this file I have learned that some simple actions can really help users get the full benefit from the report. I would like to share some simple tips that I wish someone had shared with me.
One of the biggest downsides to the report is that it will only start logging from when you create the report so you cannot see historical data. That brings me to my first point, get your CUR setup in your billing account as soon as possible. That way you can start tracking costs and use them as soon as possible to understand your accounts.
When creating your report there are a few different options you can choose and these options can affect how you use your CUR in the future and as I said, its not possible to go back and get historical data so these initial choices are really important. Starting with a simple option, you can select the box which records resource ID’s in the CUR. When I first created a CUR I missed this option; not immediately seeing it’s relevance. However, later down the line I regretted it as, across a very large estate, it becomes a lot easier to track down anomalies in spending if you have the resources ID. So another tip is op-in to the resource ID option.
Another simple option is choosing the overwrite report option. This will ensure you have the most up to date data in your bucket and make using a tool to view and analyse that data a lot simpler.
The next configuration can depend on how much detail you need in your report. It is possible to choose the granularity of your data by selecting daily or hourly. Choosing hourly will simply times your data collection by 24. Having this data will give a more in-depth look at usage for items like EC2, enabling you to view spikes and trends in your data. I would, therefore, recommend selecting the hourly granularity. However, this will increase the data in your CUR, and therefore the storage cost, and depending on how you manage this data this can be either a burden or a blessing. For example if you have a small estate then this won't affect you too much as the daily data size will still be quite small, but a huge estate - well, you will have access to a huge amount of valuable data but its still a lot.
Finally you need to choose what AWS tool you wish to integrate your data with. The options are:
Athena
Redshift
QuickSight
Once you have selected your options you can create the report. It will take approximately 24 hours to generate. So sit down, relax, maybe watch an entire series of 24, while you wait for your report to be created!
How you can use you CUR
This data is stored in a CSV in the selected S3 bucket. I would suggest turning on versioning for the S3 bucket to avoid losing any data. Now, I bet you're thinking, that’s cool, I can have a scan through that CSV and see what’s going on. Well, if you only have one or two resources in your account, sure, go for it. However, if you have a production level deployment with multiple accounts then be prepared for the scale of your CUR file. For my estate, a single days CSV is a couple of million lines long, this is to big for my laptop to handle let alone me reading through it. So how do you deal with it? Well this where your options of the integrated tool come in. Selecting options such as Redshift, Athena or QuickSight will allow you to manage and query this data to find the results you need. I will be doing posts on those aspects in more detail. But as a personal recommendation I would recommend Athena. However if you're not sure which road you want to go down you can always edit your report.
So, there are my Cost and Usage Report tips:
Include Resource IDs
If you can, go hourly
Start as soon as you can
Version your bucket
Overwrite report
Setup Cost allocation tags
If you follow the link below you can set the report up manually 😴 or you can use a python script I made.
This script fits inside a larger tool I am developing so look out for her in the future.
Sources
Comentários