AWS is the leading cloud vendor today, with one out of three companies that operate in the cloud using its services. It’s a safe bet that these companies want to know how to keep their applications secure on AWS Security is top of mind, just as it would on any cloud. While there are many considerations that factor into a holistic security strategy, one important one is log management. In this post, I explain how to leverage log management to optimize security for deployments on AWS.
AWS is vocal about its shared responsibility model where it manages security ‘of’ the cloud, but as a customer, you manage security ‘in’ the cloud. This means that AWS takes efforts to secure the hardware layer, and build security into the products it delivers. However, you are responsible for securing the data you manage within these AWS services. A key ingredient to enforce robust security in the cloud is log data. Logs are essential, as they are the closest you can get to the source of truth when an incident occurs—because they provide deep visibility into every part of the system end-to-end. Fortunately, AWS provides a range of security products to help you with the complex task of securing your AWS data. Let’s look at some of the ways you can leverage log analysis to be secure in the AWS cloud.
Log Management for AWS Security
There are two types of logs you can track in AWS—application event logs, and API call logs. CloudWatch Logs is the default logging service for all your AWS resources, like EC2, DynamoDB, or RDS. It captures application event and error logs, and lets you monitor and troubleshoot application performance. For example, if any application performance metric crosses a threshold, you can dig deeper in the logs to investigate what is causing the anomaly. During an incident, you can view your CloudWatch Logs to estimate the extent of the damage to your system, and get a good idea of how user experience is being affected.
CloudTrail, on the other hand, works at a lower level, monitoring APIs for various AWS services. It tracks and logs API calls to any AWS API, complete with details like IP address, and the account from which the call originated. CloudTrail is essential when investigating an attack.
Together, CloudWatch Logs and CloudTrail give you deep and wide visibility across your AWS services. However, to get the most out of the log data collected by these two tools, you need to integrate them with other AWS services, and even external log analysis platforms.
Log IAM and KMS Activity in CloudTrail
Identity and Access Management (IAM) is the most important AWS service related to security. It controls the permissions for users and accounts, to be able to access the various services and data you have across AWS. The IAM console gives you an overview of your security status. However, for more in-depth analysis, you should log all your IAM API calls in CloudTrail. This would include events like sign-ins, sessions, role changes, and permission changes. With the recent launch of AWS Organizations, you can now set up unified logging for all your AWS accounts if you have multiple accounts.
While IAM controls access to AWS services, AWS Key Management Service (KMS) controls access to all data stored in AWS storage solutions like S3. KMS encrypts data with a key, and only authorizes data access to the services and users who have the appropriate encryption key. Using CloudTrail, you can log all API calls to KMS, and in case of a breach, you can analyze the origin of each call.
When configuring security policies in AWS, IAM and KMS are necessary, but to monitor how these policies are being followed or violated after initial setup, you need to have all activity logged in CloudTrail.
Lambda and Logs for Event-based Automation
AWS Lambda is a serverless computing platform that executes code in the form of functions. It does this without the need for you to manage any servers. Lambda, when combined with log data, can be a powerful way to enforce security.
While CloudTrail gives you alerts of basic violations like calls coming from unknown sources, it is unable to spot and notify you of more complex concerns, like known sources making bad API calls. In this case, you need to bolster CloudTrail with a custom Lambda function so that it automatically alerts you when a suspicious API call occurs.
As you can see from this illustration, CloudTrail stores its logs in an S3 bucket. Once in S3, Lambda executes a function to parse and filter the log data to identify logs that are suspicious. When found, it routes the specific lines of logs to SNS, from which you can subscribe to receive an email notification.
It definitely helps to be notified in real time when there’s a security violation, but even better, is the ability to set up automated rules that can be triggered from your log data. To do this, you could subscribe AWS Lambda to a real-time feed of logs from CloudWatch Logs. When a particular event occurs, you can execute a custom function in Lambda. For example, you could set up your CloudWatch Logs to notify you if a new instance is created with a vulnerable configuration, and you can set up a Lambda function to automatically delete this instance and notify you of it.
This sort of event-based automation using logs takes security to the next level. It lets you be proactive about security by building complex mechanisms to spot and fix vulnerabilities, and it does this in real time. With security, every second matters, and automation, when done right, can save the day.
AWS services are deeply intertwined with each other, and this involves a steep learning curve, but it’s well worth the effort considering the potential. CloudWatch Logs and CloudTrail are the default logging services from AWS, but to get the most out of your logs, you need to integrate these services with other AWS services like IAM, KMS, and Lambda. It’s crucial to use logs not just when an incident occurs, but to stay proactive and respond in real time when it comes to security in AWS.
If you’d like to learn more about how to manage Security using Sumo Logic, AWS Best Practices – How to Achieve a Well Architected Framework and AWS Well Architected Framework – Security Pillar.
About This Post
AWS Security Best Practices: Log Management is published by the Sumo Logic DevOps Community. If you’d like to learn more or contribute, visit devops.sumologic.com. Also, be sure to check out Sumo Logic Developers for free tools and code that will enable you to monitor and troubleshoot applications from code to production.
