Leveraging logs to better secure cloud-native applications

With the growing popularity of cloud computing, security incidents related to it have been on the rise. Logs are indispensable resources for countering these threats, and they can be utilized for alerting, taking remedial action, and even preventing future attacks. In this post, we will examine ways to better secure cloud-native applications using logs.

Audit Logs Are a Lifesaver

One of the most significant recent attacks occurred in 2019, when a hacker gained access to Capital One’s Amazon Web Services (AWS) S3 data store and stole 30GB of users’ financial data and personally identifiable information.

The Capital One attack was due to a misconfigured web application firewall (WAF) and over-privileged access to an AWS S3 bucket. It could have been prevented by correctly configuring the WAF or by more stringent provisioning of identity and access management (IAM) permissions. During the course of the attack, however, Capital One was able to use CloudTrail’s audit logs to quickly spot and respond to the breach.

While all organizations want perfectly configured firewalls, and ideally, employees who perfectly implement the principle of least privilege, they will inevitably experience misconfigurations and privilege creep. Along with following best practices, organizations need to invest in better security monitoring and auditing. Logs are invaluable in this regard.

Organizations should aim to identify vulnerabilities immediately, or better yet, before they occur. The logs that help spot these vulnerabilities need to capture all user activities – especially the suspicious ones. They need to be used to audit configurations and privileges across the system.

Logs need to be intentionally captured for specific parts of cloud services, including S3 bucket logs as well as object logs. Object logs are more comprehensive, and they capture API-level activity such as the creation and deletion of buckets.

Use Logs to Enforce the Principle of Least Privilege

Role-based access control (RBAC) can be easily abused to give large groups of users privileges that they don’t need. This greatly increases the attack surface of the system. With the principle of least privilege, users are granted only the privileges they need to perform their day-to-day routine tasks, and they must request any additional permissions from the Administrator. The latter should be able to set up automated rules to allow certain users access to certain privileges. This access can be temporary and set to expire after a certain time.

Logs can help enforce the principle of least privilege – they can be scanned in order to spot privilege creep, or suspicious activity initiated by privileged users. However, the logs must be scanned continuously for this type of security scanning to be effective. Organizations must intentionally set up mechanisms to alert when vulnerabilities are spotted.

Istio’s Access Logs Add Context

Service meshes are becoming commonplace in cloud-native systems. They provide additional opportunities for better observability and security monitoring. Istio generates telemetry data on service requests in the form of logs. The various components of Istio, such as Envoy, Mixer, and Pilot, have their own logs that can help with monitoring.

Envoy, for example, generates access logs. Megan O'Keefe of Google Tweeted the following image that shows the rich information contained in Envoy access logs:

While these logs aren’t the first place to look during an investigation of a security incident, they can help add important context.

Falco for Powerful Rules-Based Scanning

Falco is a very interesting open source project that is being incubated by the Cloud Native Computing Foundation (CNCF). It features a set of default checks for suspicious activity during runtime. These include activities such as privilege escalation, namespace changes, and the execution and mutation of binaries. It generates alerts in the form of logs to STDOUT or via gRPC calls.

The best part is that Falco allows you to create your own rules. You can even create very intricate rules using Falco’s Macros feature. The macros are like building blocks that can be reused across various rules. This feature can save you time when you are creating rules, and it can help unify them as well.

Falco’s Lists feature lets you create lists of items that can be used as input when creating rules. This enables you to build a very powerful and complex rules engine that is completely customized to your stack, and it makes your rules all the more powerful.

Route Logs to Better Control Them

Cloud-native systems generate large volumes of log data. It’s easy to miss the signal in the midst of the noise. The solution is to have a mature system that routes and aggregates log data from multiple destinations to multiple sources.

Fluentd and Logstash enable this type of log routing and bring granular control to the ways in which logs are viewed and analyzed. They ensure that logs are captured end-to-end, and they can transport logs to specific destinations as needed. This helps with security, because logs are more likely to yield insights and speed investigations when they are viewed in a consolidated context.

In conclusion, cloud-native applications are dynamic – and they are like moving targets for DevSecOps teams. However, logs are powerful tools in the hands of security professionals. If your logs are set up appropriately, aggregated from all of your sources, and analyzed using modern tools, they can help mount a coordinated response and even prevent security breaches from happening in the first place.