How to Monitor Amazon ECS

As a system administrator, you will need to monitor your containers for a wide array of reasons. Between analyzing the health of your containers, avoiding resource constraints, and collecting, parsing, and visualizing data, one could easily get lost.

ECS Monitoring and Logging

When monitoring and logging data for ECS containers, system administrators and cloud engineers gain deeper insights which allow them to make better, data-driven decisions when it comes to orchestration, architecture, and performance efficiency.

It’s important to monitor your systems in order to detect patterns and abnormalities. Cutting out the noise by reducing false positives increases productivity of your team by making actual alerts actionable. Logging and monitoring container data allows your team to prevent data outages, forecast intense resource needs and have a strategic, scalable plan.

Gathering Data

There are a few ways you can collect data and logs from ECS and ingest them into Sumo Logic.

1. Installing a collector/agent on the ECS host
2. Leverage the Sumo Logic Docker collector container.
3. Using the Sumo Logic Docker logging driver

For simplicity sake and in the context of this post, we’ll focus on the most common option, by installing the collector/agent on the ECS host.

Installing the Sumo Logic collector on your ECS host allows for several things. You can measure things like CPU, Memory, Network latency and Disk Use, as well as take advantage of your hosts local file source for things such as OS, ECS, and other system service logs directly from the source versus being relayed from another service or location.

How to Leverage Data

Collecting data is one thing. Harnessing that data, transforming it into something useful, and leveraging it to make impactful business and technical decisions is some next-level monitoring strategy.

By leveraging your data you collect from your ECS containers, you and your team can make data-driven decisions such as how to optimize your resources, save the organization money, highlight key improvement areas, and address any technical or performance issues.

If your team is running a web application and not collecting and leveraging data from your ECS containers, you might as well be driving at night with no headlights on. It is nearly impossible to make any kind of accurate technical decision without having a baseline level to measure your metrics with.

How will you know to increase your ECS container load during peak hours if you have no idea when customers access your web application the most? How will you know to scale your environment so you can anticipate the 3:00AM EST traffic spike you get from Ireland since it’s 8:00AM across the pond?

By leveraging your data from your Sumo Logic data collector agent, you and your team can guide your organization to make better, more efficient decisions, with less error.

Amazon ECS Metrics

You can monitor your organization's Amazon ECS container resources by utilizing other Amazon AWS resources such as Amazon CloudWatch, or gain deeper insights consolidating your data inside Sumo Logic.

Amazon CloudWatch is a fairly simple tool that collects and processes raw system and server data from your environment, in this case your Amazon ECS containers, and parses it into human-readable real-time metrics. However, your features are limited and you’ll have difficulty correlating the metrics.

As previously covered in the prior post, Amazon ECS includes several metrics that can be tracked, monitored, and logged. From CPUReservation to Memory Utilization, Amazon ECS offers a number of ways to track your system and container health.

How to Monitor ECS with Sumo Logic

As already mentioned, collecting data with Sumo Logic and Amazon ECS can be done in several manners. The important thing is that data is collected so we can get to monitoring.

Amazon ECS can be monitored in a variety of ways as well by offering a clean, easy to use, and human readable dashboard that can provide insight into data points not often covered by the automated and manual monitoring tools that already come with Amazon ECS.

Some of those panels include:

• Cluster Count
• Service Count
• Number of Services
• Average CPU Utilization by Service
• Average Memory Utilization by Service

Sumo Logic can also help cover key Events within your Amazon ECS environment. Events can be filtered and monitored by:

• Type
• Time frame
• Location
• Resources Created & Deleted Over Time
• Events by IAM User.

Conclusion

As you can see, Sumo Logic goes several levels deeper than the basic and surface level monitoring tools Amazon AWS and Amazon ECS offer.

In the next post, we’ll be going over Amazon ECS Monitoring with Sumo Logic to give you the ins and outs of the power of Sumo Logic and how you can fully customize your monitoring tools for your teams needs.