ECS" class="redactor-autoparser-object">https://www.sumologic.com/blog... (EC2 Container Service) is a container management service that lets you manage your containerized applications in AWS. It uses EC2 instances to host Docker containers, and is a great option if you’re a long-time AWS user, and want the familiarity of the AWS platform as you make the transition to containers. In this post we discuss the challenges of monitoring an app in ECS, and why you need more than just CloudWatch to monitor ECS.
In ECS, container instances are EC2 instances that can be connected to form a cluster. These instances are run by Tasks which can be controlled by Task Definitions. To run an app on ECS, it needs to be architected with containerization in mind.
For many organizations, a key motivation for moving to ECS is to run their apps as a collection of microservices. Each service is handled by a cluster of containers on ECS. By architecting your app this way, you can manage each service individually, deploy them separately, isolate faults between them, and easily replace failed instances. However, it makes the entire process of developing and shipping an app more complicated. Let’s discuss some of the challenges with running containerized apps on ECS.
Challenges with ECS Monitoring
- Distributed apps are complex
Microservices bring a lot of management overhead that can’t be handled with traditional tools and approaches. ECS is made up of many components like tasks, task definitions, container instances, and clusters. The health of each part of the system and how they communicate with each other needs to be continuously monitored. - New container instances need real-time monitoring
You’ll likely be spinning up new container instances on a daily basis. Research shows that the average lifespan of containers is 10 hours, and that number is bound to decrease further as Docker becomes mainstream. This frantic pace of operations needs real-time monitoring. - Root-cause analysis takes longer
With the high volume of data and logs to monitor, troubleshooting needs a different approach. You can’t make do with real-time data streams—You need to be able to gain insights and take action on the data in real-time. You need a monitoring tool that does the heavy lifting and identifies patterns and anomalies from log data.
CloudWatch Delivers the Vital Monitoring Stats
CloudWatch is AWS’ default monitoring tool that provides monitoring for all AWS services. It captures performance data like CPU, memory, and network utilization. While the free tier gives you updates in five-minute intervals, you can pay for one-minute update intervals, which makes a big difference when monitoring containerized apps.
CloudWatch Logs collect log data of ECS cluster performance by setting up a purpose-built container within your ECS cluster. CloudWatch works great to collect performance data on all your AWS services. However, it lacks the advanced analysis tools to gain operational insight from the data. To gain deep visibility into your ECS clusters you need to augment CloudWatch with a monitoring tool that’s purpose-built for analyzing performance data at scale.
Sumo Logic for Advanced Machine Data Analysis in ECS
The Sumo Logic App for Amazon ECS, which is currently in Preview, provides preconfigured searches and Dashboards that allow you to monitor various metrics like CPU and Memory Utilization, CPU and Memory Reservation, across ECS clusters and services. The App also monitors API calls made by or on behalf of Amazon ECS in your AWS account.
The App collects ECS logs and metrics for:
Benefits of of using the Amazon EC2 Container Service (ECS) App include:
- Quicker root-cause analysis and resolution
Sumo Logic sources the data from CloudWatch in real-time, and applies machine learning so you can identify patterns and drill down to the root-cause in minutes. You won’t have to wade through thousands of lines of repetitive log data. Sumo Logic will surface the exact outliers for you so that you can get straight to resolving the issue. - Powerful machine data visualization
Sumo Logic presents data in an intuitive dashboard that lets you find correlations between errors and how they affect the various components of your ECS clusters. For example, in just a few clicks you can overlay CPU with memory performance and confirm if a spike happened at the same time. - Integrated reporting for ECS and infrastructure
If you plan to make a phased migration to ECS, and need an integrated platform that can monitor not just ECS, but the rest of your infrastructure as well, Sumo Logic is a great choice. Integrated monitoring will allow you to compare ECS performance with other parts of your infrastructure.
ECS is gaining traction as more organizations make the move to containerize their applications. They want an integrated, easy-to-manage platform that they’re familiar with, and that’s what AWS ECS is. However, the move to microservices comes at the cost of added complexity at every step of the application lifecycle. It takes a deliberate monitoring strategy to realize the benefits of microservices. That’s what the combination of CloudWatch and Sumo Logic delivers.
