Automatically detect operational issues in Lambda functions with Amazon DevOps Guru for Serverless

foreword

We announced in May 2022 the Amazon DevOps Guru for Serverless , a brand new feature for Amazon DevOps Guru https://aws.amazon.com/devops-guru/ . This feature enables developers to improve the performance and availability of serverless applications. The product link is available here: https://aws.amazon.com/devops-guru/features/devops-guru-for-serverless

Amazon Cloud launched Amazon Lambda in 2014 https://aws.amazon.com/blogs/aws/run-code-cloud/ , pioneering the field of serverless computing. Thousands of customers are using Amazon Lambda today, and Amaon Lambda allows you to configure many parameters for your functions, such as memory allocation, provisioned concurrency, and timeouts. Finding the right balance between all these parameters to optimize the performance and usability of their functions is a daunting task for many customers. The product link can be clicked https://aws.amazon.com/cn/lambda/

In December 2020, we announced Amazon DevOps Guru, a fully managed AIOps (Artificial Intelligence for IT Operations) service that automatically detects application issues and alerts customers to help them improve application availability https ://aws.amazon.com/es/blogs/aws/amazon-devops-guru-machine-learning-powered-service-identifies-application-errors-and-fixes/ . Today, we are announcing Amazon DevOps Guru for Serverless, a new feature of Amazon DevOps Guru that helps developers using Amazon Lambda automatically detect abnormal behavior at the function level and use machine learning-based recommendations to remediate detected any questions .

Amazon DevOps Guru for Serverless uses machine learning to automatically identify and analyze various performance and availability-related issues with Amazon Lambda functions, such as low provisioned concurrency or under-utilization of memory. You don't need to be a serverless or ML expert to use this feature.

The reactive insights from this feature help you effectively resolve ongoing issues affecting serverless applications and provide actionable recommendations to help you identify root causes and resolve issues in the shortest possible time.

Amazon DevOps Guru for Serverless also provides proactive insights to help you spot broader operational anomalies before serverless application performance is impacted. It also gives you advice on how to find the root cause and fix the problem.

When a problem is detected, Amazon DevOps Guru for Serverless displays the findings in the Amazon DevOps Guru console using Amazon EventBridge https://aws.amazon.com/eventbridge/ or Amazon Simple Notification Service (Amazon SNS) https:// aws.amazon.com/sns/ to send notifications. This allows developers to automatically manage discovered issues and take immediate action.

Amazon DevOps Guru for Serverless Proactive Insights

Amazon DevOps Guru for Serverless enables developers to proactively detect application issues before incidents affecting customers occur. For example, if an Amazon Lambda function's provisioned concurrency is set too low and the application's traffic is growing, Amazon DevOps Guru will detect the increased traffic and decrease in application latency and generate proactive insights that show the problem.

Machine learning algorithms derive these insights from operational data and application metrics. Insights provide high-level information, severity, status, and recommendations on how to resolve the issue.

Today, Amazon DevOps Guru for Serverless provides proactive insights for Amazon Lambda and Amazon DynamoDB. The following are currently available operational issues and proactive insights:

• Amazon Lambda Concurrent Executions Reached Account Limit – Triggered when the number of concurrent executions consecutively reaches the account limit.
• Amazon Lambda Provisioned Concurrent Function Limit Exceeded – Triggered when the reservation for provisioned concurrency is insufficient for a period of time.
• High Amazon Lambda timeout compared to SQS visibility timeout – Triggered when the duration of an Amazon Lambda function exceeds the visibility timeout of the event source Amazon Simple Queue Service (Amazon SQS).
• Amazon Lambda Provisioned Concurrency Utilization Lower Than Expected – Triggered when the provisioned concurrency utilization is too low.
• Account read/write capacity consumed by Amazon DynamoDB reaches account limit – Triggered when an account consumes capacity close to the account-level limit for a period of time.
• Amazon DynamoDB Table Read/Write Consumed Capacity Reached Table Limit – Triggered when a write or read in the table reaches the table's ProvisionedWriteCapacityUnits or ProvisionedReadCapacityUnits limit for a period of time.
• Amazon DynamoDB Table Consumed Capacity Reaches AutoScaling Max Parameter Limit – Triggered when the table consumes capacity reaches the AutoScaling Max parameter limit for a period of time.
• Amazon DynamoDB read/write consumption is lower than expected – Triggered when the value of ProvisionedWriteCapacityUnits or ProvisionedReadCapacityUnits is significantly different from the consumption over a period of time.

Getting Started with Amazon DevOps Guru for Serverless

To get started, navigate to the Amazon DevOps Guru console to enable the service for Amazon Lambda-based applications, other supported resources, or your entire account.

In this demo, create a new Amazon Lambda function with a provisioned concurrency of 1. You can do this through the Amazon Cloud Console or programmatically. After creation, you can see the preset concurrency is set to 1 on the function overview page.

Add a CloudWatch event to the Amazon Lambda function that triggers the function every minute. You can do this through the Amazon Cloud Console or programmatically. You can follow this tutorial to learn how. Repeat the process five more times. Now the function will fire six times per minute with different events.

To trigger proactive insights, you need to make six concurrent calls to this Amazon Lambda function. To do this, you need to make sure that each call lasts long enough. In this demo, you can put the function to sleep for 30 seconds.

 'use strict';

exports.handler = async (event) => {

    console.log('Sleep for 30 seconds')
    await new Promise(r => setTimeout(r, 30000));
    console.log('finish sleeping')

    return;
};

Swipe left to see more

This configuration will trigger proactive insight that this function exceeds the Amazon Lambda provisioned concurrent function limit. You should see this insight in the console three hours or less after the issue started.

How to View Insights Through the Amazon DevOps Guru Console

After a few hours, you can access the Amazon DevOps Guru console and can verify that proactive insights were triggered by exceeding provisioned concurrency.

Select Ongoing insights to see more details. The Insights page opens, displaying information related to insights, metrics, events, and suggested actions for this issue.

Let's take a look at this page in more detail. At the top of the page is the insight overview, which contains the content of the insight and a description of the severity of the problem. This is a proactive insight, so the user experience is not affected by this issue. You can also find out if the problem persists and when it started. If the problem doesn't recur, you can find out the end date for the insight. If you select the link for the affected application, you can identify all Amazon Lambda functions affected by this insight.

The next info box contains information about CloudWatch metrics related to proactive insights. This chart shows the ProvisionedConcurrecySpilloverInvocations metric, along with a summary of all invocations over the last few hours for provisioned concurrency spills.

Relevant events are the next information boxes on the page. These are Amazon CloudTrail events that Amazon DevOps Guru uses in conjunction with CloudWatch metrics and operational data to identify anomalous behavior that yields insight.

At the end of the page is the Recommendations information box, where the Amazon DevOps Guru will output any recommendations generated to help you resolve the issue. You can use these suggestions to see what steps you can take to fix the problem right away.

In this proactive insight, the Amazon DevOps Guru recommends that you adjust the provisioned concurrency of your Amazon Lambda functions. This suggestion tells you which value to set the function to based on past usage of the function. You can also find reasons why the Amazon DevOps Guru recommends this insight.

Pricing and Availability

Amazon DevOps Guru for Serverless is free to customers.

Amazon DevOps Guru for Serverless is available in all Amazon Cloud Tech coverage regions where Amazon DevOps Guru is available: US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney) , Asia Pacific (Tokyo), Europe (Frankfurt), Europe (Ireland), and Europe (Stockholm).

Learn more about Amazon DevOps Guru for Serverless https://aws.amazon.com/cn/devops-guru/features/devops-guru-for-serverless/ to learn more about this new release.

Author of this article

Marcia Villalba

Amazon Cloud Technology Developer Evangelist

She has been designing and developing software professionally for 15 years and has worked in all stages of building scalable and high-performance software. She has a solid understanding of building applications in the cloud and using DevOps processes.