Restful API-based CloudFront Distribution Copy/Clone via Amazon API Gateway and Amazon Lambda
background
Amazon CloudFront is a global Content Delivery Network (CDN) that lets you deliver content to viewers or end users with low latency and high availability. Generally speaking, Amazon CloudFront customers have multiple CloudFront Distributions, and each distribution contains a set of content to be accelerated and related configuration information, such as origin site, accelerated domain name, cache policy, certificate, log, access control, etc. .
The following topics explain some of the basics about CloudFront Distribution and provide information about the settings you can choose to configure your Distribution to meet your business needs. In the process of CloudFront automation, a common task is to create multiple distributions. The configuration parameters of these distributions are exactly the same under the same type of job. For example, in the dynamic acceleration scenario, the cache TTL is set to 0. In this scenario, customers often need a configuration item that clones an existing distribution, and hope to use the standard Restful API interface to quickly create multiple domain name distributions with the help of programs. At present, the API for new distribution supported by Amazon Cloud Technology: CreateDistribution , needs to provide complete configuration parameter information, and temporarily does not support the function of copying or cloning distribution.
The solution introduced in this article is to use Amazon API Gateway and Amazon Lambda , the backend is based on Amazon SDK for Python (Boto3) , to realize the configuration information based on a reference distribution, and only modify the changes such as domain name and origin site to copy a new distribution Restful API.
Solution Brief
The basic implementation ideas of the solution for the replication/clone function of CloudFront Distribution are as follows:
1) Copy the configuration information that already has a reference Distribution;
2) Modify variable items, such as the acceleration domain name CNAME of the new Distribution and the domain name of the origin site, to form new complete configuration information;
3) Perform the operation of creating a new Distribution.
During this process, the relevant API of Amazon Certificate Manager will be called to find the ARN of the ACM certificate corresponding to the new CNAME, so as to complete the certificate association operation of the accelerated domain name. The process of querying the certificate ARN is transparent to the user.
The detailed implementation process of the solution is as follows:
1) Obtain the configuration information of the reference domain name as the benchmark configuration (function: get_reference_config)
Call the Amazon Boto3 API get_distribution_config , enter the distribution id, and obtain the distribution configuration information DistributionConfig in the form of Json;
2) Obtain the corresponding list of ACM certificates and domain names under the account (function get_certificate_mapping)
Call the Amazon Boto3 API list_certificates , enter CertificateStatuses='ISSUED', and query the corresponding list of issued certificates and domain names in ACM under the account;
3) Get the ACM certificate corresponding to the CNAME (function get_certificate_arn)
Obtain the ARN (Amazon Resource Names, the unique identifier of the resource name) of the ACM certificate corresponding to the origin site origin from the list found in step 3, which is used for the certificate parameters when creating a distribution in the next step;
4) Construct the configuration information of the new Distribution (function set_config_based_on_ref)
According to the benchmark DistributionConfig obtained in step 1, modify the domain name and origin site, add the certificate ARN, and create a new DistributionConfig
5) Create a new Distribution (function create_distribution)
Call the Amazon Boto3 API create_distribution , enter the DistributionConfig constructed in step 4, and create the required distribution.
Solution Architecture
The solution user interface generates a Restful API clone_distribution through API Gateway and the Lambda function cf_distribution_clone, and the function cf_distribution_clone will create a cloned Distribution according to the queryStringParameters in the trigger event. In order to further strengthen security management and restrict API access, in this example, API Gateway will enable Cognito authorization, and users who access the interface need to carry Cognito tokens to request the API normally. The architecture diagram of the scheme is shown below.
Solution deployment
1. Deploy the Lambda function
Create an IAM role cf-clone-distribution-role for Assume when Lambda executes, and create the following IAM policy for the role. Note that you need to replace <S3_bucket_name> and <account_id> with the bucket name and account ID of the S3 Bucket where CloudFront logs are located. . This policy has configuration query permissions for existing distributions, new distribution permissions, ACM certificate listing permissions, and permissions for querying and modifying the Bucket ACL of the S3 bucket where CloudFront logs are located. The example uses us-east-1 in the US East Region as a reference, which can be replaced according to the actual situation.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"acm:ListCertificates",
"cloudfront:CreateDistribution",
"cloudfront:GetDistributionConfig"
],
"Resource": "*"
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": [
"s3:PutBucketAcl",
"s3:GetBucketAcl",
"logs:CreateLogGroup"
],
"Resource": [
"arn:aws:s3:::<S3_bucket_name >",
"arn:aws:logs:us-east-1:<account_id>:*"
]
},
{
"Sid": "VisualEditor2",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:us-east-1: :<account_id>:log-group:/aws/lambda/cf_distribution_clone:*"
}
]
}Create the Lambda function cf_distribution_clone and set the Lambda execution role to the cf-clone-distribution-role created above. The runtime environment used by the solution is Python 3.9, and the corresponding complete Lambda code is shown below.
import boto3
from botocore.config import Config
import botocore.exceptions
from datetime import datetime, timezone
import logging
import json
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
cf_client = boto3.client('cloudfront')
my_config = Config(
region_name = 'us-east-1'
)
acm_client = boto3.client('acm', config=my_config)
def lambda_handler(event, context):
conf_domain = event['queryStringParameters']['domain']
conf_origin = event['queryStringParameters']['origin']
conf_ref_dist = event['queryStringParameters']['ref_dist']
# 1) 获取参考域名的配置信息做为基准配置
ref_config = get_reference_config(conf_ref_dist)
# 2) 获取账号下ACM证书与域名的对应列表
certs = get_certificate_mapping()
# 3) 获取该CNAME对应的ACM证书
certArn = get_certificate_arn(certs, conf_domain)
# 4) 构造新Distribution的配置信息
new_config = set_config_based_on_ref(ref_config, conf_domain, conf_origin, certArn)
# 5) 创建新Distribution
distribution = create_distribution(new_config)
response_body = {}
response_body['requestId'] = context.aws_request_id
response_body['distributionId'] = distribution['Distribution']['Id']
responseObject = {}
responseObject['statusCode'] = 200
responseObject['body'] = json.dumps(response_body)
return(responseObject)
def get_reference_config(ref_dist):
try:
return cf_client.get_distribution_config(Id=ref_dist)
except botocore.exceptions.ClientError as error:
logger.exception(f"{format(error)}")
raise error
def get_certificate_mapping():
try:
response = acm_client.list_certificates(
CertificateStatuses=[
'ISSUED'
],
MaxItems=1000
)
certs = response['CertificateSummaryList']
while "NextToken" in response:
response = acm_client.list_certificates(
CertificateStatuses=[
'ISSUED'
],
MaxItems=1000,
NextToken= response['NextToken']
)
certs.extend(response["CertificateSummaryList"])
cert_dict = {}
for cert in certs:
cert_dict[cert['DomainName']] = cert['CertificateArn']
return(cert_dict)
except botocore.exceptions.ClientError as error:
logger.exception(f"{format(error)}")
raise error
def get_certificate_arn(certs, domain):
if domain in certs:
cert = certs[domain]
else:
cert_domain = '*.' + domain.split(".", 1)[-1]
if cert_domain in certs:
cert = certs[cert_domain]
else:
logger.info(f"No certificate for domain - {format(domain)} in ACM. Please create or import one.")
exit(1)
logger.info(f"Use ACM certificate for domain \'{format(domain)}\': {format(cert)}.")
return cert
def set_config_based_on_ref(ref_config, conf_domain, conf_origin, certArn):
ref_config['DistributionConfig']['Aliases'] = {
'Quantity': 1,
'Items': [
conf_domain
]
}
new_config = ref_config['DistributionConfig']
new_config['CallerReference'] = str(datetime.now(tz=None).timestamp())
new_config['Origins']['Items'][0]['Id'] = conf_origin
new_config['Origins']['Items'][0]['DomainName'] = conf_origin
new_config['DefaultCacheBehavior']['TargetOriginId'] = conf_origin
new_config['Comment'] = conf_domain
new_config['ViewerCertificate']['ACMCertificateArn'] = certArn
return new_config
def create_distribution(config):
try:
distribution = cf_client.create_distribution(DistributionConfig=config)
logger.info(f"Done! Created distribution {format(distribution['Distribution']['Id'])}.")
except botocore.exceptions.ClientError as error:
logger.exception(f"{format(error)}")
raise error
return(distribution)The operation of the Lambda function part can refer to the example in the following figure.
2. Create API Gateway
Create an API Gateway execution method and add URL query string parameters (*Note: the following parameters are all lowercase).
- domain: The CNAME associated with the created Distribution, that is, the acceleration domain name;
- origin: the domain name of the origin site pointed to by the newly created Distribution;
- ref_dist: Refer to Distribution. For new distribution parameters, refer to the parameter configuration of Ref_dist, and only modify the CNAME and Origin domain names.
The operation of the API Gateway part can refer to the example in the following figure.
3. Deploy the identity authentication service Amazon Cognito
The API created by the default API Gateway is public and accessible to everyone, lacking the authentication part. This solution will create a Cognito user pool, domain name, resource server and application client to implement authentication, that is, only after the authentication is passed, the API can be accessed.
The operation of the Cognito part can refer to the example in the figure below.
Then return to the API Gateway page, create a Cognito authorizer in API Gateway and configure it in the corresponding API resource. The Cognito token needs to be configured in the Authorization header, as shown in the following figure.
4. Test verification
curl -X POST -u <application client ID>:<application client secret> ' https://clone-distribution.auth.us-east-1.amazoncognito.com/oauth2/token?grant_type=client_credentials ' -H 'Content-Type: application/x-www-form-urlencoded'
After executing the command, you will get an access token, as shown in the following figure:
Here, select Postman as the API testing tool, open the tool and add the header (the key is Authorization, and the value is the access_token in the above figure), enter the API link, and add the query string to send the request.
As shown in the image above, the CloudFront Distribution replicates successfully, returning the newly created CloudFront Distribution ID.
Summarize
This article introduces a Restful API implementation that replicates/clones CloudFront Distribution through serverless services Amazon API Gateway and AWS Lambda and Amazon CloudFront SDK, and provides a secure access interface in API Gateway access requests through Cognito. This solution is suitable for situations where the same configuration item needs to be applied to multiple distribution domain names. It can effectively simplify the customer's configuration management workload, reduce the probability of manual operation errors, and achieve the purpose of fast one-click deployment.
Author of this article
Ma Yuhong
AWS technical account manager, responsible for operation and maintenance and architecture optimization, cost management, project delivery, technical consultation, etc. Before joining AWS, he worked in IBM China Software Development Center and has experience in distributed software development. Currently committed to the research and practice of Edge, DevOps, Serverless and other directions.
Shi Tian
Senior Solution Architect of Amazon Cloud Technology. With rich experience in cloud computing, data analysis and machine learning, he is currently committed to the research and practice of data science, machine learning, serverless and other fields. Translations include "Machine Learning as a Service", "Kubernetes-based DevOps Practice", "Kubernetes Microservices Practice", "Prometheus Monitoring Practice", "CoreDNS Learning Guide in the Cloud Native Era", etc.
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。