How to gracefully control CDN traffic in cloud development?

In the community, many small partners who use cloud development reported that they encountered the situation of "CDN traffic consumption is like water".

Woke up for an excessive amount:

Some are "scared" by the loading of high-quality images:

So the question is, how to gracefully control CDN traffic consumption in cloud development? This article will talk to you in detail!

01. Pay-as-you-go and pipeline payment

In order to facilitate understanding, let's take a look at the traffic accounting model developed by the cloud.

To put it simply, billing by volume means: as much as you have run out, you will be counted as much; but it does not limit the amount you run out at the same time, that is, there is no limit to the peak traffic.

If you need to run out of 100M at the same time, then the peak value will drive you to 100M, and you will run at the same time, and the final cost will be 100M.

The pipeline payment is to limit the width of the "road" you can open, and charge according to the width of the "road". For example, if you choose a 1M network road, then you will be charged at the price of 1M; when you use it, your amount will be charged. It can only reach a speed of 1M, and it can't be higher anymore. This maximum speed is called bandwidth.

But even when there is not much volume at the low peak, you still have to pay for this "road". Still in the above example, you need to run out 100M at the same time, but the speed can only drive you to 1M, then these quantities can be completed after 100 seconds, and the final cost is 1M road rent * rental duration.

In real projects, there will always be business peaks and low peaks, and it is almost impossible for the traffic to maintain a constant rate at all times. This will cause the pipeline billing to have a speed ceiling, but it is still a waste of billing resources when it is idle. problem.

As a serverless cloud-native integrated back-end service, cloud development provides traffic that is pay-as-you-go. It does not support pipeline payment for the time being, and does not limit the traffic speed at the same time. Therefore, if the developer uses it unreasonably, it will cause the flow rate to be too fast. This results in a faster flow rate.

After understanding the difference between pay-as-you-go and pay-by-channel, let's move on to the topic and talk about CDN.

02. What is CDN traffic?

When we use cloud development, we see the term CDN almost everywhere.

CDN is also known as content distribution network. Generally speaking, it copies the files in your main storage (origin site) to storage points (CDN nodes) in various places. When users access this resource, directly from the nearest storage point (CDN) Node).

Cloud storage and static website hosting developed by the cloud naturally support CDN acceleration, so the traffic that your users download files through the client is CDN traffic.

The above CDN traffic is only applicable to the situation where various channels are used to speed up the download of files from the public network, such as browser loading websites, client downloading resources, external systems requesting files, and various access to temporary addresses to open files are all consuming CDN traffic.

However, if the file is accessed from the intranet, CDN traffic consumption is not used, such as accessing files through fileID in cloud functions.

03. What is CDN back-to-source traffic?

When there are file updates in our storage, how to deal with the old files stored in the CDN node? Introduce a knowledge point here-cache time.

The cache time here is actually the valid time of the file copy in the local storage point (CDN node), for example, the default is two hours, then the valid time of each file copy in the local storage point is two hours, and the request will be received after this time. At the time, the storage point (CDN node) will discard the expired old files and request the latest file from the main storage (origin site), and the traffic generated by this request is called the CDN back-to-source traffic.

The caching time cannot be too long or too short. If the CDN caching interval is too short, the data on the CDN node will often become invalid, resulting in frequent return to the source, increasing the load on the origin site, and affecting the overall transmission efficiency; if If the cache interval is too long, it will cause serious business problems such as untimely data updates.

Cloud storage developed by the cloud provides very fine-grained cache time settings. You can set them separately for a file, a path, or even a file suffix.

In addition, there are priority policies in multiple cache rule settings, making deployment more flexible.

Cloud storage is based on a back-to-front configuration mode for strategy calculation. For example, a cloud storage domain name has the following cache configuration:

Now request the resource /test/abc.jpg under this path, and its matching method from back to front is as follows:

Match all files of the first item, hit, and the cache time is 2 minutes at this time.

Match the second one, miss.

Match the third one, hit, and the cache time is 300 seconds at this time.

Match the fourth item, hit, and the cache time is 400 seconds at this time.

Match the fifth item, hit, and the cache time is 200 seconds at this time.

04. How to manage CDN reasonably?

We need to grasp a principle:

Reduce the size and make good use of the cache.

Reducing the size means that when we develop applications, the multimedia files that we need, if there are no special requirements (such as sending the original image of a camera), need to be compressed as much as possible. Only by reducing the size of business resources can the consumption of traffic resources be fundamentally reduced.

In addition, when uploading multimedia resources, users can still use the platform or framework capabilities to compress the resources before uploading, ensuring that the resources are optimized and stored in storage, which will reduce a lot of burden when requesting downloads.

Making good use of caching means that the same user and the same resource should not be requested more than once as much as possible. The local caching capability of the client should be used reasonably to cache all fixed resources. When the user enters here again, the cached resources are used directly.

I would like to give you another golden sentence from the elder Yunyun: The cache is used well, and the effect of PV only uses the amount of UV.

In the specific use, you still need to reasonably grasp it according to your own business situation. For example, if your resource changes very frequently, it is not suitable for cache optimization; and your resource cannot be compressed, and if you want to keep the original size, it is not suitable for compression optimization.

05、Conclusion

The above guide is for everyone who can’t hold CDN traffic. If you think that CDN is consuming like water and users are active but there is not much improvement, you may need to check where you are wasting. Diligence and thrift are the traditional virtues of the Chinese nation. Although cloud development is good, it also needs to be used sparingly~

Author: Yunyun Xuechang