How to use Serverless to achieve batch, automation and customization of video clips

foreword

Before we start talking, let’s solve the 3 doubts in everyone’s mind when they see this title:

1. Can video editing be done without Adobe software?
2. Why use Serverless?
3. How to write code for video editing?

First of all, let’s talk about which video editing scenes cannot be done by software such as Adobe.

The video editing that everyone usually comes into contact with usually uses professional tools such as Premiere and AE to complete the video editing. They can complete some complex effects, such as making promotional videos, advertising videos, etc.

However, some companies expect to complete video in and automatically in certain business scenarios.

For example the following scenarios:

1. Suppose the school expects to present all the wonderful videos of the students' learning process immediately after the students finish the online class, with the school's logo and slogans, etc., so that students can share their achievements with one click. Suppose there are 10,000 students, and a unique video needs to be made for each student, so 10,000 different video clips need to be completed in batches and automatically.
2. In a marketing campaign, different avatar videos need to be generated for different users to attract users to participate. Each user's avatar is unique, and the generated video is also unique. There may be thousands of users, so automation is a must.
3. The Internet celebrity operating company expects to generate a unified business video for all anchors. There may be 100 anchors, and it seems unacceptable to find one person to edit 100 videos, but what if you have to cut a different video every week? So automation, batch and customizable editing became the main requirement.

The above scenario has three characteristics:

1. batch
2. automation
3. customizable

For scenes that meet the above characteristics, traditional video editing tools or template-based video processing software cannot be easily completed.

Let's talk about why to use Serverless

Because a business like video editing has several characteristics:

• Use time focus.
• The amount of calculation is large.

The utilization rate of high-spec servers purchased separately is very low, and the computing power of cheap servers cannot keep up.

Therefore, the features of serverless billing and high-performance computing capabilities perfectly match such demand scenarios.

It can not only achieve 100% utilization, but also use its high-performance computing power according to the amount.

At the same time, Serverless has a changeable programmable environment, can use familiar programming languages, and is highly flexible.

Finally, let's talk about how to write code to make video clips

All the functions of video editing mentioned in this article use the tool FFmpeg, so let me tell you what FFmpeg is first.

FFmpeg is an open source tool for video processing, it has very powerful functions, it supports video editing, video transcoding, video editing, audio processing, adding text, video splicing, pull streaming and live streaming and other functions.

We can program different video editing functions through different FFmpeg commands, and arrange them to deal with various batch automation scenarios.

Clip Batching, Automation and Customization Practice

Common video editing scenes mainly include the following:

1. video transcoding
2. video cropping
3. Video plus text
4. Video plus picture
5. Video stitching
6. video plus audio
7. video transition
8. Video effects
9. Video accelerated slow playback

Next, I will show you some specific FFmpeg command examples. If you have installed FFmpeg locally, you can also execute these commands locally. About how to install FFmpeg, you can go to the official website tutorial.

// 将MOV视频转成mp4视频
ffmpeg -i input.mov output.mp4

// 将原视频的帧率修改为24
ffmpeg -i input.mp4 -r 24 -an output.mp4

// 将mp4视频转为可用于直播的视频流
ffmpeg -i input.mp4 -codec: copy -bsf:v h264_mp4toannexb -start_number 0 -hls_time 10 -hls_list_size 0 -f hls output.m3u8

// 将视频分别变为480x360，并把码率改400
ffmpeg -i input.mp4 -vf scale=480:360,pad=480:360:240:240:black -c:v libx264 -x264-params nal-hrd=cbr:force-cfr=1 -b:v 400000 -bufsize 400000 -minrate 400000 -maxrate 400000 output.mp4

// 给视频添加文字，比如字幕、标题等。
// `fontfile`是要使用的字体的路径，`text`是你要添加的文字，
// `fontcolor`是文字的颜色，`fontsize`是文字大小，`box`是给文字添加底框。
// `box=1`表示enable，`0`表示disable，`boxcolor`是底框的颜色，black@0.5表示黑色透明度是50%，`boxborderw`是底框距文字的宽度
// `x`和`y`是文字的位置，`x`和`y`不只支持数字，还支持各种表达式，具体可以去官网查看
ffmpeg -i input.mp4 -vf "drawtext=fontfile=/path/to/font.ttf:text='你的文字':fontcolor=white:fontsize=24:box=1:boxcolor=black@0.5:boxborderw=5:x=(w-text_w)/2:y=(h-text_h)/2" -codec:a copy output.mp4

// 给视频添加图片，比如添加logo、头像、表情等。filter_complex表示复合的滤镜，overlay表示表示图片的x和y，enable表示图片出现的时间段，从0-20秒
ffmpeg -i input.mp4 -i avatar.JPG -filter_complex "[0:v][1:v] overlay=25:25:enable='between(t,0,20)'" -pix_fmt yuv420p -c:a copy output.mp4

// 视频拼接，list.txt里面按顺序放所有要拼接的视频的文件路径，如下。
// 注意，如果视频的分辨率不一致会导致拼接失败。
ffmpeg -f concat -safe 0 -i list.txt -c copy -movflags +faststart output.mp4
// list.txt的格式如下
file 'xx.mp4'
file 'yy.mp4'

// 视频加音频，stream_loop表示是否循环音频内容，-1表示无限循环，0表示不循环。shortest表示最短的MP3输入流结束时完成编码。
ffmpeg -y -i input.mp4 -stream_loop -1 -i audio.mp3 -map 0:v -map 1:a -c:v copy -shortest output.mp4

There are many things that FFmpeg can do, so I won't explain them one by one here. More ways to play can be explored on FFmpeg official website .

The same is true for audio editing, FFmpeg also supports audio editing alone.

How to run FFmpeg commands

Because Python is more convenient to run these commands, we can use python to run all FFmpeg commands. At the same time, python's performance on serverless cloud functions is also better, and the deployment is also convenient.

There is an open source link at the end of the article to use FFmpeg's video clip code through Python. And there are also templates on the official website that can be used directly, covering common operations such as audio and video editing.

Here is a simple calling code example.

child = subprocess.run('./ffmpeg -i input.mov output.mp4',
                               stdout=subprocess.PIPE,
                               stderr=subprocess.PIPE, close_fds=True, shell=True)
if child.returncode == 0:
  print("success:", child)
else:
  print("error:", child)
    raise KeyError("处理视频失败, 错误: ", child)

Deploy on serverless

I have implemented and open-sourced the common video editing scenes mentioned above, and the downloaded code can be deployed directly to serverless.

https://github.com/woodyyan/ffmpeg-composition

https://github.com/woodyyan/ffmpeg-splice

There are two functions here, one is responsible for processing a single video, and the other is responsible for splicing multiple videos into one video with background music.

The following features are currently supported:

1. Add text to video
2. video resolution conversion
3. Add an image to the video
4. Video stitching
5. Add background music

The source code shows only some common video editing scenes. You can write your own video editing logic according to your own business needs.

Serverless deployment

Method 1: Github Action automatically deploys

1. Fork warehouse.
2. Add two keys TENCENT_SECRET_ID and TENCENT_SECRET_KEY to the Settings-Secrets-Actions of the repository. ID and KEY can be obtained in Tencent Cloud's access control.
3. After adding, deployment can be initiated in Action. Action deployment is also automatically triggered after each modified code is pushed.
4. If you need some custom configuration, please modify serverless.yml.
5. Cloud functions will eventually be automatically deployed to the account where TENCENT_SECRET_ID is located.

Method 2: Manually deploy in the cloud function console

1. Download the code.
2. Pack all files and folders together into a ZIP file in the root directory.
3. Go to cloud function console and create a new function.

4. Choose to start from scratch:

   1. Choose python language.
   2. Upload the ZIP file.
   3. It is recommended to choose a larger memory for function memory.
   4. Enable asynchronous execution.
   5. The execution timeout is recommended to be set longer according to the size of the video, such as more than 30 seconds.
   6. Configure triggers, select API Gateway triggers, turn off integration responses.
5. After the deployment is complete, you can start calling through the URL of the API gateway.

Real case review

A school that conducts online classes needs to make a 30-second video of the online class every time after the students finish the online class, as the student's learning outcome.

There are several key information points for this case:

1. Usually there are 200 students in a class, and 200 videos need to be produced at the same time.
2. The 1-hour class video needs to be edited into 30 seconds.
3. Since each student's class screen is different, the recorded video is different.
4. The final result video also needs to add the student's name and avatar.
5. Students end the class with a concentrated amount of time, so there will be short-term high concurrency when making videos.
6. Videos need to be made after each class, and the time period is relatively fixed and focused.

Combining the above characteristics, using Serverless to do such video clips brings multiple benefits:

1. Solved the problem of 200 concurrency, and you don't need to build too many servers yourself.
2. Solved the problem that it is only used in the occurrence period, and no cost is incurred in other periods.
3. It solves the problem of fast video production that requires strong computing power.

Below is a reference architecture diagram for this case.

Summarize

By arranging, combining, and multiplexing the scenes of various audio and video clips listed above, you can create a variety of desired effects.

Then, the parameters used to control various effects in the video clip are changed into the parameters passed in when calling the service, and the customization of various effects can be realized.

Finally, let's summarize the usage scenarios of video clips completed by this way of writing code:

1. Solve the scene of batch production of videos by modifying individual parameters.
2. Addresses scenarios where user-triggered video production is automated.
3. Solve different scenarios that require different customized video production scenarios.

At the same time, using serverless to complete video editing also solves the following problems:

1. Because video clips typically don't run all day, using the serverless pay-as-you-go feature can optimize costs.
2. Because video clips are often recomputing scenarios, take advantage of the optional high-spec configuration of serverless to handle this recomputing.
3. In the scene of batch production of videos, there is usually high concurrency. The automatic elastic scaling feature of serverless can easily cope with high concurrency.

If you have any questions about the use of Serverless or video editing, please leave me a message.