Ffmpeg combat-audio and video synthesis case

Forwarded from Bailang Stack: View original text

Many friends left messages in private and said that they have not been in touch with audio and video before, and they are still a little confused about what ffmpeg can do.

Today, let’s take a look at what can we do with ffmpeg?

Many friends should have played Douyin, have you paid attention to the "in synthesis" prompt when you "shoot the same style", the longer the video, the longer the synthesis time, ranging from a few seconds to tens of seconds , This is actually the local video synthesis in the app.

Let the user wait so long, feel that the experience is very poor. Can the server be synthesized? The answer is "Yes".

But in fact, ffmpeg eats cpu and memory very much when encoding, especially cpu, server cpu is 100% commonplace. Wouldn't it be over if you increase the cpu? We will actually test 8-core 8G and 16-core 16G machines later, and find that it will still explode, so this is a bottomless pit and can't afford it.

So can Douyin let the server do it? Definitely not. The number of users is large, and the app can do it locally. The most important thing is free users.

So considering the cost of technology, this is why many special effects on Douyin are short videos, some even only 10 seconds, at least this will not make users wait too long.

Before learning audio and video related knowledge, let's briefly look at two application scenarios. If you happen to have some basic audio and video skills, you can practice your hands by the way.

Scene 1. The company has a lot of original videos, and I want to better show it to users with a certain effect (such as TV playback)

The desired effect is shown in the figure

The material is more difficult to find. The picture is from Baidu, and the part of the video displayed in the middle of the TV has been processed by PS, which is slightly rough; the video is downloaded by Douyin, and the original video is also partially blurred.

I believe that every company has a good UI, and the current case is only for learning.

Points to note:

1. The video can be cropped, zoomed, and cannot be deformed
2. Note that the four corners of the inner screen of the TV are rounded
3. Only the middle part of the original video is taken, and there is no Gaussian blurred background
4. The part with Douyin at the end of the original video is also cropped

Material has been uploaded to github, please click on the link for .

download picture material

download video material

Download synthesis effect

Scene 2. The company has a lot of material videos and corresponding titles and subtitles, and I want to open it to users to dub and synthesize my own works

The desired effect is shown in the figure

The material is also from Douyin, the left video is the user video, the middle part of the right video is the video material, and the bottom is the Gaussian blur video of the material. In addition, the title, subtitles, and fonts are all prepared in advance.

Material has been uploaded to github, please click on the link for .

download the left user video

download the right material video

download the subtitle file 1 on the right , subtitle file 2

download font

final synthesized video effect

There may be many different ways of doing the same effect. If you spend some time doing it all, congratulations.

In order to reduce server resources and synthesis time, please consider how these two video effects can be completed with one command each.

Friends who are exposed to audio and video for the first time may be surprised by such a synthesis effect. After learning the basics of ffmpeg, you should at least use one command to solve the above and even more complex audio and video needs.