JavaCV's Camera Practice Five: Push Streaming

Welcome to my GitHub

All original works of Xinchen (including supporting source code) are classified and summarized here: https://github.com/zq2599/blog_demos

Overview of this article

• This article is the fifth part of "JavaCV's Camera Actual Combat". Let's consider a question: How can the content of the local camera be seen by others on the Internet?
• This involves streaming. As shown in the figure below, the application based on JavaCV pushes the video frame of the camera to the media server. The viewer can use the player software to remotely connect to the media server and watch the content of the camera:

• Today's main work is to develop the JavaCV application shown above, and then verify whether the function is normal;

coding

• "One of JavaCV's Camera Practices: The Basics" The parent class <font color="blue">AbstractCameraApplication</font> has been written in the <font color="red">simple-grab-push</font> project created by the article font>, this article will continue to use the project, just create subclasses to implement those abstract methods
• Before coding, review the basic structure of the parent class, as shown in the figure below, the bold font is each method defined by the parent class, and the red blocks all require subclasses to implement abstract methods, so next, we aim to achieve these three with the local window preview. The red method can be:

• Create a new file <font color="blue">RecordCamera.java</font>, which is a subclass of AbstractCameraApplication. Its code is very simple. Next, it will be explained in the order of the above figure.
• "One of JavaCV's Camera Practices: Basics" The media server has been deployed in . Here, a member variable is defined to save the streaming address of the media server. Please adjust it according to your own situation:

private static final String RECORD_ADDRESS = "rtmp://192.168.50.43:21935/hls/camera";

• Also prepare a member variable to add a timestamp to the frame when pushing the stream:

protected long startRecordTime = 0L;

• The function of pushing video frames to the media server comes from FrameRecorder, which is an abstract class. This article uses its subclass FFmpegFrameRecorder, so define member variables of FrameRecorder type:

    // 帧录制器
    protected FrameRecorder recorder;

• Then there is the initialization operation, please pay attention to the parameter settings (in the case of a 1280*720 resolution camera):

    @Override
    protected void initOutput() throws Exception {
        // 实例化FFmpegFrameRecorder，将SRS的推送地址传入
        recorder = FrameRecorder.createDefault(RECORD_ADDRESS, getCameraImageWidth(), getCameraImageHeight());

        // 降低启动时的延时，参考
        // https://trac.ffmpeg.org/wiki/StreamingGuide)
        recorder.setVideoOption("tune", "zerolatency");
        // 在视频质量和编码速度之间选择适合自己的方案，包括这些选项：
        // ultrafast,superfast, veryfast, faster, fast, medium, slow, slower, veryslow
        // ultrafast offers us the least amount of compression (lower encoder
        // CPU) at the cost of a larger stream size
        // at the other end, veryslow provides the best compression (high
        // encoder CPU) while lowering the stream size
        // (see: https://trac.ffmpeg.org/wiki/Encode/H.264)
        // ultrafast对CPU消耗最低
        recorder.setVideoOption("preset", "ultrafast");
        // Constant Rate Factor (see: https://trac.ffmpeg.org/wiki/Encode/H.264)
        recorder.setVideoOption("crf", "28");
        // 2000 kb/s, reasonable "sane" area for 720
        recorder.setVideoBitrate(2000000);

        // 设置编码格式
        recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);

        // 设置封装格式
        recorder.setFormat("flv");

        // FPS (frames per second)
        // 一秒内的帧数
        recorder.setFrameRate(getFrameRate());
        // Key frame interval, in our case every 2 seconds -> 30 (fps) * 2 = 60
        // 关键帧间隔
        recorder.setGopSize((int)getFrameRate()*2);

        // 帧录制器开始初始化
        recorder.start();
    }

• Next is the output method, the key is recorder.record, and pay attention to the calculation and setting of the timestamp:

    @Override
    protected void output(Frame frame) throws Exception {
        if (0L==startRecordTime) {
            startRecordTime = System.currentTimeMillis();
        }

        // 时间戳
        recorder.setTimestamp(1000 * (System.currentTimeMillis()-startRecordTime));

        // 存盘
        recorder.record(frame);
    }

• The last thing to do before the program exits after the loop that processes the video is over is to close the frame grabber:

    @Override
    protected void releaseOutputResource() throws Exception {
        recorder.close();
    }

• In addition, pay attention to the delay between two frames. Since the push flow involves the network, it cannot be calculated strictly according to the frame rate like the local preview, and the actual interval is smaller:

    @Override
    protected int getInterval() {
        // 相比本地预览，推流时两帧间隔时间更短
        return super.getInterval()/4;
    }

• So far, the push function has been developed, and then write the main method. Note that the parameter <font color="blue">600</font> means that the capture and recording operations are executed for 600 seconds:

    public static void main(String[] args) {
        new RecordCamera().action(600);
    }

• Run the main method, and wait until the console outputs the content of the red box in the figure below, which means that the streaming has started:

• Use VLC software to open the address <font color="blue">rtmp://192.168.50.43:21935/hls/camera</font> that you just pushed with the local computer or another computer in the local area network, wait a few seconds Normal playback starts after a few minutes:

• You can also use VLC's tools to view encoding information:

• So far, we have completed the streaming function and verified that the remote playback is also normal. Thanks to the power of JavaCV, the whole process is so relaxing and pleasant. Next, please continue to pay attention to Xinchen Original, the "JavaCV Camera Actual" series will also be presented. More rich applications;
• At this moment, you must have discovered the problem: do you only push videos? Not even a sound, that's all? That's right, in the next actual combat, let's challenge audio processing

Source code download

• The complete source code of "Camera Combat of JavaCV" can be downloaded from GitHub. The address and link information are shown in the following table ( https://github.com/zq2599/blog_demos ):

name	Link	Remark
Project homepage	https://github.com/zq2599/blog_demos	The project's homepage on GitHub
git repository address (https)	https://github.com/zq2599/blog_demos.git	The warehouse address of the source code of the project, https protocol
git repository address (ssh)	[e-mail]:zq2599/blog_demos.git	The warehouse address of the project source code, ssh protocol

• There are multiple folders in this git project. The source code of this article is in the <font color="blue">javacv-tutorials</font> folder, as shown in the red box below:

• There are multiple sub-projects in <font color="blue">javacv-tutorials</font>, and the code of the "JavaCV Camera Actual" series is in <font color="red"> simple-grab-push </font> Under the project:

You are not alone, Xinchen Original is with you all the way

https://github.com/zq2599/blog_demos