[Raspberry Pi] aarch64 system compiles the latest ffmpeg and supports hard decoding h264

Raspberry Pi aarch64

No need to hard-decode or output rtsp, just download the compiled amr64 to https://johnvansickle.com/ffmpeg/
The aarch64 hard solution uses h264_v4l2m2m, and the output of rtsp will cause a black screen because of the header in the encoding
The following compilation has fixed this bug

System version: openfans aarch64
Reference: https://gist.github.com/wildrun0/86a890585857a36c90110cee275c45fd

Installation dependencies

Conventional dependence

apt -y install \
    autoconf \
    automake \
    build-essential \
    cmake \
    doxygen \
    git \
    meson \
    nasm \
    pkg-config \
    python3-dev \
    python3-pip \
    texinfo \
    wget \
    yasm \
    zlib1g-dev

Compression/SSL/audio/video/picture library

apt -y install \
    graphviz \
    imagemagick \
    libaom-dev \
    libnuma-dev \
    liblzma-dev \
    libssl-dev \
    libdrm-dev \
    libdav1d-dev \
    libfdk-aac-dev \
    libasound2-dev \
    libass-dev \
    libavcodec-dev \
    libavdevice-dev \
    libavfilter-dev \
    libavformat-dev \
    libavutil-dev \
    libfreetype6-dev \
    libgmp-dev \
    libgme-dev \
    libmp3lame-dev \
    libopencore-amrnb-dev \
    libopencore-amrwb-dev \
    libopus-dev \
    libopenjp2-7-dev \
    librtmp-dev \
    libsdl2-dev \
    libsdl2-image-dev \
    libsdl2-mixer-dev \
    libsdl2-net-dev \
    libsdl2-ttf-dev \
    libspeex-dev \
    libsrt-dev \
    libsnappy-dev \
    libsoxr-dev \
    libssh-dev \
    libssl-dev \
    libtool \
    libtheora-dev \
    libv4l-dev \
    libva-dev \
    libvdpau-dev \
    libvo-amrwbenc-dev \
    libvorbis-dev \
    libwebp-dev \
    libx264-dev \
    libx265-dev \
    libxcb-shape0-dev \
    libxcb-shm0-dev \
    libxcb-xfixes0-dev \
    libxcb1-dev \
    libxvidcore-dev \
    libxml2-dev \
    libvidstab-dev \
    lzma-dev

libdav1d-dev ubuntu arm64 , you can compile AV1

HEVC for Apple

git clone --depth 1 https://github.com/ultravideo/kvazaar.git ~/ffmpeg-libraries/kvazaar \
  && cd ~/ffmpeg-libraries/kvazaar \
  && ./autogen.sh \
  && ./configure --prefix=/usr/ \
  && make -j$(nproc) \
  && sudo make install

VP8 and VP9

git clone --depth 1 https://github.com/webmproject/libvpx.git ~/ffmpeg-libraries/libvpx \
  && cd ~/ffmpeg-libraries/libvpx \
  && ./configure --prefix=/usr/ --disable-examples --disable-tools --disable-unit_tests --disable-docs \
  && make -j$(nproc) \
  && sudo make install

zimg

git clone --depth 1 https://github.com/sekrit-twc/zimg.git ~/ffmpeg-libraries/zimg \
  && cd ~/ffmpeg-libraries/zimg \
  && sh autogen.sh \
  && ./configure --prefix=/usr/ \
  && make -j$(nproc) \
  && sudo make install

AP1

No need to install
libaom-dev in the aarch system

git clone --depth 1 https://aomedia.googlesource.com/aom ~/ffmpeg-libraries/aom \
  && mkdir ~/ffmpeg-libraries/aom/aom_build \
  && cd ~/ffmpeg-libraries/aom/aom_build \
  && cmake -G "Unix Makefiles" AOM_SRC -DCMAKE_INSTALL_PREFIX=/usr/ -DENABLE_NASM=on -DPYTHON_EXECUTABLE="$(which python3)" -DCMAKE_C_FLAGS="-mfpu=vfp -mfloat-abi=hard" .. \
  && sed -i 's/ENABLE_NEON:BOOL=ON/ENABLE_NEON:BOOL=OFF/' CMakeCache.txt \
  && make -j$(nproc) \
  && sudo make install

AAC

No need to install
aarch64 system has libfdk-acc-dev

git clone --depth 1 https://github.com/mstorsjo/fdk-aac.git ~/ffmpeg-libraries/fdk-aac \
  && cd ~/ffmpeg-libraries/fdk-aac \
  && autoreconf -fiv \
  && ./configure --prefix=/usr/ \
  && make -j$(nproc) \
  && sudo make install

AV1

debian aarch64 system has libdav1d-dev
ubuntu aarch64 system does not have

git clone --depth 1 https://code.videolan.org/videolan/dav1d.git ~/ffmpeg-libraries/dav1d \
  && mkdir ~/ffmpeg-libraries/dav1d/build \
  && cd ~/ffmpeg-libraries/dav1d/build \
  && meson -Dprefix=/usr .. \
  && ninja \
  && sudo ninja install

Compile ffmpeg

Download source code

git clone --depth 1 https://github.com/FFmpeg/FFmpeg.git ~/FFmpeg \
  && cd ~/FFmpeg

Fix the bug that the output cannot be output due to multiple headers when reading the camera in v4l2

wget https://dl.dropboxusercontent.com/s/eja8vz5jh1felnp/patch.diff
git apply patch.diff

Modify the configuration file of ffmpeg

vim ./configure

This librtmp cannot be detected even if it is installed. Comment this line and use -lrtmp open it below.

# enabled librtmp           && require_pkg_config librtmp librtmp librtmp/rtmp.h RTMP_Socket

./configure \
    --prefix=/usr/ \
    --extra-cflags="-I/usr/local/include" \
    --extra-ldflags="-L/usr/local/lib" \
    --extra-libs="-lpthread -lm -latomic -lrtmp -lx265" \
    --arch=aarch64 \
    --disable-shared \
    --enable-neon \
    --enable-hwaccel=h264_vaapi \
    --enable-hwaccel=h264_vaapi \
    --enable-hwaccel=h264_dxva2 \
    --enable-hwaccel=mpeg4_vaapi \
    --enable-hwaccels \
    --enable-static \
    --disable-debug \
    --enable-gmp \
    --enable-gpl \
    --enable-gray \
    --enable-fontconfig \
    --enable-libaom \
    --enable-libass \
    --enable-libdav1d \
    --enable-libdrm \
    --enable-libgme \
    --enable-libfdk-aac \
    --enable-libfreetype \
    --enable-libkvazaar \
    --enable-libmp3lame \
    --enable-libopencore-amrnb \
    --enable-libopencore-amrwb \
    --enable-libopus \
    --enable-libopenjpeg \
    --enable-librtmp \
    --enable-libsnappy \
    --enable-libsoxr \
    --enable-libspeex \
    --enable-libssh \
    --enable-libsrt \
    --enable-libxvid \
    --enable-libtheora \
    --enable-libvidstab \
    --enable-libvorbis \
    --enable-libvpx \
    --enable-libzimg \
    --enable-libwebp \
    --enable-libx264 \
    --enable-libx265 \
    --enable-libxml2 \
    --enable-nonfree \
    --enable-omx \
    --enable-omx-rpi \
    --enable-version3 \
    --target-os=linux \
    --enable-pthreads \
    --enable-openssl \
    --enable-hardcoded-tables

Note: aarch64 does not support

--enable-mmal

If there is an error that omx (OpenMAX IL) does not exist
omx is a hardware decoder for Raspberry Pi
This is generally the result of compiling in a non-Raspberry Pi environment,

# 安装aarch64的GPU驱动
git clone https://github.com/raspberrypi/userland ~/userland
cd ~/userland
./buildme --aarch64

Compile

make -j$(nproc)
make install

Video streaming

rtsp server

Use EasyDarwin, download the release and run it on windows with one click

The Raspberry Pi has limited capabilities. If you use the software solution, the CPU occupies a lot, and you cannot capture the camera and use it as an rtsp server.

https://github.com/EasyDarwin/EasyDarwin

Push stream

200ms delay of soft solution, high CPU usage

ffmpeg  \
-f video4linux2 \
-framerate 24 \
-s 800x600 \
-i /dev/video0 \
-vcodec h264 \
-preset ultrafast \
-tune zerolatency  \
-g 6 \
-threads 4 \
-rtsp_transport tcp -f rtsp rtsp://rtsp.server/test

Key parameter explanation

-g 6 key frame 6
-tune zerolatency 0 delay
-preset veryfast x264 fast/faster/verfast/superfast/ultrafast

Hard decode 200ms delay

ffmpeg \
-f video4linux2  \
-framerate 24 \
-s 800x600 \
-i /dev/video0 \
-pix_fmt yuv420p \
-codec:v:0 h264_v4l2m2m \
-b:v 9990684 -maxrate 9990684 -bufsize 19981368 \
-preset ultrafast \
-tune zerolatency \
-g 6 -keyint_min 6 \
-force_key_frames:0 "expr:gte(t,0+n_forced*2)" \
-vf "scale=trunc(min(max(iw\,ih*dar)\,1920)/64)*64:trunc(ow/dar/2)*2" \
-threads 4 \
-vsync -1 \
-r 25 \
-rtsp_transport tcp -f rtsp rtsp://rtsp.server/test

[Raspberry Pi] aarch64 system compiles the latest ffmpeg and supports hard decoding h264