How to implement video calling in an application based on Agora Android SDK?

In many products, real-time video calling is not a new feature, such as video conferencing, social applications, online education, and may even appear in some metaverse scenarios.

This article will teach you how to implement a video calling application on the Android side through the Agora Video SDK. Shengwang SDK will provide 10,000 minutes of free usage every month, which can realize various real-time audio and video scenarios. Without further ado, let's get started.

Experience video calling through open source Demo

There may be some people who still don't know what the function we want to achieve will be in the end. So we provide an open source basic video call sample project on GitHub, you can experience the audio and video call effect through this sample project before starting development.

Github：https://github.com/AgoraIO/API-Examples/blob/master/Android/APIExample/app/src/main/java/io/agora/api/example/examples/basic/JoinChannelVideo.java

The technical principle of video calling

What we're trying to achieve here is a one-on-one video call. You can understand it as the audio and video communication between two users by joining the same channel. The data of this channel will be transmitted with low latency through the Agora SD-RTN real-time network of SoundNet.

After the App integrates the Agora SDK, the basic workflow of a video call is shown in the following figure:

As shown in the figure, the steps to implement a video call are as follows:

1. Get Token: When the app client joins the channel, you need to use the Token to authenticate the user. In a test or production environment, get the token from the app server.
2. Join a channel: call joinChannel to create and join a channel. App clients using the same channel name join the same channel by default. A channel can be understood as a channel dedicated to transmitting real-time audio and video data.
3. Publish and subscribe audio and video streams in a channel: After joining a channel, the app client can publish and subscribe audio and video streams in the channel.

App clients need the following information to join a channel:

• App ID: A string randomly generated by Agora to identify your app. It can be obtained from the Agora console .
• User ID: The unique identification of the user. You need to set the user ID yourself and make sure it is unique within the channel.
• Token: In a test or production environment, the app client obtains a token from your server. In the process described in this article, you can obtain a temporary token from the Agora console. Temporary tokens are valid for 24 hours.
• Channel Name: A string that identifies the video call channel.

development environment

The Agora SDK has good compatibility and does not have high requirements on hardware devices and software systems. The development environment and test environment can meet the following conditions:

• Android SDK API Level >= 16
• Android Studio 2.0 or above
• Real phone with voice and video capabilities
• App requires Android 4.1 or above device

The following are the development and test environments for this article:

Development Environment

• Windows 10 Home Chinese Edition
• Java Version SE 8
• Android Studio 3.2 Canary 4

test environment

• Samsung Nexus (Android 4.4.2 API 19)
• Mi Note 3 (Android 7.1.1 API 25)

If you have not been exposed to the Agora SDK before, then you need to do the following preparations:

• Register a account, enter the background to create an AppID, and obtain a Token. For the detailed method, please refer to this tutorial ;

https://sso2.agora.io/cn/signup?

• Download the official latest video SDK

Project settings

1. Before implementing the video call, please refer to the following steps to set up your project:

To create a new project, in Android Studio , select Phone and Tablet > Empty Activity , and create Android 2 project .

After creating the project, Android Studio will automatically start syncing gradle. Please make sure the synchronization is successful before proceeding to the next step.

2. Integrate the SDK, this article recommends using gradle to integrate the Agora SDK:

a. Add the following code to the /Gradle Scripts/build.gradle(Project: ) file to add mavenCentral dependencies:

buildscript {
     repositories {
         ...
         mavenCentral()
     }
     ...
}
 
  allprojects {
     repositories {
         ...
         mavenCentral()
     }
}

b. Add the following code to the /Gradle Scripts/build.gradle(Module: .app) file to integrate the Agora Video SDK into your Android project:

...
dependencies {
 ...
 // x.y.z，请填写具体的 SDK 版本号，如：3.5.0。
 // 通过发版说明获取最新版本号。
 implementation 'io.agora.rtc:full-sdk:x.y.z'
}

3. Permission settings, add the following network and device permissions after `` in the /app/Manifests/AndroidManifest.xml file:

<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.BLUETOOTH" />

Client implementation

This section introduces a few tips on how to use the Agora Video SDK to implement video calling in your app:

1. Obtain the necessary permissions

When launching the app, check that the required permissions for video calling have been granted in the app.

Call the following code in the onCreate function:

private static final int PERMISSION_REQ_ID = 22;
 
private static final String[] REQUESTED_PERMISSIONS = {
     Manifest.permission.RECORD_AUDIO,
     Manifest.permission.CAMERA
};
 
private boolean checkSelfPermission(String permission, int requestCode) {
    if (ContextCompat.checkSelfPermission(this, permission) !=
            PackageManager.PERMISSION_GRANTED) {
        ActivityCompat.requestPermissions(this, REQUESTED_PERMISSIONS, requestCode);
        return false;
    }
    return true;
}

2. Implement video call logic

The following figure shows the API call sequence for video calls:

a. Initialize the engine

The RtcEngine class contains the main methods called by the application. It is better to call the interface of RtcEngine in the same thread, and it is not recommended to call it in different threads at the same time.

Currently, Agora Native SDK only supports one RtcEngine instance, and each application only creates one RtcEngine object. All interface functions of the RtcEngine class, unless otherwise specified, are called asynchronously. It is recommended to call the interface in the same thread. For all APIs whose return value is int, unless otherwise specified, the return value 0 means the call succeeds, and the return value less than 0 means the call fails.

The IRtcEngineEventHandler interface class is used by the SDK to send callback event notification to the application, and the application obtains the event notification of the SDK by inheriting the method of this interface class.

All methods of the interface class have default (empty) implementations, and applications can inherit only the events they care about as needed. In the callback method, the application should not do time-consuming or call APIs (such as SendMessage) that may cause blocking, otherwise it may affect the running of the SDK.

engine = RtcEngine.create(context.getApplicationContext(), getString(R.string.agora_app_id), iRtcEngineEventHandler);
...
private final IRtcEngineEventHandler iRtcEngineEventHandler = new IRtcEngineEventHandler()
{
    /**Reports a warning during SDK runtime.
     * Warning code: https://docs.agora.io/en/Voice/API%20Reference/java/classio_1_1agora_1_1rtc_1_1_i_rtc_engine_event_handler_1_1_warn_code.html*/
    @Override
    public void onWarning(int warn)
    {
        Log.w(TAG, String.format("onWarning code %d message %s", warn, RtcEngine.getErrorDescription(warn)));
    }
 
    /**Reports an error during SDK runtime.
     * Error code: https://docs.agora.io/en/Voice/API%20Reference/java/classio_1_1agora_1_1rtc_1_1_i_rtc_engine_event_handler_1_1_error_code.html*/
    @Override
    public void onError(int err)
    {
        Log.e(TAG, String.format("onError code %d message %s", err, RtcEngine.getErrorDescription(err)));
        showAlert(String.format("onError code %d message %s", err, RtcEngine.getErrorDescription(err)));
    }
 
    /**Occurs when a user leaves the channel.
     * @param stats With this callback, the application retrieves the channel information,
     *              such as the call duration and statistics.*/
    @Override
    public void onLeaveChannel(RtcStats stats)
    {
        super.onLeaveChannel(stats);
        Log.i(TAG, String.format("local user %d leaveChannel!", myUid));
        showLongToast(String.format("local user %d leaveChannel!", myUid));
    }
 
    /**Occurs when the local user joins a specified channel.
     * The channel name assignment is based on channelName specified in the joinChannel method.
     * If the uid is not specified when joinChannel is called, the server automatically assigns a uid.
     * @param channel Channel name
     * @param uid User ID
     * @param elapsed Time elapsed (ms) from the user calling joinChannel until this callback is triggered*/
    @Override
    public void onJoinChannelSuccess(String channel, int uid, int elapsed)
    {
        Log.i(TAG, String.format("onJoinChannelSuccess channel %s uid %d", channel, uid));
        showLongToast(String.format("onJoinChannelSuccess channel %s uid %d", channel, uid));
        myUid = uid;
        joined = true;
        handler.post(new Runnable()
        {
            @Override
            public void run()
            {
                join.setEnabled(true);
                join.setText(getString(R.string.leave));
            }
        });
    }
 
    @Override
    public void onRemoteAudioStats(io.agora.rtc.IRtcEngineEventHandler.RemoteAudioStats remoteAudioStats) {
        statisticsInfo.setRemoteAudioStats(remoteAudioStats);
        updateRemoteStats();
    }
 
    @Override
    public void onLocalAudioStats(io.agora.rtc.IRtcEngineEventHandler.LocalAudioStats localAudioStats) {
        statisticsInfo.setLocalAudioStats(localAudioStats);
        updateLocalStats();
    }
 
    @Override
    public void onRemoteVideoStats(io.agora.rtc.IRtcEngineEventHandler.RemoteVideoStats remoteVideoStats) {
        statisticsInfo.setRemoteVideoStats(remoteVideoStats);
        updateRemoteStats();
    }
 
    @Override
    public void onLocalVideoStats(io.agora.rtc.IRtcEngineEventHandler.LocalVideoStats localVideoStats) {
        statisticsInfo.setLocalVideoStats(localVideoStats);
        updateLocalStats();
    }
 
    @Override
    public void onRtcStats(io.agora.rtc.IRtcEngineEventHandler.RtcStats rtcStats) {
        statisticsInfo.setRtcStats(rtcStats);
    }
};

b. Set local video parameters

The enableVideo() method is used to enable video mode. It can be called before joining a channel or during a call. If called before joining a channel, the video mode will be automatically turned on, and if it is called during a call, the audio mode will be switched to the video mode. Call the disableVideo() method to turn off video mode.

The setupLocalVideo( VideoCanvas local ) method is used to set the local video display information. The application binds the display window (view) of the local video stream by calling this interface, and sets the video display mode. In application development, this method is usually called after initialization for local video settings before joining the channel. After exiting the channel, the binding is still valid, if you need to unbind, you can call setupLocalVideo(null).

// Create render view by RtcEngine
SurfaceView surfaceView = RtcEngine.CreateRendererView(context);
if(fl_local.getChildCount() > 0)
{
    fl_local.removeAllViews();
}
// Add to the local container
fl_local.addView(surfaceView, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));
// Setup local video to render your local camera preview
engine.setupLocalVideo(new VideoCanvas(surfaceView, RENDER_MODE_HIDDEN, 0));
// Enable video module
engine.enableVideo();

c. Join a channel

The joinChannel() method allows users to join the call channel. Users in the same channel can talk to each other, and multiple users can join the same channel and chat in groups. Apps using different App IDs cannot communicate with each other. If already on a call, the user must call leaveChannel() to exit the current call to enter the next channel.

ChannelMediaOptions option = new ChannelMediaOptions();
option.autoSubscribeAudio = true;
option.autoSubscribeVideo = true;
int res = engine.joinChannel(accessToken, channelId, "Extra Optional Data", 0, option);
if (res != 0)
{
    // Usually happens with invalid parameters
    // Error code description can be found at:
    // en: https://docs.agora.io/en/Voice/API%20Reference/java/classio_1_1agora_1_1rtc_1_1_i_rtc_engine_event_handler_1_1_error_code.html
    // cn: https://docs.agora.io/cn/Voice/API%20Reference/java/classio_1_1agora_1_1rtc_1_1_i_rtc_engine_event_handler_1_1_error_code.html
    showAlert(RtcEngine.getErrorDescription(Math.abs(res)));
    return;
}

d. Leave the current channel

The leaveChannel() method is used to leave the channel, i.e. hang up or exit the call.

After calling the joinChannel() API method, you must call leaveChannel() to end the call, otherwise the next call cannot be started. leaveChannel() can be called regardless of whether it is currently on a call or not, with no side effects. This method will release all resources related to the session. The method is an asynchronous operation and the call returns without actually exiting the channel. After actually exiting the channel, the SDK will trigger the onLeaveChannel callback.

e. Manage cameras

The switchCamera() method is used to switch between front/rear cameras. In addition, Agora also provides a way to manage cameras: for example, setCameraTorchOn(boolean isOn) sets whether to turn on the flash, setCameraAutoFocusFaceModeEnabled(boolean enabled) sets whether to turn on the face focus function, and so on.

f. When the remote user joins the channel, update the remote user interface.

private final IRtcEngineEventHandler iRtcEngineEventHandler = new IRtcEngineEventHandler()
{
    ...
    /**Occurs when a remote user (Communication)/host (Live Broadcast) joins the channel.
     * @param uid ID of the user whose audio state changes.
     * @param elapsed Time delay (ms) from the local user calling joinChannel/setClientRole
     *                until this callback is triggered.*/
    @Override
    public void onUserJoined(int uid, int elapsed)
    {
        super.onUserJoined(uid, elapsed);
        Log.i(TAG, "onUserJoined->" + uid);
        showLongToast(String.format("user %d joined!", uid));
        /**Check if the context is correct*/
        Context context = getContext();
        if (context == null) {
            return;
        }
        if(remoteViews.containsKey(uid)){
            return;
        }
        else{
            handler.post(() ->
            {
                /**Display remote video stream*/
                SurfaceView surfaceView = null;
                // Create render view by RtcEngine
                surfaceView = RtcEngine.CreateRendererView(context);
                surfaceView.setZOrderMediaOverlay(true);
                ViewGroup view = getAvailableView();
                remoteViews.put(uid, view);
                // Add to the remote container
                view.addView(surfaceView, new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));
                // Setup remote video to render
                engine.setupRemoteVideo(new VideoCanvas(surfaceView, RENDER_MODE_HIDDEN, uid));
            });
        }
    }
 
    /**Occurs when a remote user (Communication)/host (Live Broadcast) leaves the channel.
     * @param uid ID of the user whose audio state changes.
     * @param reason Reason why the user goes offline:
     *   USER_OFFLINE_QUIT(0): The user left the current channel.
     *   USER_OFFLINE_DROPPED(1): The SDK timed out and the user dropped offline because no data
     *              packet was received within a certain period of time. If a user quits the
     *               call and the message is not passed to the SDK (due to an unreliable channel),
     *               the SDK assumes the user dropped offline.
     *   USER_OFFLINE_BECOME_AUDIENCE(2): (Live broadcast only.) The client role switched from
     *               the host to the audience.*/
    @Override
    public void onUserOffline(int uid, int reason)
    {
        Log.i(TAG, String.format("user %d offline! reason:%d", uid, reason));
        showLongToast(String.format("user %d offline! reason:%d", uid, reason));
        handler.post(new Runnable() {
            @Override
            public void run() {
                /**Clear render view
                 Note: The video will stay at its last frame, to completely remove it you will need to
                 remove the SurfaceView from its parent*/
                engine.setupRemoteVideo(new VideoCanvas(null, RENDER_MODE_HIDDEN, uid));
                remoteViews.get(uid).removeAllViews();
                remoteViews.remove(uid);
            }
        });
    }
    ...
};

done, run

Take two mobile phones and install the compiled app. If you can see two of yourselves, you have succeeded. If you encounter problems during the development process, you can visit the forum to ask questions and communicate with sound network engineers

https://rtcdeveloper.agora.io/

You can also visit the background for further technical support

https://console.agora.io/