Develop a distributed IM (Instant Messaging) system!

Author: Xiao Fu Ge
Blog: https://bugstack.cn

Precipitate, share, and grow, so that you and others can gain something! 😄

I. Introduction

I learned this knowledge, but I didn’t forget it quickly? !

Does it feel like a lot of information, is cool to collect, when watching the video, read the article at that time, as long as you pass that energy, it's over, I don't remember what was said in it. Time is wasted and things have not been learned yet, why is this?

In fact, because learning is also divided into upper, middle and lower strategies:

• The next step: just look at it, sitting, nesting, lying down, all do, anyway, you don’t get tired, and you can reply to the WeChat group that blows water at the same time.
• Zhongce: After reading the materials, make notes, organize and summarize, and accumulate materials for a long time.
• The best strategy: practice, get started, apply, debug, summarize, organize information, summarize experience, output documents

To sum up, I feel that I have learned a lot quickly when I learn about the next strategy. The middle strategy is a bit too lazy to move, and the superior strategy is very time-consuming and laborious. I must be able to do everything personally to every knowledge point. So when you learn unconsciously we chose worst , and therefore really did not learn anything.

Learning can learn knowledge by hand, and what you pay attention to is practice. In the articles written by Xiao Fu, basically the results of practical code verification are the core, and the content of the article is described. 😁Since I was young, I like to use . Take an instant messaging project as an example. It has been implemented 5 or 6 times based on different technical solutions. It is only practice technology . The screenshot is as follows:

• Some are just after learning Socket and Swing, and want to try these technologies if they can write a QQ.
• There are also projects that need to be completed because of internship training, but after having some foundation, you can write all the functions in a week.
• Although these items still look ugly and ugly, and the code logic may not be so perfect. But in every realization of the learning phase, it can bring a lot of technical growth for itself.

---

So, this opportunity of IM practice is for you, I hope you can use it! Next, I will introduce to you an IM system architecture, communication protocol, single chat group chat, emoticon sending, UI event-driven and other content, and provide a full set of source code so that you can learn.

2. Demo

Before starting to learn, let’s demonstrate the operation effect imitating the PC-side WeChat interface.

chat page

add friends

Video demo

https://www.bilibili.com/video/BV1BZ4y1W7fC

Three, system design

In this set of IM , the server adopts the DDD domain-driven design pattern to build. SpringBoot over the Netty function to 06125c04b8720c for start-stop control, and at the same time build a console on the server side to conveniently operate the communication system and manage users and communication. In the construction of the client, the UI separation method is used to build to ensure that the business code is UI display, and the control is very easy to expand.

In addition, the functional implementation includes; perfectly imitating the core functions of WeChat desktop version client, login, search and add friends, user communication, group communication, and emoticon sending. If there are functions that actually need to be used, they can be expanded in accordance with this system framework.

• UI development : Use JavaFx and Maven build a UI desktop project, and explain step by step various UI display and operation events such as login box, chat box, dialog box, and friend bar. So in this chapter, let Java programmers learn to develop desktop applications.
• Architecture Design : In this chapter, we will use the four-layer model structure of DDD domain-driven design in combination with Netty to construct a reasonable layered framework. At the same time, there is a design of the corresponding library table function. I believe that after studying these contents, you can also assume a better framework.
• function realization : In this part, we mainly implement various functions in communication, including; login, add friends, dialogue notification, message sending, disconnection reconnection and other functions. Finally complete the development of the entire project, but also allows you to learn skills from practice.

Four, UI development

1. Overall structure definition, sidebar

The chat form, compared to the login form, has more content and is relatively complicated. Therefore, we will implement these forms, events and interface functions step by step in chapters. In this article, we will mainly explain the construction of the chat box and the development of the sidebar UI.

• The first is the definition of our entire chat main form, which is a blank panel and removes the default border buttons (minimize, exit, etc.)
• After that is our left sidebar, which we call the bar Bar, the realization of the functional area.
• Finally, add a form event to change the fill information in the content panel when the button is clicked.

2. Dialogue chat box

The content area after the dialog box is selected is displayed, that is, information is sent and displayed between users. On the whole, this is a linked process. Click on the dialog user on the left, and the corresponding content will be filled on the right. Then the ListView of the filled dialogue list on the right side needs to be associated with each dialogue user. When the chat user is clicked, it is a process of repeatedly switching and filling.

• Click each dialog box on the left, and the filling content of the chat box on the right will change accordingly. At the same time, the corresponding dialogue name will also change.
• The left side of the dialog box displays the information sent by friends, and the right side displays the information sent by individuals. At the same time, the message content will increase in height and width as the content increases.
• At the bottom is the text input box. In the following implementation, our text input box is designed in a public way. Of course, you can also design it for individual personal use.

3. Friends bar

Everyone often uses WeChat on the PC side. You can know that there are several paragraphs of content in the Friends column, including: new friends, official accounts, groups, and bottom friends.

• The content of the top search box remains unchanged, the same as the previous one. The method we currently use is fxml design. For example, this part is a common function, which can be extracted and put into the code to be designed as a component element class.
• After our analysis, based on the use of JavaFx component development, this part is a nested ListView, that is, the bottom panel is a ListView, and friends and groups are each a ListView, so we will be very convenient after processing For data filling.
• In addition, this structure is mainly conducive to the operation of our program, if you add a friend, then we need to refresh the friend information to the friend column, and when the data is filled, in order to be more convenient and efficient, so we designed the nesting ListView. If you still don't understand it, you can get the answer from the code that follows.

4. Event Definition

In the desktop UI development, in order to isolate the UI from the business logic, we need to provide an interface for the display effect of the operation interface and an abstract class for interface operation events after we package the UI. Then you can understand it according to the following figure;

Serial number	Interface name	describe
1	void doShow()	Open window
2	void setUserInfo(String userId, String userNickName, String userHead)	Set login user ID, nickname, avatar
3	void addTalkBox(int talkIdx, Integer talkType, String talkId, String talkName, String talkHead, String talkSketch, Date talkDate, Boolean selected)	Populate the dialog list
4	void addTalkMsgUserLeft(String talkId, String msg, Date msgData, Boolean idxFirst, Boolean selected, Boolean isRemind)	Fill in the dialog message-friends (other people's messages)

• The above interfaces are all the behavioral interfaces provided by our current UI to the outside world. A link description of these interfaces is: open windows, search for friends, add friends, open dialog boxes, and send messages.

Five, communication design

1. System Architecture

Earlier, we talked about a more suitable architecture, which is the best architecture that meets your current needs. So how to design such an architecture is basically to find a target that meets the point. The reason why we designed this way, then there are the following points in this system;

• Our system must have a web page on the server side to manage communication users and control and monitor the server side.
• The object class of the database should not be contaminated by the outside, but should be isolated. For example; your database class is exposed for external use as a display class, so now you need to add a field, and this field is not an attribute of your database. Then this time has already polluted the database class.
• Because we are currently implementing Netty communication in the Java language, both the server and the client need to use the protocol definition and analysis in the communication process. Then we need to extract the Jar package from this layer.
• Interfaces, business processing, underlying services, and communication interactions must be clearly distinguished and implemented to avoid confusion and difficulty in maintenance.

Combining our four goals above, what model structure do you have in your mind? And is there a plan for the corresponding technology stack selection? Next, we will introduce two architectural design models, one is MVC DDD domain-driven design that you may have heard of.

2. Communication protocol

Judging from the artwork, we need to add a frame identifier to the transmission package when transmitting objects to determine which object the current business object is, which can make our business clearer and avoid using a large number of if Sentence judgment.

Protocol framework

agreement
└── src
    ├── main
    │   ├── java
    │   │   └── org.itstack.naive.chat
    │   │       ├── codec
    │   │       │    ├── ObjDecoder.java
    │   │       │    └── ObjEncoder.java
    │   │       ├── protocol
    │   │       │    ├── demo
    │   │       │    ├── Command.java
    │   │       │    └── Packet.java
    │   │       └── util
    │   │             └── SerializationUtil.java
    │   ├── resources    
    │   │   └── application.yml
    │   └── webapp
    │       └── chat
    │       └── res
    │       └── index.html
    └── test
         └── java
             └── org.itstack.demo.test
                 └── ApiTest.java

protocol package

public abstract class Packet {

    private final static Map<Byte, Class<? extends Packet>> packetType = new ConcurrentHashMap<>();

    static {
        packetType.put(Command.LoginRequest, LoginRequest.class);
        packetType.put(Command.LoginResponse, LoginResponse.class);
        packetType.put(Command.MsgRequest, MsgRequest.class);
        packetType.put(Command.MsgResponse, MsgResponse.class);
        packetType.put(Command.TalkNoticeRequest, TalkNoticeRequest.class);
        packetType.put(Command.TalkNoticeResponse, TalkNoticeResponse.class);
        packetType.put(Command.SearchFriendRequest, SearchFriendRequest.class);
        packetType.put(Command.SearchFriendResponse, SearchFriendResponse.class);
        packetType.put(Command.AddFriendRequest, AddFriendRequest.class);
        packetType.put(Command.AddFriendResponse, AddFriendResponse.class);
        packetType.put(Command.DelTalkRequest, DelTalkRequest.class);
        packetType.put(Command.MsgGroupRequest, MsgGroupRequest.class);
        packetType.put(Command.MsgGroupResponse, MsgGroupResponse.class);
        packetType.put(Command.ReconnectRequest, ReconnectRequest.class);
    }

    public static Class<? extends Packet> get(Byte command) {
        return packetType.get(command);
    }

    /**
     * 获取协议指令
     *
     * @return 返回指令值
     */
    public abstract Byte getCommand();

}

3. Add friends

• As you can see from the above process, there are two parts here; (1) search for friends, (2) add friends. After finishing friends on the same day, the friends will appear in our friends column.
• And here we use unilateral consent to add friends, that is, when you add a friend, the other party also has your friend information.
• If you need to add friends and agree to it in your business, you can add a piece of status information when you initiate the addition of friends, requesting to add friends. After the other party agrees, the two users can become friends and communicate.

add friends, case code

public class AddFriendHandler extends MyBizHandler<AddFriendRequest> {

    public AddFriendHandler(UserService userService) {
        super(userService);
    }

    @Override
    public void channelRead(Channel channel, AddFriendRequest msg) {
        // 1. 添加好友到数据库中[A->B B->A]
        List<UserFriend> userFriendList = new ArrayList<>();
        userFriendList.add(new UserFriend(msg.getUserId(), msg.getFriendId()));
        userFriendList.add(new UserFriend(msg.getFriendId(), msg.getUserId()));
        userService.addUserFriend(userFriendList);
        // 2. 推送好友添加完成 A
        UserInfo userInfo = userService.queryUserInfo(msg.getFriendId());
        channel.writeAndFlush(new AddFriendResponse(userInfo.getUserId(), userInfo.getUserNickName(), userInfo.getUserHead()));
        // 3. 推送好友添加完成 B
        Channel friendChannel = SocketChannelUtil.getChannel(msg.getFriendId());
        if (null == friendChannel) return;
        UserInfo friendInfo = userService.queryUserInfo(msg.getUserId());
        friendChannel.writeAndFlush(new AddFriendResponse(friendInfo.getUserId(), friendInfo.getUserNickName(), friendInfo.getUserHead()));
    }

}

4. Message response

• It can be seen from the overall process that when the user initiates a friend or group communication, an event behavior is triggered, and then the client sends a dialogue request with the friend to the server.
• After the server receives the conversation request, if it is a friend conversation, it needs to save the communication information with the friend in the dialog box. At the same time, inform friends that I am going to communicate with you. You are in your dialog list, add me.
• So if it’s a group communication, you don’t need to do this notification, because it’s impossible to notify all group users who are not online yet (people haven’t logged in yet), so this part only needs to be created after the user goes online and receives the information. Just exit the dialog box to the list. You can understand it carefully, and you can also think about other ways to achieve it.

message response, case code

public class MsgHandler extends MyBizHandler<MsgRequest> {

    public MsgHandler(UserService userService) {
        super(userService);
    }

    @Override
    public void channelRead(Channel channel, MsgRequest msg) {
        logger.info("消息信息处理：{}", JSON.toJSONString(msg));
        // 异步写库
        userService.asyncAppendChatRecord(new ChatRecordInfo(msg.getUserId(), msg.getFriendId(), msg.getMsgText(), msg.getMsgType(), msg.getMsgDate()));
        // 添加对话框[如果对方没有你的对话框则添加]
        userService.addTalkBoxInfo(msg.getFriendId(), msg.getUserId(), Constants.TalkType.Friend.getCode());
        // 获取好友通信管道
        Channel friendChannel = SocketChannelUtil.getChannel(msg.getFriendId());
        if (null == friendChannel) {
            logger.info("用户id：{}未登录！", msg.getFriendId());
            return;
        }
        // 发送消息
        friendChannel.writeAndFlush(new MsgResponse(msg.getUserId(), msg.getMsgText(), msg.getMsgType(), msg.getMsgDate()));
    }

}

5. Reconnect after disconnection

• From the above process, we can see that when the network connection is disconnected, it will send a reconnection request like the server.
Then the process of initiating the link is different from the initial link of the system. Reconnection after disconnection requires the user's ID information to be sent to the server, so that the server can update the binding relationship between the user and the communication channel.
• At the same time, it is necessary to update the reconnection information in the group, and add the user's reconnection to the group mapping. At this point, the communication function between the user and friends and groups can be restored.

message response, case code

// Channel 状态定时巡检；3 秒后每 5 秒执行一次
scheduledExecutorService.scheduleAtFixedRate(() -> {while (!nettyClient.isActive()) {System.out.println("通信管道巡检：通信管道状态" + nettyClient.isActive());
        try {System.out.println("通信管道巡检：断线重连 [Begin]");
            Channel freshChannel = executorService.submit(nettyClient).get();
            if (null == CacheUtil.userId) continue;
            freshChannel.writeAndFlush(new ReconnectRequest(CacheUtil.userId));
        } catch (InterruptedException | ExecutionException e) {System.out.println("通信管道巡检：断线重连 [Error]");}
    }
}, 3, 5, TimeUnit.SECONDS);

6. Cluster communication

• Cross-service cases use redis publish and subscribe to deliver messages. If you are a large service, you can use zookeeper
• When user A sends a message to user B, he needs to pass B’s channeId to the server to find out whether the channeId belongs to its own service
• A single machine can also start multiple Netty services, and the program will automatically find available ports

Six, source code download

This project is an imitation desktop version of WeChat built by the author, Xiao Fu, using JavaFx, Netty4.x, SpringBoot, Mysql and other technology stacks and a DDD field-driven design method to achieve core communication functions.

This set of IM code is divided into three groups of modules; UI, client, and server. The reason for this split is to isolate the UI display from the business logic and use events and interfaces to drive it to make the code level cleaner and easier to extend and maintain.

Serial number	project	introduce
1	itstack-naive-chat-ui	The UI terminal developed using JavaFx is provided in our UI terminal; login box, chat box, and a large number of behavior interaction interfaces, interfaces and events in the chat box. Finally, my UI side uses the Maven packaging method to provide the Jar package to the outside to achieve the separation of the UI interface and the business behavior process.
2	itstack-naive-chat-client	The client is our communication core project, mainly using Netty4.x as our socket framework to complete the communication interaction. And in this project is responsible for introducing the Jar package of the UI, completing the events defined by the UI (login verification, searching and adding friends, dialog notification, sending information, etc.), and the need to use the communication protocol defined in the server project to complete the information Interactive operation.
3	itstack-navie-chat-server	The server also uses Netty4.x as the socket communication framework, and at the same time Layui is used as the management background page on the server, and our server adopts a DDD domain-driven design approach to integrate with Netty to achieve our framework The structure is clean and easy to expand.
4	itstack.sql	System engineering database table structure and initialization data information, a total of 6 core tables; user table, group table, user group association table, friend table, conversation table and chat record table. Users can expand and improve by themselves in actual business development. At present, the library table structure is only based on core functions.

• source code to get : https://github.com/fuzhengwei/NaiveChat , give me a star from the source code, don’t get a white jacket! ! !

Seven, summary

• This IM system involves a lot of technology stacks, such as the use of technology stacks such as Netty4.x, SpringBoot, Mybatis, Mysql, JavaFx, layui, and the entire system frame structure using DDD four-layer architecture + Socket module to build, all The UI of the previous backend was designed in an event-driven way. As long as you can keep learning in this process, you will definitely gain a lot of content. enough to brag! 🌶
• The learning process of any new technology stack will include such a route; running HelloWorld, proficient use of API, project practice, and finally deep source code mining. So when hearing such a demand, Java programmers will definitely think of a series of technical knowledge points to fill in the various modules in our project, for example; the interface uses JavaFx, Swing, etc., the communication uses Socket or knows the Netty framework, server-side control Use MVC model plus SpringBoot and so on. But how to build our system rationally for these various technology stacks is indeed the most important part of the learning, practice, and growth process.