UE implements mouse click model

wedge

In a twinned scene, clicking on a 3D object is a common operation. For example, click on the model to display related properties and pictures, click on the camera model to play the video, click on the building to expand the floor and so on.

Therefore, clicking the model is the most necessary basic capability of the digital twin.

prepare knowledge

Introduction to UE Blueprints

This article will involve some knowledge of blueprints. If you do not know blueprints, you need to understand the knowledge of UE blueprints first. Blueprints are Unreal Engine 4's visual scripting approach. That is, tasks that would normally be done by scripting can now be created from a graph of nodes and connections without having to output any actual code.

For basic knowledge about blueprints, you can refer to the official documentation.
https://docs.unrealengine.com/4.27/en-US/ProgrammingAndScripting/Blueprints/
https://docs.unrealengine.com/4.27/en-US/Resources/Showcases/BlueprintExamples/
In the future, related articles will be written to introduce blueprint knowledge.

mathematical theory

The technical theory of UE mouse click model is roughly as follows:

1. Get the position and direction of the mouse click (involving the conversion of the mouse position to three-dimensional space coordinates, you can set the built-in method to obtain)
2. Fires a ray with position and direction.
3. It is judged that the ray intersects with those objects (which can be obtained by using the built-in method), and the most recent intersecting model is the 3D object obtained by clicking the mouse.

If you are familiar with threejs, you know that this is a bit similar to Threejs' raycaster.

Overload Pawn

Our Blueprint is implemented in an overloaded Pawn class, which is the base class for all Actors controlled by the player or AI. A Pawn is the embodiment of the player or AI entity in the game scene. This means that Pawns determine not only how players or AI entities look, but also how they collide with the scene and interact with other physics. Some games may not have player models or avatars visible in the game, so this can be confusing in some cases. However, Pawns still represent the physical orientation, rotation, etc. of the player or entity in the game. A Character is a special kind of Pawn that can walk.

More about Pawn, Pawn

Right-click in the content browser, create a new blueprint -> blueprint class:

Select Pawn:

Then enter the name in the browser:

Double-click the newly created blueprint class to enter the blueprint editing page.

The basic operations of blueprints, such as adding nodes, moving nodes, wiring, etc., are not described in detail here.

monitor mouse

Listen for mouse events in the blueprint (this article is the right button) as follows:

Pressed means pressed, Released means released.

Get mouse position and direction

Blueprint node to get mouse position "Translate mouse position to scene space"

The target is the player controller, and the player controller is obtained through the following node:

Construct rays

The node "Convert mouse position to scene space" can get the world coordinates and forward direction of the mouse, which are set as origin and direction respectively. The origin is not the origin of the ray, and the end point of the ray can be obtained by vector calculation:

 end = origin + directon * length

Where length is a constant, we can specify, so the blueprint for calculating end is as follows:

This involves a Blueprint node that multiplies a constant by a vector, and a node that adds two vectors. First, multiply the world direction by a constant 10000, and add the calculated result to the world location to get the end point.

The start and end points are ultimately used as input values for the next calculation.

Obtain inspection results by radiography

The node to get click result by ray is "Detect Line by Channel"

in:

• start indicates the starting point of the ray
• end indicates the focus of the ray
• Out Hit represents the detected object
• Return Value is a bool, true means there is an object hit, false means no object hit.

Interrupt hit result

The meaning of the so-called interrupt hit result can be understood to divide the hit packaging result into items.
First, judge by the return value of the detection result, if the hit is hit, the hit result will be interrupted, and the conditional judgment will be carried out through the branch node:

• Condition represents the condition of the input,
• True indicates the execution when the condition is true
• False indicates the execution when the condition is false

In this example, when the condition is true, execute the interrupt result:

Get result information

Among the hit results above:

• Hit Actor represents the actor that was hit
• Hit Component The component in the actor that was hit, if there is a subcomponent mesh that can be detected by the ray

After obtaining the relevant information, you can perform relevant operations. The relevant information printed here is as follows:

Show mouse cursor

After running the program by default, the mouse cursor is not displayed. In order to be able to see the click point, the mouse cursor needs to be displayed. For example, press the tap key to display the cursor, as follows:

Set up Pawns

After rewriting the Pawn class, in the program settings, you need to change the Pawn of the model to the Pawn class we rewritten to take effect, as shown in the following figure:

Epilogue

This paper illustrates the function of detecting mouse click models by means of rays. The final effect is shown in the following figure:

When clicked, print the name of the corresponding component.

If you have a good experience, you are also welcome to communicate with me. Follow the official account "ITMan Biao Shu", you can add the author's WeChat to communicate, and receive more valuable articles in time.