Post-processing-in-depth camera deformation effects

Post-processing is a technology for algorithmic processing of original game screens to improve screen quality or enhance screen effects. It can be implemented through the Shader program.

Overview

deformation special effects is a kind of post-processing technology for processing and enhancing picture effects. It is often used in various camera short video app special effects, such as beauty slimming and haha mirror special effects.

This article mainly sorts out the following commonly used deformation effects from various types of beauty cameras:

• Local distortion (twirl effect)
• Local expansion (inflate effect)
• Pinch effect in any direction

Among them, distortion can be used for partial rotation of the eyes, expansion can be used for big eyes, and extrusion/stretching can be used for facial plasticity and face thinning. How to achieve these deformations through the Shader is the focus of this article. (Ps: See the end of the article for children's shoes anxious to preview the code)

Principles of Deformation Technique

Although the effects of deformation are strange, they are often inseparable from these three elements: the position of the deformation, the range of influence and the degree of deformation.

Therefore, its realization in Shader is by constructing a deformation function, passing in the original uv coordinates, the position, range and strength of the deformation, and generating the deformed sampling coordinates after calculation. The code is as follows:

#iChannel0 "src/assets/texture/joker.png"

vec2 deform(vec2 uv, vec2 center, float range, float strength) {
  // TODO: 变形处理
  return uv;
}

void mainImage(out vec4 fragColor, vec2 coord) {
    vec2 uv = coord / iResolution.xy;
    vec2 mouse = iMouse.xy / iResolution.xy;
    uv = deform(uv, mouse, .5, .5);
    vec3 color = texture(iChannel0, uv).rgb;
    fragColor.rgb = color;
}

The shader code in this article adopts the GLSL specification and follows the Shader-Toy writing method, which is convenient for everyone to preview.

Deformation tips: sampling distance field transformation

We set the fixed-point coordinate O, the distance from any point to point O is dist , with different dist values as the radius, and point O as the center can form countless equidistant sampling circles, which are called the distance field of point O.

We can change the texture sampling position by changing the size and position of the sampling circle to achieve the deformation effects of expansion/contraction, extrusion/stretching.

vec2 deform(vec2 uv, vec2 center, float range, float strength) {
  float dist = distance(uv, center);
  vec2 direction = normalize(uv - center);
  dist = transform(dist, range, strength); // 改变采样圈半径
  center = transform(center, dist, range, strength); // 改变采样圈中心位置
  return center + dist * direction;
}

The application of this technique is not in a hurry, but now we will start with a simple distortion.

distortion

The twisting effect is similar to the shape of a vortex, and the characteristic is that the closer to the center point, the more intense the rotation. We can use a decreasing function to express the relationship between the distance d from the center point and the corresponding rotation angle θ.

As shown in the figure below, a simple θ = -A/R *d + A , where A represents the rotation angle of the distortion center, A is a positive number to indicate that the direction of rotation is clockwise, a negative number indicates that the direction of rotation is counterclockwise, and R indicates the boundary of the distortion;

As shown in the figure above, the distortion function input parameters A (central rotation angle Angle) and R (deformation range Range) can be described as follows:
1) A represents the central rotation angle, the greater the absolute value, the higher the degree of distortion;
2) A> 0 means that the twisting direction is clockwise, otherwise A<0 means counterclockwise;
3) R represents the distortion boundary, the larger the value, the larger the influence range.

We can introduce the time variable time to dynamically change the value of A to produce twisting special effects, such as the clown twisting effect in the figure above. The specific shader code is as follows:

#iChannel0 "src/assets/texture/joker.png"
#define Range .3
#define Angle .5
#define SPEED 3.
mat2 rotate(float a) // 旋转矩阵
{
    float s = sin(a);
    float c = cos(a);
    return mat2(c,-s,s,c);
}
vec2 twirl(vec2 uv, vec2 center, float range, float angle) {
    float d = distance(uv, center);
    uv -=center;
    // d = clamp(-angle/range * d + angle,0.,angle); // 线性方程
    d = smoothstep(0., range, range-d) * angle;
    uv *= rotate(d);
    uv+=center;
    return uv;
}
void mainImage(out vec4 fragColor, vec2 coord) {
    vec2 uv = coord / iResolution.xy;
    vec2 mouse = iMouse.xy / iResolution.xy;
    float cTime = sin(iTime * SPEED);
    uv = twirl(uv, mouse, Range, Angle * cTime);
    vec4 color = texture(iChannel0, uv);
    fragColor = color;
} 

It is worth mentioning that in addition to expressing the distortion relationship with a linear equation, you can also use the smoothstep method. Compared with the linear linear function, the smoothstep method is smoother at the distortion boundary, as shown in the figure below.

Considering the smoothness of the boundary, the following deformation methods often use the smoothstep function to replace the linear equation.

Expansion/contraction

The texture near the expansion center is stretched, and the texture near the expansion boundary is extruded. This means that within the expansion range, with the expansion center as the distance field, each sampling circle should be smaller than the original radius, and the circle The spacing gradually expands from the inside to the outside.

As shown on the right side of the figure below, we map the equally spaced black sampling circles to the more cohesive red sampling circles, so that the spacing between the new sampling circles increases monotonically from the inside to the outside.

We sample the smooth increase function smoothstep to calculate the scale value scale through the sampling circle radius dist:

The function in the figure above shows that near the center of expansion, the sampling circle has the most obvious scaling and the smallest scaling value (1-S); as dist increases, the scaling value scale increases by 1, until it reaches the R boundary range, the scale is constant 1. The sampling circle is no longer zoomed.

 float scale = (1.- S) + S * smoothstep(0.,1., dist / R); // 计算膨胀采样半径缩放值

So we get the above sampling radius scaling formula, where Strength (0 <S <1) is set to represent the degree of expansion.
For the transformation process of the expansion distance field, it is easy to infer that to achieve the reverse effect of expansion contract , just let S be in the interval [-1,0].

As shown in the figure above, the expansion function input parameters S (Strength of deformation) and R (Range of deformation) can be described as follows:
1) When S is in the range of [0,1], the expansion effect appears. The larger the S value, the higher the degree of expansion;
2) When S is in the range of [-10], shrinks . The smaller the S value, the higher the shrinkage;
3) R represents the boundary of deformation, the larger the value, the larger the affected area;

We can introduce the time variable time to dynamically change the value of Strength to simulate breathing animation, as shown in the figure above, the clown bulging effect, the specific shader code is as follows:

#iChannel0 "src/assets/texture/joker.png"
#define SPEED 2. // 速度
#define RANGE .2 // 变形范围
#define Strength .5 * sin(iTime * SPEED) // 变形程度

vec2 inflate(vec2 uv, vec2 center, float range, float strength) {
    float dist = distance(uv , center);
    vec2 dir = normalize(uv - center);
    float scale = 1.-strength + strength * smoothstep(0., 1. ,dist / range);
    float newDist = dist * scale;
    return center + newDist * dir;
}
void mainImage(out vec4 fragColor, vec2 coord) {
    vec2 uv = coord / iResolution.xy;
    vec2 mouse = iMouse.xy / iResolution.xy;
    uv = inflate(uv, mouse, RANGE, Strength);
    vec3 color = texture(iChannel0, uv).rgb;
    fragColor.rgb = color;
}

Longitudinal/horizontal stretch

The previous expansion is achieved by scaling the sampling circle of the distance field, and the vertical/horizontal stretching is to scale only the x-axis or y-axis of the sampling circle, which can generally be used for the "long-leg effects" of beauty.

It can be found that the horizontal stretched distance field is transformed into multiple elliptical sampling circles. The code implementation is as follows:

vec2 inflateX(vec2 uv, vec2 center, float radius, float strength) {
    // 前面代码跟膨胀实现一样
    ...
    return center + vec2(newDist, dist) * dir; // 横向拉伸则scale只作用于想x轴
}

extrusion

Extrusion generally indicates an action point and an extrusion direction, and its characteristic is to push the texture near the point of action to the end of the extrusion.

As shown in the figure below, the green point of action P is the starting point of the extrusion, and the arrow is the extrusion vector V, where the vector direction indicates the direction of the extrusion, the vector length length(V) represents the distance of the extrusion, and the end of the vector is the position after the extrusion.
To achieve texture extrusion, the center of the sampling circle is offset to the extrusion vector V, and the sampling center point should be translated to the position of point P.

As the radius dist of the sampling circle gradually increases from the inside to the outside, the transformed circle center offset offset gradually decreases. We can use the -smoothstep smoothly decreasing function to process the relationship between the sampling circle radius dist and the circle offset offset.

Formula: offset = length(V) - length(V) * smoothstep(0, R, dist) , where R represents the extrusion boundary range.

Similarly, we introduce the time variable time to dynamically change the length and direction of the extrusion vector, which can realize the jitter effect, as shown in the figure above, the clown top crotch effect, the specific shader code is as follows:

#iChannel0 "src/assets/texture/joker.png"
#define RANGE .25  // 变形范围
#define PINCH_VECTOR vec2( sin(iTime * 10.), cos(iTime * 20.)) * .03 // 挤压向量

vec2 pinch(vec2 uv, vec2 targetPoint, vec2 vector, float range) 
{ 
    vec2 center = targetPoint + vector;
    float dist = distance(uv, targetPoint);
    vec2 point = targetPoint +  smoothstep(0., 1., dist / range) * vector;
    return uv - center + point;
}
void mainImage(out vec4 fragColor, vec2 coord) {
    vec2 uv = coord / iResolution.xy;
    vec2 mouse = iMouse.xy / iResolution.xy;
    uv = pinch(uv, mouse, PINCH_VECTOR, RANGE);
    vec3 color = texture(iChannel0, uv).rgb;
    fragColor.rgb = color;
}

to sum up

This article mainly introduces the realization principles of three types of local deformation shaders. The expansion/contraction and extrusion effects are realized by sampling distance field transformation. The former transforms the size of the sampling circle, and the latter transforms the circle position.

In addition to these three types of local deformations, there are also some interesting global deformations such as waves, dislocations, and mirroring effects. The shader is relatively easy to implement, so I won't introduce more.

Preview code and effect

Distortion: https://www.shadertoy.com/view/slfGzN
/Zoom: 160ab6b218e7ee https://www.shadertoy.com/view/7lXGzN
Squeeze/stretch: https://www.shadertoy.com/view/7tX3zN

related articles

Walk into the graphic noise: https://zhuanlan.zhihu.com/p/68507311
glsl basic transformation: https://thebookofshaders.com/08/?lan=ch
Photoshop extrusion special effect algorithm: https://blog.csdn.net/kezunhai/article/details/41873775