Apache APISIX v2.14.1 Exploratory Release Released, Entering More Fields

It has been more than two months since the last release of APISIX v2.13 LTS. In the past, every minor version release of APISIX will bring you new functions. However, the functions released in APISIX v2.14.1 will keep up with the cutting-edge technology, bringing you many exploratory new functions, and asking for directions for the release of APISIX v3. Welcome to explore these new functions.

Next, let's take a look at what exploratory new features APISIX supports.

WebSocket-based pubsub proxy framework

Before APISIX v2.14.1, whether it was proxying gRPC requests or ordinary HTTP requests, the upstream of APISIX was a docking application server, which could not meet the needs of diversified scenarios. For example, if the user needs to use other upstream types (such as Kafka), it can only be achieved by other means. But in APISIX v2.14.1, APISIX adds a new Websocket-based message subscription broker framework, which allows clients to subscribe to messages in the specified message queue (upstream) through APISIX. Now you can use APISIX to subscribe to messages in Kafka.

Taking Kafka as an example, we need the following configuration:

 curl -X PUT 'http://127.0.0.1:9080/apisix/admin/routes/kafka' \
    -H 'X-API-KEY: ${api-key}' \
    -H 'Content-Type: application/json' \
    -d '{
    "uri": "/kafka",
    "upstream": {
        "nodes": {
            "kafka-server1:9092": 1,
            "kafka-server2:9092": 1,
            "kafka-server3:9092": 1
        },
        "type": "none",
        "scheme": "kafka"
    }
}'

The above example adds a Kafka-type upstream to the route and includes multiple brokers.

You can subscribe to this upstream by referring to the following steps:

1. First please establish a connection via WebSocket.
2. Get the current offset of a Partition in the topic. The following example uses Protobuf to encode the associated request and response:

 message PubSubReq {
    int64 sequence = 1;
    oneof req {
        CmdEmpty cmd_empty                       = 31;
        CmdPing cmd_ping                         = 32;
        CmdKafkaFetch      cmd_kafka_fetch       = 33;
        CmdKafkaListOffset cmd_kafka_list_offset = 34;
    };
}

message PubSubResp {
    int64 sequence = 1;
    oneof resp {
        ErrorResp error_resp                       = 31;
        PongResp pong_resp                         = 32;
        KafkaFetchResp kafka_fetch_resp            = 33;
        KafkaListOffsetResp kafka_list_offset_resp = 34;
    };
}

For example, the request to get the offset is:

 message CmdKafkaListOffset {
    string topic = 1;
    int32 partition = 2;
    int64 timestamp = 3;
}

Please refer to pubsub.proto for the specific meaning of each field.

3. After each subscription operation, the latest news can be obtained according to the current offset.

Note: After successfully obtaining the message, the current offset needs to be updated, and the updated offset is the previously returned offset + 1.

For specific operations, please refer to the source code and test cases:

• https://github.com/apache/apisix/blob/master/t/pubsub/kafka.t
• https://github.com/apache/apisix/blob/master/t/lib/pubsub.lua

Although the current Pubsub framework only provides a low-level interface, it already fulfills two basic requirements:

• The Kafka service capability is exposed through the commonly used 80/443 port, and there is no need to encapsulate an additional layer of the application server.
• Allows adding authentication plugins to add security protection to Kafka's services like using a general Websocket framework.

If you encounter problems during actual use, you can report to the Apache APISIX community by submitting an issue, and the community will continue to improve and enhance this function based on user feedback.

Manage non-HTTP layer 7 protocols based on the xRPC framework

APISIX supports proxy TCP protocol in early versions, but in some scenarios, pure TCP protocol proxy cannot meet user needs. Because some functions can only be implemented after encoding and decoding the application protocol, users need a proxy for a specific application protocol, such as Redis Proxy, Kafka Proxy, etc.

Starting from APISIX v2.14.1, APISIX provides the xRPC framework, which allows developers to customize specific application protocols on the framework. Based on the xRPC framework, APISIX can provide proxy support for several mainstream application protocols. At the same time, users can also support their own private TCP-based application protocols based on this framework, so that they have the precise granularity and higher-level 7-layer control similar to HTTP protocol proxy.

At present, APISIX has implemented the proxy function of Redis on the xRPC framework, which supports injecting delay and selectively recording log content according to the command. Although APISIX needs to encode and decode the Redis protocol, in a simple SET/GET performance test, using APISIX with dual worker processes as a proxy, its performance can reach 80% of the direct connection to Redis.

You can create a stream route proxying the Redis protocol by referring to the following command:

 curl http://127.0.0.1:9080/apisix/admin/stream_routes/1 \
-H 'X-API-KEY: ${api-key}' -X PUT -d '
{
    "upstream": {
        "type": "none",
        "nodes": {
            "127.0.0.1:6379": 1
        }
    },
    "protocol": {
        "name": "redis",
        "conf": {
            "faults": [{
                "commands": ["get", "ping"],
                "delay": 5
            }]
        },
        logger = {
            [
                "name": "syslog",
                "filter": [
                    ["rpc_time", ">=", 1],
                ],
                "conf": {
                    "host": "127.0.0.1",
                    "port": 8125,
                    "sock_type": "udp",
                    "batch_max_size": 1,
                    "flush_limit": 1
                }
            ]
        }
    }
}'

The above configuration is explained as follows:

When the command is GET or ping, there will be a 5 second delay. At the same time, after each command is executed, it will judge whether it takes more than 1 second, if so, trigger the corresponding logger object, and send syslog UDP log to 127.0.0.1 the 8125 port.

Control plane supports service discovery

Before v2.14.1, APISIX only supported service discovery on the data plane. In this case, each APISIX instance needs to obtain service discovery data, but users have reported the following problems in the actual application process:

1. Each APISIX instance needs to pull data from the service discovery system, which complicates the network topology.
2. Service discovery configuration needs to be configured on each APISIX instance. To change the password, you must modify the configuration file, and then publish to each APISIX instance.
3. Currently, many service discovery systems do not provide Lua SDK. If you want to use these service discovery systems, you need to directly connect to the HTTP API provided by the server (if it exists).

Therefore, starting from v2.14.1, APISIX will support service discovery on the control plane. Service discovery will be implemented through APISIX-Seed .

The principle of this function is to monitor the Upstream-related resources in etcd and the corresponding upstream service resources in the service discovery component through apisix-seed and update the relevant Upstream in etcd when the upstream service resources in the service discovery component change. information.

The specific implementation process is as follows:

At present, the solution of the control plane service discovery also has shortcomings, such as the pressure on etcd. Therefore, APISIX will keep two service discovery schemes at the same time, and prove which scheme will be better through more practical applications.

Initial support for Istio

In order to adapt to a wider range of application scenarios, starting from v2.14.1, APISIX will try to be compatible with Istio, using Istio as the control plane and APISIX as the data plane, to start exploring in the field of service mesh.

Since the configuration of Istio is issued through the xDS protocol, the Amesh project was developed to convert the xDS issued by Istio into the configuration of APISIX. At present, APISIX has been able to run through the official Istio Simple Bookstore App demo. In subsequent releases, APISIX will continue to expand its support for xDS, bringing the capabilities of Istio and APISIX closer together.

More plugins and features

In addition to the exploratory features mentioned above, this release also provides users with some more traditional features:

• Added casdoor plugin to improve the interaction experience with Casdoor.
• response-rewrite The plugin adds a replacement filter for Body.

For more feature updates and bug fixes details, please check the official Releases CHANGELOG .