Peripheral ecology | StreamNative announces open source AWS SQS Connector

About Apache Pulsar

Apache Pulsar is the top-level project of the Apache Software Foundation. It is the next-generation cloud-native distributed message flow platform. It integrates messaging, storage, and lightweight functional computing. It uses a separate architecture design for computing and storage to support multi-tenancy, persistent storage, Multi-computer room and cross-regional data replication, with strong consistency, high throughput, low latency and high scalability and other streaming data storage characteristics.
GitHub address: http://github.com/apache/pulsar/

Today, we are proud to announce that StreamNative has released and open sourced the first version of AWS Simple Queue Service (SQS) connector ! Through the SQS connector, you can easily move data between Pulsar and SQS, and take full advantage of the advantages of these two messaging systems. This marks another big step forward for StreamNative in improving the diversity, integrity and productivity of the Pulsar ecosystem.

What is SQS connector?

As we all know, one of the important indicators for testing the message system is its integration with the surrounding ecology. Like other Pulsar connector , SQS connector is an important part of building the surrounding ecology of Apache Pulsar. The SQS connector can help you move data between Pulsar and SQS quickly and efficiently, and has the following two types of connectors:

• SQS source connector-Import SQS data to Pulsar.

• SQS sink connector – Export Pulsar data to SQS.

Why develop SQS connector?

In today's cloud-native era, Apache Pulsar and AWS SQS are important components and basic software for building a distributed architecture.

Pulsar is a next-generation cloud-native distributed messaging system with strong scalability, high consistency and high flexibility, and has a very broad application prospect. In recent years, Pulsar has developed rapidly and its global user base has grown rapidly, which has given rise to the need for more integrated streaming data pipelines and event-driven applications.

SQS is Amazon's fully managed message queue service, suitable for separating and scaling microservices, distributed systems, and serverless applications, and supports dynamic and elastic scaling. Because of its simple operation and quick start, it is quite popular among users.

Both Pulsar and SQS have a wide range of user groups and strong development support, and more and more users are using these two messaging systems. At the same time, we have also received the needs of users, hoping to move data between Pulsar and SQS, and make full use of the advantages of these two messaging systems.

What are the advantages of SQS connector?

In order to make Pulsar and SQS have stable streaming integration, SQS connector has many advantages, including but not limited to:

• Easy to use. No need to write any code, just some simple configuration, you can move data between Pulsar and SQS through the SQS connector. Simplified data import and export process can greatly improve the efficiency of enterprise data usage.
• Easy to expand. According to different business needs, you can choose to run the SQS connector on a single machine or a cluster to help enterprises build real-time data pipelines.
• Sustainable. The simple installation and deployment process of SQS connector enables enterprises to explore the business value of data more focused and more efficiently.

How to participate?

This time, we released SQS connector 2.7.0, and built a wealth of peripheral resources and an active development community. You can participate in the project in any of the following ways:

• Want to try it?
  If you need to download SQS connector 2.7.0 and view the user guide (to help you get started quickly), click here view

• Want to solve the problem?
  If you have any questions, you can create the SQS connector GitHub repo in Issue , or sent to Pulsar mailing List , or Twitter message on, or by adding Pulsar Slack # Connector-SQS channel or join Pulsar WeChat group for discussion. Either way, Pulsar senior experts are always ready to stand by.

• Want to contribute?
  source code of 16198ea50323fb SQS connector is open and hosted on GitHub. If you need to improve features or fix bugs, please submit a PR.

StreamNative has been committed to building a rich and complete ecosystem for Pulsar. Like other Pulsar integrations and plug-ins, the SQS connector is also an important part of the StreamNative Hub As the "living water source" of Pulsar, StreamNative has worked tirelessly to build channels to attract gurgling water, and make every effort to empower users, increase efficiency for enterprises, and work together to break the next cloud era.

Click the link to try out AWS SQS Connector now!