Use WasmEdge to embed WebAssembly functions to extend Golang applications

The GO programming language (Golang) is an easy-to-use and safe programming language that can be compiled into high-performance native applications. Golang is a very popular choice for writing software infrastructure and frameworks.

A key requirement of the software framework is that users can extend and customize it with their own code. However, in Golang, it is not easy to add user-defined functions or extensions to existing applications. Generally, it is necessary to integrate at the source code level through the source code of the combo framework and user-defined functions. Although Golang can be used to create dynamic shared modules, these ARM-based systems that are widely used in edge computing lack support for shared modules. In addition, neither source code integration nor dynamic modules provide isolation for user-defined functions. Extensions may interfere with the framework itself, and it would be unsafe to integrate multiple user-defined functions. Therefore, as a "cloud native" language, Golang needs a better extension mechanism.

WebAssembly provides a powerful, flexible, safe and simple extension mechanism that can embed user-defined functions into Golang applications. Originally invented for web browsers, but increasingly used in standalone and server-side applications, WebAssembly is a lightweight software container for its bytecode applications. WebAssembly is high-performance, portable, and supports multiple programming languages.

In this tutorial, we will discuss how to run WebAssembly functions from a Golang application. WebAssembly functions are written in Rust. They are well isolated from the Golang host application, and functions are also isolated from each other.

Ready to work

Obviously, we need to install Golang . It is assumed that you have already installed it.

The Golang version should be higher than 1.15 for our example to work.

Next, please install the WasmEdge shared library. WasmEdge is a leading WebAssembly runtime hosted CNCF We will use WasmEdge to embed and run WebAssembly programs in Golang applications.

$ wget https://github.com/second-state/WasmEdge-go/releases/download/v0.8.1/install_wasmedge.sh
$ chmod +x ./install_wasmedge.sh
$ sudo ./install_wasmedge.sh /usr/local

Finally, since our demo WebAssembly function is written in Rust, you also need to install the Rust compiler and the rustwasmc tool chain .

Embed a function

Currently, we need Rust compiler version 1.50 or lower to allow WebAssembly functions to be used with WasmEdge's Golang API. Once the interface type specification is finalized and supported, we will catch up with the latest Rust compiler version .

In this example, , we will demonstrate how to call some simple WebAssembly functions from a Golang application. These functions are written in Rust and require complex call parameters and return values. The compiler tool needs the #[wasm_bindgen] macro to automatically generate the correct code to pass the call parameters from Golang to WebAssembly.

The WebAssembly specification only supports some simple data types out of the box. Wasm does not support such as string and array types. In order to pass the rich types in Golang to WebAssembly, the compiler needs to convert them into simple integers. For example, it converts a string to an integer memory address and integer length. The wasm_bindgen tool embedded in rustwasmc will automatically perform this conversion.

use wasm_bindgen::prelude::*;
use num_integer::lcm;
use sha3::{Digest, Sha3_256, Keccak256};

#[wasm_bindgen]
pub fn say(s: &str) -> String {
  let r = String::from("hello ");
  return r + s;
}

#[wasm_bindgen]
pub fn obfusticate(s: String) -> String {
  (&s).chars().map(|c| {
    match c {
      'A' ..= 'M' | 'a' ..= 'm' => ((c as u8) + 13) as char,
      'N' ..= 'Z' | 'n' ..= 'z' => ((c as u8) - 13) as char,
      _ => c
    }
  }).collect()
}

#[wasm_bindgen]
pub fn lowest_common_multiple(a: i32, b: i32) -> i32 {
  let r = lcm(a, b);
  return r;
}

#[wasm_bindgen]
pub fn sha3_digest(v: Vec<u8>) -> Vec<u8> {
  return Sha3_256::digest(&v).as_slice().to_vec();
}

#[wasm_bindgen]
pub fn keccak_digest(s: &[u8]) -> Vec<u8> {
  return Keccak256::digest(s).as_slice().to_vec();
}

First, we use the rustwasmc tool to compile the Rust source code into WebAssembly bytecode functions. Please use Rust 1.50 or lower.

$ rustup default 1.50.0
$ cd rust_bindgen_funcs
$ rustwasmc build
# The output WASM will be pkg/rust_bindgen_funcs_lib_bg.wasm

Golang source code An example of the WebAssembly function running in WasmEdge is as follows. ExecuteBindgen() function calls the WebAssembly function and uses #[wasm_bindgen] pass in the parameters.

package main

import (
    "fmt"
    "os"
    "github.com/second-state/WasmEdge-go/wasmedge"
)

func main() {
    /// Expected Args[0]: program name (./bindgen_funcs)
    /// Expected Args[1]: wasm or wasm-so file (rust_bindgen_funcs_lib_bg.wasm))

    wasmedge.SetLogErrorLevel()

    var conf = wasmedge.NewConfigure(wasmedge.WASI)
    var vm = wasmedge.NewVMWithConfig(conf)
    var wasi = vm.GetImportObject(wasmedge.WASI)
    wasi.InitWasi(
        os.Args[1:],     /// The args
        os.Environ(),    /// The envs
        []string{".:."}, /// The mapping directories
        []string{},      /// The preopens will be empty
    )

    /// Instantiate wasm
    vm.LoadWasmFile(os.Args[1])
    vm.Validate()
    vm.Instantiate()

    /// Run bindgen functions
    var res interface{}
    var err error
    
    res, err = vm.ExecuteBindgen("say", wasmedge.Bindgen_return_array, []byte("bindgen funcs test"))
    if err == nil {
        fmt.Println("Run bindgen -- say:", string(res.([]byte)))
    } 
    res, err = vm.ExecuteBindgen("obfusticate", wasmedge.Bindgen_return_array, []byte("A quick brown fox jumps over the lazy dog"))
    if err == nil {
        fmt.Println("Run bindgen -- obfusticate:", string(res.([]byte)))
    } 
    res, err = vm.ExecuteBindgen("lowest_common_multiple", wasmedge.Bindgen_return_i32, int32(123), int32(2))
    if err == nil {
        fmt.Println("Run bindgen -- lowest_common_multiple:", res.(int32))
    } 
    res, err = vm.ExecuteBindgen("sha3_digest", wasmedge.Bindgen_return_array, []byte("This is an important message"))
    if err == nil {
        fmt.Println("Run bindgen -- sha3_digest:", res.([]byte))
    } 
    res, err = vm.ExecuteBindgen("keccak_digest", wasmedge.Bindgen_return_array, []byte("This is an important message"))
    if err == nil {
        fmt.Println("Run bindgen -- keccak_digest:", res.([]byte))
    } 

    vm.Delete()
    conf.Delete()
}

Next, let's use the WasmEdge Golang SDK to build a Golang application.

$ go get -u github.com/second-state/WasmEdge-go/wasmedge
$ go build

Run the Golang application and it will run the WebAssembly functions embedded in WasmEdge Runtime.

$ ./bindgen_funcs rust_bindgen_funcs/pkg/rust_bindgen_funcs_lib_bg.wasm
Run bindgen -- say: hello bindgen funcs test
Run bindgen -- obfusticate: N dhvpx oebja sbk whzcf bire gur ynml qbt
Run bindgen -- lowest_common_multiple: 246
Run bindgen -- sha3_digest: [87 27 231 209 189 105 251 49 159 10 211 250 15 159 154 181 43 218 26 141 56 199 25 45 60 10 20 163 54 211 195 203]
Run bindgen -- keccak_digest: [126 194 241 200 151 116 227 33 216 99 159 22 107 3 177 169 216 191 114 156 174 193 32 159 246 228 245 133 52 75 55 27]

Embed an entire program

You can use the latest Rust compiler and main.rs create a separate WasmEdge application, and then embed it in a Golang application.

In addition to functions, WasmEdge Golang SDK can also into a stand-alone WebAssembly application , which is to compile a Rust application main()

Our demo Rust application from a file. #{wasm_bindgen] is not needed here, because the input and output data of the WebAssembly program is now passed STDIN and STDOUT

use std::env;
use std::fs::File;
use std::io::{self, BufRead};

fn main() {
    // Get the argv.
    let args: Vec<String> = env::args().collect();
    if args.len() <= 1 {
        println!("Rust: ERROR - No input file name.");
        return;
    }

    // Open the file.
    println!("Rust: Opening input file \"{}\"...", args[1]);
    let file = match File::open(&args[1]) {
        Err(why) => {
            println!("Rust: ERROR - Open file \"{}\" failed: {}", args[1], why);
            return;
        },
        Ok(file) => file,
    };

    // Read lines.
    let reader = io::BufReader::new(file);
    let mut texts:Vec<String> = Vec::new();
    for line in reader.lines() {
        if let Ok(text) = line {
            texts.push(text);
        }
    }
    println!("Rust: Read input file \"{}\" succeeded.", args[1]);

    // Get stdin to print lines.
    println!("Rust: Please input the line number to print the line of file.");
    let stdin = io::stdin();
    for line in stdin.lock().lines() {
        let input = line.unwrap();
        match input.parse::<usize>() {
            Ok(n) => if n > 0 && n <= texts.len() {
                println!("{}", texts[n - 1]);
            } else {
                println!("Rust: ERROR - Line \"{}\" is out of range.", n);
            },
            Err(e) => println!("Rust: ERROR - Input \"{}\" is not an integer: {}", input, e),
        }
    }
    println!("Rust: Process end.");
}

Use the rustwasmc tool to compile the application into WebAssembly.

$ cd rust_readfile
$ rustwasmc build
# The output file will be pkg/rust_readfile.wasm

The Golang source code runs on the WebAssembly function in WasmEdge, as follows:

package main

import (
    "os"
    "github.com/second-state/WasmEdge-go/wasmedge"
)

func main() {
    wasmedge.SetLogErrorLevel()

    var conf = wasmedge.NewConfigure(wasmedge.REFERENCE_TYPES)
    conf.AddConfig(wasmedge.WASI)
    var vm = wasmedge.NewVMWithConfig(conf)
    var wasi = vm.GetImportObject(wasmedge.WASI)
    wasi.InitWasi(
        os.Args[1:],     /// The args
        os.Environ(),    /// The envs
        []string{".:."}, /// The mapping directories
        []string{},      /// The preopens will be empty
    )

    /// Instantiate wasm. _start refers to the main() function
    vm.RunWasmFile(os.Args[1], "_start")

    vm.Delete()
    conf.Delete()
}

Next, let's use the WasmEdge Golang SDK to build a Golang application.

$ go get -u github.com/second-state/WasmEdge-go
$ go build

Run the Golang application.

$ ./read_file rust_readfile/pkg/rust_readfile.wasm file.txt
Rust: Opening input file "file.txt"...
Rust: Read input file "file.txt" succeeded.
Rust: Please input the line number to print the line of file.
# Input "5" and press Enter.
5
# The output will be the 5th line of `file.txt`:
abcDEF___!@#$%^
# To terminate the program, send the EOF (Ctrl + D).
^D
# The output will print the terminate message:
Rust: Process end.

Next

In this article, we show two ways to embed WebAssembly functions in Golang applications: embedding a WebAssembly function and embedding a complete program. For more examples, please refer to WasmEdge-go-examples GitHub repo .

In the next article, we will study a complete example of embedding AI inference (image recognition) functions into a Golang-based real-time streaming data processing framework. This has practical applications in smart factories and automobiles.