How we run KrakenD on Javascript with WebAssembly
by Daniel Lopez
Dec 9, 2018
KrakenD is an API Gateway written in Go that uses a single configuration file to define its whole behavior. As the configuration file might be complicated, the KrakenDesigner is a javascript-based user interface to edit this file, and we were missing the capability of reproducing directly on javascript the existing gateway pipes so that users could run manual tests over the editing configuration.
In this post, we are going to explain how we included KrakenD framework components in a .wasm file and how we integrated it into our existing SPA. This is our go code running on javascript.
Let’s start!
Golang 1.11 and WebAssembly
During the last 3 years, the popularity of WebAssembly (and its promise to replace JS) has been increasing at a breakneck pace.
From the project’s home page:
WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.
The latest release of the golang language comes with an experimental port to WebAssembly.
Quick environment setup
There are two dependencies in this project, be sure to have both installed correctly in your environment
- golang 1.11
- dep 0.5.0
To reproduce the steps we did, let’s start with a clean Go project. Copy the JS and HTML already bundled with your golang distribution to your new project.
cd myproject/
$ cp "$(go env GOROOT)/misc/wasm/wasm_exec.js" .
$ cp "$(go env GOROOT)/misc/wasm/wasm_exec.html" index.html
Now add a very simple Makefile that is going to download any dependencies and generate the .wasm file:
all: prepare build
prepare:
dep ensure -v
build:
GOARCH=wasm GOOS=js go build -o test.wasm main.go
Running make is all it takes to generate the test.wasm file, but to get a faster development environment, we’ve also used two handy tools:
- reflex watches changes in the
main.gofile and recompiles automatically. - goexec helps us launch Go one-liners, that we are using to launch a local web server.
Open a new terminal, move to your project and start the reflex task:
$ reflex -r '^main.go$' make build
We are creating the main.go content in a minute. Now, any changes happening to the main.go file execute a make build every time.
In a second terminal, move to your project and create the basic local HTTP server for serving the files index.html, wasm_exec.js, and test.wasm
$ goexec 'http.ListenAndServe(":8080", http.FileServer(http.Dir(".")))'
It’s time to test if everything is properly working. Create a file called main.go with this content:
package main
import "fmt"
func main() {
fmt.Println("Hello, WebAssembly!")
}
Open your browser at http://localhost:8080 and click the run button. You should find the Hello, WebAssembly! message in your console.
Using go libs
With our environment ready, we can start hacking our way in. So let’s create the minimal required code to expose a golang function to the JS space called parse.
package main
import (
"fmt"
"syscall/js"
)
func parse(i []js.Value) {
fmt.Println(i)
}
func main() {
fmt.Println("WASM Go Initialized")
js.Global().Set("parse", js.NewCallback(parse))
select {}
}
You can test it in the console (after clicking run):
WASM Go Initialized
> parse("aaaa", {a:42,b:true}, ["one","two"]);
[aaaa [object Object] one,two]
As we are going to use external dependencies, we’ll use the same system as in the KrakenD framework: dep. There are some problems using the versions locked in the framework in our WASM, so you can create a file called Gopkg.toml with the following content:
[[override]]
name = "github.com/mattn/go-isatty"
revision = "3fb116b820352b7f0c281308a4d6250c22d94e27"
[[override]]
name = "github.com/gin-gonic/gin"
branch = "master"
[[override]]
version = "v2.2.1"
source = "github.com/go-yaml/yaml"
name = "gopkg.in/yaml.v2"
[[constraint]]
name = "github.com/devopsfaith/krakend"
branch = "master"
[prune]
go-tests = true
unused-packages = true
With the Gopkg.toml in the root of your project, prepare it by typing
make prepare
The easiest way to create a KrakenD is by using just the framework. Let’s alter the parse function so that it dumps the result of the actual KrakenD config parser:
func parse(i []js.Value) {
cfg, err := config.NewParserWithFileReader(func(s string) ([]byte, error) {
return []byte(s), nil
}).Parse(i[0].String())
if err != nil {
fmt.Println("error:", err.Error())
return
}
fmt.Printf("%d endpoints parsed:\n", len(cfg.Endpoints))
fmt.Printf("%+v\n", cfg)
}
So far, so good. We already have a running environment and a clear way to port existent golang code into JS.
Building an ephemeral KrakenD instance
Before digging more in-depth in the function bindings and translations required to consume the golang libs from the JS space, we need some wrapper over the ephemeral KrakenD instances so we can consume them without starting an HTTP server inside the browser.
All the router implementations of the framework accept a RunServerFunc and delegate to it the setup of the server for exposing the http.Handler containing the KrakenD instance. So, if we want to avoid the server instantiation and capture the handler, we can create a custom implementation of the client.RunServer. As you can see, we also catch the cancel function of the instance context, so the wrapper can offer a way to close the embedded KrakenD service.
type LocalServer struct {
close func()
handler func(rw http.ResponseWriter, req *http.Request)
}
func newServer(cfg string) (*LocalServer, error) {
// parse the received config string
serviceConfig, err := config.NewParserWithFileReader(func(s string) ([]byte, error) {
return []byte(s), nil
}).Parse(cfg)
if err != nil {
return nil, err
}
serviceConfig.Debug = true
// instantiate the framework logger
logger, err := logging.NewLogger("DEBUG", os.Stdout, "[KRAKEND]")
if err != nil {
return nil, err
}
// create a context for the ephimeral instance
ctx, cancel := context.WithCancel(context.Background())
s := &LocalServer{
// capture the context cancel function
close: cancel,
// empty handler to override at the RunServer func
handler: func(rw http.ResponseWriter, req *http.Request) {},
}
routerFactory := krakendgin.NewFactory(krakendgin.Config{
Engine: gin.New(),
ProxyFactory: proxy.DefaultFactory(logger),
Logger: logger,
HandlerFactory: krakendgin.EndpointHandler,
// RunServer just captures the handler and waits for a context cancelation
RunServer: func(ctx context.Context, _ config.ServiceConfig, handler http.Handler) error {
s.handler = handler.ServeHTTP
<-ctx.Done()
return ctx.Err()
},
})
// start the service in a goroutine
go func(ctx context.Context, serviceConfig config.ServiceConfig) {
routerFactory.NewWithContext(ctx).Run(serviceConfig)
}(ctx, serviceConfig)
return s, nil
}
With the LocalServer ready, we can build the simplest consumer/client with some help from the httptest package:
type Client struct {
Server *LocalServer
}
func (c Client) Do(req *http.Request) *http.Response {
req.Header.Add("js.fetch:credentials", "omit")
rw := httptest.NewRecorder()
c.Server.handler(rw, req)
return rw.Result()
}
func (c *Client) Close() {
c.Server.close()
}
Notice we add the header js.fetch:credentials so no cookies are sent with the requests. This is a feature of the golang adapter of the JS fetch function. The comments at the source code of the golang stdlib show the possible values.
// jsFetchMode is a Request.Header map key that, if present,
// signals that the map entry is actually an option to the Fetch API mode setting.
// Valid values are: "cors", "no-cors", "same-origin", "navigate"
// The default is "same-origin".
//
// Reference: https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/fetch#Parameters
const jsFetchMode = "js.fetch:mode"
// jsFetchCreds is a Request.Header map key that, if present,
// signals that the map entry is actually an option to the Fetch API credentials setting.
// Valid values are: "omit", "same-origin", "include"
// The default is "same-origin".
//
// Reference: https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/fetch#Parameters
const jsFetchCreds = "js.fetch:credentials"
And that’s all we need to embed a KrakenD instance into a simple client. As a footnote, trying to avoid the CORS validation of the browser will give you the pleasure of meeting the CORB layer.
Adding callbacks
Since we are not allowed to return any values from the transpiled functions, we must accept some kind of callback function both for success and for the error case… you can think about it as if it was some special kind of promise constructor.
With that important point already in mind, we can define the single function we will add to the DOM: parse.
func parse(i []js.Value) {
if len(i) < 2 {
println("not enough args")
return
}
if i[1].Type() != js.TypeFunction {
println("arg 1 should be a function")
return
}
logger := func(msg string) {
fmt.Println(msg)
}
if len(i) > 2 {
if i[2].Type() == js.TypeFunction {
logger = func(msg string) {
i[2].Invoke(msg)
}
}
}
client, err := newJSClient(i[0].String(), logger)
if err != nil {
logger(err.Error())
return
}
i[1].Invoke(client.Value())
}
As you can see, this function defines a logger, creates a JS client of the KrakenD gateway and invokes a callback. The default logger is based on the fmt.Println function but parse also accepts a third param as an error logger.
The newJSClient function creates a JS adapter over the already defined Client.
func newJSClient(cfg string, logger func(string)) (*JSClient, error) {
server, err := newServer(cfg)
if err != nil {
return nil, err
}
return &JSClient{
client: &Client{server},
logger: logger,
}, nil
}
type JSClient struct {
client *Client
logger func(string)
}
The JSClient struct has a single exported method Value so it can generate a JS object with bindings to the Close and Do methods exposed by the Client.
func (j *JSClient) Value() js.Value {
opt := js.Global().Get("Object").New()
opt.Set("close", js.NewCallback(j.close))
opt.Set("test", js.NewCallback(j.test))
return opt
}
func (j *JSClient) close(_ []js.Value) {
j.client.Close()
}
The most important part of the JSClient is the test method, where the request is built and dispatched in an independent goroutine. After parsing the response into a JS object, the goroutine executes the injected callback and ends.
func (j *JSClient) test(i []js.Value) {
// check if there are enough arguments
if len(i) < 5 {
j.logger("the test function requires at least 5 arguments: method, path, body, headers and a callback")
return
}
// inject the body if present
var reqBody io.Reader
if b := i[2].String(); b != "" {
reqBody = bytes.NewBufferString(b)
}
// create the http request
req, err := http.NewRequest(i[0].String(), i[1].String(), reqBody)
if err != nil {
j.logger(err.Error())
return
}
// parse and add the injected headers
headers := map[string][]string{}
if err := json.Unmarshal([]byte(i[3].String()), &headers); err != nil {
j.logger(err.Error())
return
}
req.Header = headers
go func() {
// dispatch the request to the client
resp := j.client.Do(req)
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
j.logger(err.Error())
return
}
resp.Body.Close()
// parse the response
headers := js.Global().Get("Object").New()
for k, v := range resp.Header {
headers.Set(k, v[0])
}
jsResp := js.Global().Get("Object").New()
jsResp.Set("statusCode", resp.StatusCode)
jsResp.Set("header", headers)
jsResp.Set("body", string(body))
// invoke the injected callback
i[4].Invoke(jsResp)
}()
}
Testing the KrakenD from JS
We are not front-end developers, and our skills and knowledge in the domain are very restricted. That’s why we thought this should be the most time-consuming part of the project. However, it was not!
We just needed to add this fragment to our template, and we were ready to call the parse function, create a client and test our KrakenD configuration (notice the wasm was renamed and both files placed inside the wasm folder):
<script src='wasm/wasm_exec.js'></script>
<script>
if (!WebAssembly.instantiateStreaming) { // polyfill
WebAssembly.instantiateStreaming = async (resp, importObject) => {
const source = await (await resp).arrayBuffer();
return await WebAssembly.instantiate(source, importObject);
};
}
var krakendClient = {};
var onKrakendClientReady = function() {};
var krakendClientReady = new Promise(function(resolve, reject) {
onKrakendClientReady = resolve;
});
const go = new Go();
let mod, inst;
WebAssembly.instantiateStreaming(fetch("wasm/main.wasm"), go.importObject).then(async (result) => {
mod = result.module;
inst = result.instance;
await go.run(inst);
});
</script>
The script also defines some globals in order to let the rest of the JS code know when the WASM code is loaded and available. With the callback already in place, we can invoke it from the wasm/go code:
func main() {
fmt.Println("WASM Go Initialized")
js.Global().Set("parse", js.NewCallback(parse))
js.Global().Get("onKrakendClientReady").Invoke()
select {}
}
The last step is to introduce the following bits of code into the SPA:
$rootScope.krakendPrepare = function() {
if ( 'undefined' !== typeof krakendClient.close ) {
krakendClient.close();
console.log('Resetting KrakenD client');
}
var cfg = JSON.stringify($rootScope.service);
krakendClientReady.then(function(){
parse(cfg, function(c) {
krakendClient = c;
console.log("KrakenD Client is ready");
})
})
};
$rootScope.runEndpoint = function(requestMethod, requestURL, requestBody, requestHeaders) {
if ( 'undefined' === typeof requestBody ) {
// GET or HEAD methods
requestBody = "";
}
if ( 'undefined' === typeof requestHeaders || requestHeaders == "" ) {
// GET or HEAD methods
requestHeaders = "{}";
}
krakendClient.test(requestMethod, requestURL, requestBody, requestHeaders, console.log);
};
Showing the returned values is just a matter of replacing the console.log for the name of the function responsible for binding the response properties with the view fields.
Conclusion
With very little extra code, we have embedded our essential KrakenD gateway into the browser! We can do the same with almost any existing golang codebase if we follow the lessons learned in this experiment:
- since the WASM code is loaded in an async fashion and the main function is blocked by the empty
select, we need to notify the JS code when the WASM is ready (before entering the infinite wait). - golang functions ported to WASM do not return a thing. They should accept callbacks if there is something to pass/return to the caller.
- HTTP requests should run in a dedicated goroutine. Running on the same goroutine than the caller generates deadlocks.
- GET and HEAD HTTP requests must have a nil body.
- type conversion between JS and Go are tricky. Manual conversions are very verbose.
You can see it in action at https://designer.krakend.io
Thanks for reading! If you like our product, don’t forget to star our project!
