A tour to the runtime of kubeless

kubeless is a Kubernetes-native serverless framework and it leverages Kubernetes resources to provide auto-scaling, API routing, monitoring, troubleshooting and more.

By Kubeless, we can use a Custom Resource Definition to create functions as custom kubernetes resources and then run an in-cluster controller watching these custom resources and launching runtimes on-demand.

In this article, we are focused on the implementation of runtime. Just as the showing of the below chart, Kubeless provides different runtimes for different program languages.

According to the programing language, each runtime is really a kind of web server implemented with different web server framework such as “sinatra” in ruby, “bottle” in python, “express” in nodejs and “com.sun.net.httpserver” in java. In the runtime server, it defines one handler to wrapper the created function and handles the call from http request.

Scaffold in Runtime

The main steps for the implementation of runtime as follows:

Load defined function.
Create one web server.
Create the custom handler to add performance/statistics hook.
Wrapper the defined function with the custom handler function to serve http request.

Let’s look into the details for different runtime.

Python Runtime

The implementation of python runtime is relatively simple. It’s only involved in one source file “kubeless.py”.

The following is the analysis to describe the main logic and steps in this kind of runtime.

Use python module “mod” to load functions.

mod = imp.load_source('function',
                    '/kubeless/%s.py' % os.getenv('MOD_NAME'))
func = getattr(mod, os.getenv('FUNC_HANDLER'))

Define the wrapper function

def funcWrap(q, event, c):
    try:
        q.put(func(event, c))
    except Exception as inst:
        q.put(inst)

Implement the runtime with python http server module “bottle”

  app = application = bottle.app()

if __name__ == '__main__':
    import logging
    import sys
    import requestlogger
    loggedapp = requestlogger.WSGILogger(
        app,
        [logging.StreamHandler(stream=sys.stdout)],
        requestlogger.ApacheFormatter())
    bottle.run(loggedapp, server='cherrypy', host='0.0.0.0', port=func_port)

With “bottle”, implement one handler by the module “multiprocessing” to handle the call of function in sub process.

with func_hist.labels(method).time():
           q = Queue()
           p = Process(target=funcWrap, args=(q, event, function_context))
           p.start()
           p.join(timeout)

Ruby Runtime

Use ruby module “mod” to load functions.

MOD_NAME = ENV['MOD_NAME']
FUNC_HANDLER = ENV['FUNC_HANDLER']
MOD_ROOT_PATH = ENV.fetch('MOD_ROOT_PATH', '/kubeless/')
MOD_PATH = "#{File.join(MOD_ROOT_PATH, MOD_NAME)}.rb"
........
.........
begin
  puts "Loading #{MOD_PATH}"
  mod = Module.new
  mod.module_eval(File.read(MOD_PATH))
  # export the function handler
  mod.module_eval("module_function :#{FUNC_HANDLER}")

Define the wrapper function

def funcWrapper(mod, t)
    status = Timeout::timeout(t) {
      res = mod.send(FUNC_HANDLER.to_sym, @event, @context)
    }
end

Implement the runtime with ruby framework “sinatra” and create “get, post” handler functions

require 'sinatra'
......
......

get '/' do
  begin
    funcWrapper(mod, ftimeout)
  rescue Timeout::Error
    status 408
  end
end

post '/' do
  begin
    funcWrapper(mod, ftimeout)
  rescue Timeout::Error
    status 408
  end
end

Node JS Runtime

Define module “./lib/helper”, Create web server and use “vm.Script” to load functions.

const express = require('express');
const helper = require('./lib/helper');
......
......

const modPath = path.join(modRootPath, `${modName}.js`);
const libPath = path.join(modRootPath, 'node_modules');
const pkgPath = path.join(modRootPath, 'package.json');
const libDeps = helper.readDependencies(pkgPath);
.....
.....
const app = express();
.....
.....
const script = new vm.Script('\nrequire(\'kubeless\')(require(\''+ modPath +'\'));\n', {
    filename: modPath,
    displayErrors: true,
})

In handler function, call “script.runInNewContext” to serve request.

const sandbox = Object.assign({}, global, {
            __filename: modPath,
            __dirname: modRootPath,
            module: new Module(modPath, null),
            require: (p) => modRequire(p, req, res, end),
        });

        try {
            script.runInNewContext(sandbox, { timeout : timeout * 1000 });
        } catch (err) {

Java Runtime

Define HttpServer and use “Class.forName” to load function in java/io/kubeless/Handler.java.

try {
            HttpServer server = HttpServer.create(new InetSocketAddress(port), 0);
            server.createContext("/", new FunctionHandler());
            server.createContext("/healthz", new HealthHandler());
            server.setExecutor(java.util.concurrent.Executors.newFixedThreadPool(50));
            server.start();

            Class<?> c = Class.forName("io.kubeless."+className);
            obj = c.newInstance();
            method = c.getMethod(methodName, io.kubeless.Event.class, io.kubeless.Context.class);
        } catch (Exception e) {

Handle the http request by static class “FunctionHandler”

Event event = new Event(requestBody, eventId, eventType, eventTime, eventNamespace);
Context context = new Context(methodName, timeout, runtime, memoryLimit);

Object returnValue = Handler.method.invoke(Handler.obj, event, context);
String response = (String)returnValue;
logger.info("Response: " + response);
he.sendResponseHeaders(200, response.length());

Go Runtime

Modify the template to inject function name by “compile-function.sh”

#!/bin/bash

set -e

# Replace FUNCTION placeholder
sed "s/<<FUNCTION>>/${KUBELESS_FUNC_NAME}/g" $GOPATH/src/controller/kubeless.tpl.go > $GOPATH/src/controller/kubeless.go
# Remove vendored version of kubeless if exists
rm -rf $GOPATH/src/kubeless/vendor/github.com/kubeless/kubeless
# Build command
go build -o $KUBELESS_INSTALL_VOLUME/server $GOPATH/src/controller/kubeless.go > /dev/termination-log 2>&1

Implement web server in ProxyUtils by “net/http”

func main() {
	http.HandleFunc("/", handler)
	http.HandleFunc("/healthz", health)
	http.Handle("/metrics", promhttp.Handler())
	proxyUtils.ListenAndServe()
}

Handle http request by function chain “handler” -> “ProxyUtils” -> “handle”

func handle(ctx context.Context, w http.ResponseWriter, r *http.Request) ([]byte, error) {
	data, err := ioutil.ReadAll(r.Body)
	...
  ...
	res, err := kubeless.<<FUNCTION>>(event, funcContext)
	return []byte(res), err
}

func handler(w http.ResponseWriter, r *http.Request) {
	...
  ...
	proxyUtils.Handler(w, r, handle)
}

Refer to:

Scaffold in Runtime
Python Runtime
Ruby Runtime
Node JS Runtime
Java Runtime
Go Runtime
Refer to: