In the past months I’ve been working on vali, a C library for Varlink. Today I’m publishing the first vali release! I’d like to explain how to use it for readers who aren’t especially familiar with Varlink, and describe some interesting API design decisions.

What is Varlink anyways?

Varlink is a very simple Remote Procedure Call (RPC) protocol. Clients can call methods exposed by services (ie, servers). To call a method, a client sends a JSON object with its name and parameters over a Unix socket. To reply to a call, a service sends a JSON object with response parameters. That’s it.

Here’s an example request with a bar parameter containing an integer:

{
	"method": "org.example.service.Foo",
	"parameters": {
		"bar": 42
	}
}

And here’s an example response with a baz parameter containing a list of strings:

{
	"parameters": {
		"baz": ["hello", "world"]
	}
}

Varlink also supports calls with no reply or with multiple replies, but let’s leave this out of the picture for simplicity’s sake.

Varlink services can describe the methods they implement with an interface definition file.

method Foo(bar: int) -> (baz: []string)

Coming from the Wayland world, I love generating code from specification files. This removes all of the manual encoding/decoding boilerplate and is more type-safe. Unfortunately the official libvarlink library doesn’t support code generation (and is not actively maintained anymore), so I’ve decided to write my own. vali is the result!

vali without code generation

To better understand the benefits of code generation and vali design decisions, let’s take a minute to have a look at what usage without code generation looks like.

A client first needs to connect via vali_client_connect_unix(), then call vali_client_call() with a JSON object containing input parameters. It’ll get back a JSON object containing output parameters, which needs to be parsed.

struct vali_client *client = vali_client_connect_unix("/run/org.example.service");
if (client == NULL) {
	fprintf(stderr, "Failed to connect to service\n");
	exit(1);
}

struct json_object *in = json_object_new_object();
json_object_object_add(in, "bar", json_object_new_int(42));
struct json_object *out = NULL;
if (!vali_client_call(client, "org.example.service.Foo", in, &out, NULL)) {
	fprintf(stderr, "Foo request failed\n");
	exit(1);
}

struct json_object *baz = json_object_object_get(out, "baz");
for (size_t i = 0; i < json_object_array_length(baz); i++) {
	struct json_object *item = json_object_array_get_idx(baz, i);
	printf("%s\n", json_object_get_string(item));
}

This is a fair amount of boilerplate. In case of a type mismatch, the client will silently print nothing, which isn’t ideal.

The last parameter of vali_client_call() is an optional struct vali_error *: if set to a non-NULL pointer and the service replies with an error, the struct is populated, otherwise it’s zero’ed out:

struct vali_error err;
if (!vali_client_call(client, "org.example.service.Foo", in, &out, &err)) {
	if (err.name != NULL) {
		fprintf(stderr, "Foo request failed: %s\n", err.name);
	} else {
		fprintf(stderr, "Foo request failed: internal error\n");
	}
	vali_error_finish(&err);
	exit(1);
}

How does the service side look like? A service first calls vali_service_create() to initialize a fresh service, defines a callback to be invoked when a Varlink call is performed by a client via vali_service_set_call_handler(), and sets up a Unix socket via vali_service_listen_unix(). Let’s demonstrate how a service accesses a shared state by printing the number of calls done so far when the callback is invoked. The callback needs to end the call with vali_service_call_close_with_reply().

void handle_call(struct vali_service_call *call, void *user_data) {
	int *call_count_ptr = user_data;
	(*call_count_ptr)++;
	printf("Received %d-th client call\n", *call_count_ptr);

	struct json_object *baz = json_object_new_array();
	json_object_array_add(baz, json_object_new_string("hello"));
	json_object_array_add(baz, json_object_new_string("world"));
	struct json_object *params = json_object_new_object();
	json_object_object_add(params, "baz", baz);
	vali_service_call_close_with_reply(call, params);
}

int main(int argc, void *argv[]) {
	int call_count = 0;
	struct vali_service_call_handler handler = {
		.func = handle_call,
		.user_data = &call_count,
	};

	struct vali_service *service = vali_service_create();
	vali_service_set_call_handler(service, &handler);
	vali_service_listen_unix(service, "/run/org.example.ftl");
	while (vali_service_dispatch(service));
	return 0;
}

In a prior iteration of the API, the callback would return the reply JSON object. This got changed to vali_service_call_close_with_reply() so that services can handle a call asynchronously. If a service needs some time to reply (e.g. because it needs to send data over a network, or perform a long computation), it can give back control to its event loop so that other clients are not blocked, and later call vali_service_call_close_with_reply() from another callback.

Why bundle the callback and the user data pointer together in a struct, rather than pass them as two separate parameters to vali_service_set_call_handler()? The answer is two-fold:

• Conceptually, the user data pointer is tied to the callback. Other programming languages with support for lambdas just capture variables. Standard C doesn’t have lambdas, and the user data pointer is just a way to pass state to the callback.
• Bundling the callback and the user data pointer together as a single fat pointer unlocks more ergonomic and safer APIs: a function can return a single struct vali_service_call_handler without making the caller manipulate two separate variables to pass it down to vali_service_set_call_handler() (and risk mixing them up in case there are multiple).

This design makes wrapping a handler much easier (to create middlewares and routers, more on that below). This all might sound familiar to folks who’ve written an HTTP server: indeed, struct vali_service_call_handler is inspired from Go’s net/http.Handler.

Client side with code generation

Given the method definition from the article introduction, vali generates the following client function:

struct example_Foo_in {
	int bar;
};

struct example_Foo_out {
	char **baz;
	size_t baz_len;
};

bool example_Foo(struct vali_client *client,
	const struct example_Foo_in *in, struct example_Foo *out,
	struct vali_error *err);

It can be used this way to send the JSON request we’ve seen earlier:

struct vali_client *client = vali_client_connect_unix("/run/org.example.service");
if (client == NULL) {
	fprintf(stderr, "Failed to connect to service\n");
	exit(1);
}

const struct example_Foo_in in = {
	.bar = 42,
};
struct example_Foo_out out;
if (!example_Foo(client, &in, &out, NULL)) {
	fprintf(stderr, "Foo request failed\n");
	exit(1);
}

for (size_t i = 0; i < out.baz_len; i++) {
	printf("%s\n", out.baz[i]);
}

example_Foo_out_finish(&out);

Why does vali generates these twin structs, one for input parameters and the other for output parameters, instead of passing all parameters as function arguments? This does make calls slightly more verbose, but this has a few upsides:

• There is a clear split between input and output parameters, instead of having a variable number of function arguments for each. No need for the API user to remember where input parameters end and when output parameters begin, especially when there are a lot of these.
• On the wire and in the interface definition file, input and output parameters are objects. vali always generates structs for all objects. This is more consistent.
• If a new backwards-compatible version of the interface is published, the newly generated code is also backwards-compatible: old callers will still compile and work fine against the newly generated code. For instance, if a new optional field is added to the input parameters, it will naturally left as NULL by the caller when re-generating the code (because omitted fields are zero-initialized in C).

Service side with code generation

The service side is more complicated because it needs to handle multiple connections concurrently and needs to be asynchronous. Being asynchronous is important to not block other clients when processing a call.

The generator for the service code spits out one struct per method and a function to send a reply (and destroy the call):

struct example_Foo_service_call {
	struct vali_service_call *base;
};

void example_Foo_close_with_reply(struct example_Foo_service_call call,
	const struct example_Foo_out *params);

The per-call struct wrapping the struct vali_service_call * makes functions sending replies strongly tied to a particular call, and provides type safety: a Foo reply cannot be sent to a Bar call.

Additionally, the generator also provides a handler struct with one callback per method, and a function to obtain a generic handler from an interface handler:

struct example_handler {
	void (*Foo)(struct example_Foo_service_call call, const struct example_in *in);
};

struct vali_service_call_handler example_get_call_handler(const struct example_handler *handler);

To use all of these toys, a service implementation can define a handler for the Foo method, then feed the result of example_get_call_handler() to vali_service_set_call_handler():

static void handle_foo(struct example_Foo_service_call call, const struct example_Foo_in *in) {
	printf("Foo called with bar=%d\n", in->bar);

	example_Foo_close_with_reply(call, &(const struct example_Foo_out){
		.baz = (char *[]){ "hello", "world" },
		.baz_len = 2,
	});
}

static const struct example_handler example_handler = {
	.Foo = handle_foo,
};

int main(int argc, void *argv[]) {
	struct vali_service *service = vali_service_create();
	vali_service_set_call_handler(service, example_get_call_handler(&example_handler));
	vali_service_listen_unix(service, "/run/org.example.ftl");
	while (vali_service_dispatch(service));
}

Service registry

Some more elaborated services might want to implement more than a single interface. Additionally, services might want to add support for the org.varlink.service interface, which provides introspection: a client can query metadata about the service (e.g. service name, version) and the definition of each interface.

vali makes this easy thanks to struct vali_registry. A service can initialize a new registry via vali_registry_create(), then register each interface by passing its definition and handler to vali_registry_add(). The generated code exposes the interface definition as an example_interface constant. Finally, the registry can be wired up to the struct vali_service by feeding the result of vali_registry_get_call_handler() to vali_service_set_call_handler().

const struct vali_registry_options registry_options = {
	.vendor = "emersion",
	.product = "example",
	.version = "1.0.0",
	.url = "https://example.org",
};
struct vali_registry *registry = vali_registry_create(&registry_options);
vali_registry_add(registry, &example_interface,
	example_get_call_handler(&example_handler));
vali_registry_add(registry, &another_example_interface,
	another_example_get_call_handler(&another_example_handler));

struct vali_service *service = vali_service_create();
vali_service_set_call_handler(service, vali_registry_get_call_handler(registry));
vali_service_listen_unix(service, "/run/org.example.ftl");

while (vali_service_dispatch(service));

This is where the struct vali_service_call_handler fat pointer really shines: the wire-level struct vali_service and the higher-level registry can stay entirely separate. struct vali_service invokes the registry’s handler, then the registry is responsible for routing the call to the correct interface-specific handler. The registry’s internal state remains hidden away in the handler’s opaque user data pointer.

A complete client and service example is available in vali’s example/ directory.

What’s next?

I plan to leverage vali in the next version of the kanshi Wayland output management daemon.

We’ve discussed about async on the service side above, but we haven’t discussed async on the client side. That can be useful too, especially when a client needs to juggle with multiple sockets, and is still a TODO.

Something I’m still unhappy about is the lack of const fields generated structs. Let’s have a look at the struct for output parameters given above:

struct example_Foo_out {
	char **baz;
	size_t baz_len;
};

If a service has a bunch of const char * variables it wants to send as part of the reply, it needs to cast them to char * or strdup() them. None of these options are great.

static const char hiya[] = "hiya";

static void handle_foo(struct example_Foo_service_call call, const struct example_Foo_in *in) {
	example_Foo_close_with_reply(call, &(const struct example_Foo_out){
		// Type error: implicit cast from "const char *" to "char *"
		.baz = (char *[]){ hiya },
		.baz_len = 1,
	});
}

On the other hand, making all struct fields const would be cumbersome when dynamically constructing nested structs in replies, and would be a bit of a lie when passing a reply to example_Foo_out_finish() (that function frees all fields).

Generating two structs (one const, one not) is not an option since types are shared between client and service, and some types can be referenced from both a call input and another call’s output. Ideally, C would provide a way to propagate const-ness to fields, but that’s not a thing. Oh well, that’s life.

If you need an IPC mechanism for your tool, please consider giving vali a shot! Feel free to reach out to report any bugs, questions or suggestions.