2.5. Remote Procedure Call
Joedb can be used as a binary message serialization format for remote procedure calls. From a list of functions that take a writable database as parameter, the joedb compiler can generate code for a client and a server that allow executing these functions remotely.
2.5.1. Motivation
Journal replication can be used to implement concurrency over a network connection, but it has drawbacks:
All clients have to download the whole database. It is OK for a small number of clients or a small database, but it is a problem for a large number of concurrently writing clients: the cost of all clients downloading each-other’s writes is quadratic in the number of clients, which does not scale.
When connecting over an unreliable network connection, a write transaction may leave the server locked. The server timeout can get rid of stale locks after a while, but it will halt all processing for the duration of the timeout. This is not acceptable when the server has to remain available.
So journal sharing is a great way to handle backups, caching, or local concurrency, but is not efficient when many remote clients are writing independent parts of the database over an unreliable network connection.
These scalability and reliability issues can be solved by using remote procedure calls. With this mechanism, a disconnection can never leave a stale lock, and there is no need for clients to download the whole database. Each client can upload new data to the server in a very minimal single round-trip. The write transaction will be executed entirely on the server, so its duration is not affected by network latency, and it cannot be stalled by a disconnection.
2.5.2. Tutorial Example
This example implements 4 procedures for the joedb tutorial database:
insert_citytakes a string parameter and inserts it as a city name in the city table.
delete_citytakes a string parameter and deletes the city from the city table.
get_populationtakes a list of cities as parameter, and returns the number of persons for each city.
get_inhabitantstakes a city name as parameter, and returns the list of persons living in this city
First a Service class has to be implemented to run the procedures on the
server:
#ifndef tutorial_rpc_Service_declared
#define tutorial_rpc_Service_declared
#include "tutorial/Client.h"
#include "tutorial/rpc/city/Writable_Database.h"
#include "tutorial/rpc/population/Writable_Database.h"
namespace tutorial::rpc
{
/// A collection of procedures that will be executed in the rpc server
///
/// joedbc uses a regular expression to find procedures in the file.
/// The constructor is not called from compiled code, so it can have any
/// signature. In particular, a Service does not necessarily have to
/// access a database at all.
class Service
{
private:
Client &client;
public:
Service(Client &client): client(client)
{
}
/// Insert a city from a name string
void insert_city(city::Writable_Database &city)
{
client.transaction
(
[&city](Writable_Database &db)
{
const auto city_id = db.find_city_by_name(city.get_name());
if (city_id.is_null())
db.new_city(city.get_name());
else
throw joedb::Exception("city already exists");
}
);
}
/// Delete a city from a name string
void delete_city(city::Writable_Database &city)
{
client.transaction
(
[&city](Writable_Database &db)
{
const auto city_id = db.find_city_by_name(city.get_name());
if (city_id.is_not_null())
db.delete_city(city_id);
else
throw joedb::Exception("city does not exist");
}
);
}
/// A procedure can return values by writing them to the message database
void get_population(population::Writable_Database &population)
{
const auto &db = client.get_database();
for (const auto data: population.get_data_table())
{
const std::string &city_name = population.get_city_name(data);
const id_of_city city = db.find_city_by_name(city_name);
int64_t N = 0;
if (city.is_not_null())
for (const auto person: db.get_person_table())
if (db.get_home(person) == city)
N++;
population.set_city(data, city);
population.set_population(data, N);
}
}
/// A message can have the same schema as the main database
void get_inhabitants(tutorial::Writable_Database &message)
{
const auto message_city = message.get_city_table().first();
if (message_city.is_not_null())
{
const auto &db = client.get_database();
const auto city = db.find_city_by_name(message.get_name(message_city));
if (city.is_not_null())
{
for (const auto person: db.get_person_table())
{
if (db.get_home(person) == city)
{
message.new_person
(
db.get_first_name(person),
db.get_last_name(person),
db.get_home(person)
);
}
}
}
}
}
};
}
#endif
Then a channel can be used to connect a client to the server:
#include "joedb/concurrency/Local_Channel.h"
#include "joedb/ui/main_wrapper.h"
#include "tutorial/rpc/Client.h"
#include "tutorial/rpc/population/print_table.h"
#include "tutorial/print_table.h"
#include <iostream>
namespace joedb
{
static int rpc_client(Arguments &arguments)
{
const std::string_view endpoint_path = arguments.get_next("<endpoint_path>");
if (arguments.missing())
{
arguments.print_help(std::cerr);
return 1;
}
Local_Channel channel((std::string(endpoint_path)));
tutorial::rpc::Client rpc_client(channel);
{
tutorial::rpc::city::Memory_Database city;
city.set_name("Tombouctou");
rpc_client.insert_city(city);
try
{
rpc_client.insert_city(city);
}
catch (std::exception &e)
{
std::cout << "Caught exception: " << e.what() << '\n';
}
rpc_client.delete_city(city);
}
{
tutorial::rpc::population::Memory_Database population;
population.set_city_name(population.new_data(), "Tokyo");
population.set_city_name(population.new_data(), "Tombouctou");
population.set_city_name(population.new_data(), "Lille");
rpc_client.get_population(population);
tutorial::rpc::population::print_data_table(std::cout, population);
}
{
tutorial::Memory_Database db;
db.set_name(db.new_city(), "Lille");
rpc_client.get_inhabitants(db);
tutorial::print_person_table(std::cout, db);
}
return 0;
}
}
int main(int argc, char **argv)
{
return joedb::main_wrapper(joedb::rpc_client, argc, argv);
}
Here are the schema definitions for the procedure messages:
create_table city
add_field city name string
namespace tutorial::rpc::city
set_single_row city true
# "city" is the name of a table in the main schema.
# joedbc will generate code so that id_of_city is the same type for both, ie:
# using id_of_city = tutorial::id_of_city;
# instead of definining a new different population::id_of_city type.
# This allows easily referring to the main schema from procedure schemas.
create_table city
# This is the table for exchaning data between the client and the server.
# It will be used both for input and output.
# The client will fill input fields and push these changes to the server,
# then the server will push output fields to the client.
# A procedure can use any arbitrarily complex schema: multiple tables, etc.
create_table data
# input
add_field data city_name string
# output
add_field data city references city
add_field data population int64
namespace tutorial::rpc::population