Chains of asynchronous handlers

The problem

Version 0.6.13 introduced the ability to chain synchronous request handlers. It was an attempt to add something like Express.JS middleware to RESTinio. Please read the Chains of synchronous handlers section to learn about RESTinio’s approach.

One of the main problems with sync_chain is that the request handlers must be synchronous. This doesn’t work well when a handler has to perform a long-running operation that may take more than a millisecond.

To solve this problem, version 0.7.0 introduces a chain of asynchronous handlers. This kind of chain is somewhat similar to sync_chain, but there is also a principal difference that we’ll discuss here.

What does “asynchronous handler” mean?

Note. It’s recommended to read the RESTinio’s threading model and sync/async request processing first

When RESTinio reads an incoming HTTP request, it calls the request handler specified in the server settings. This handler blocks the thread where RESTinio read the last HTTP request. If the handler completes processing right here, it means that that it’s a synchronous handler. The synchonous handler blocks one of RESTinio’s threads until the response is prepared and the .done() method is called for the response object.

But if the request handler just sends the request object to a separate thread and then returns control back to the RESTinio, then it is an asynchronous handler. An asynchronous handler blocks a RESTinio’s thread for a small amount of time needed just to delegate the actual request processing to another thread.

The role of request_handling_status_t::accepted in case of an asynchronous handler

The request handler specified in the server settings must return one of the following values:

request_handling_status_t::rejected. It means that this request can’t be handled and RESTinio should generate negative response with “Not Implemented” status. In that case the request object becomes obsolete and should not be used anymore.
request_handling_status_t::accepted. It means that this request is accepted for processing and the actual response will be generated sooner or later. RESTinio does nothing with the request object and assumes that the user takes responsibility for making an appropriate response.

In the case of synchronous handlers the situation is very simple for the RESTinio: if the request handler returns rejected then the request can’t be processed at all, if accepted is returned then the request handler accepts the request. It’s that simple.

But in the case of asynchronous handling, the decision whether to reject or accept the request is made later in the context of the another thread. So if the request handler sends the request object to another thread and returns rejected, then RESTinio will send a negative response back to the client while another thread decides what to do with the request. That is why it’s critically important to return accepted from the request handler, even if the request won’t be actually processed.

It also means that if an incoming request can’t be processed, the asynchronous handler should generate a negative response (with a “Not Implemented” status, for example).

Note. There is also request_handling_status_t::not_handled value, but it’s treated just a synonum for request_handling_status_t::rejected.

How async_chain works?

An async_chain represents itself as an ordinary request handler for RESTinio. When RESTinio receives an incoming request it calls async_chain and passes the request object to it as a parameter.

An async_chain holds a list of functions that can be seen as schedulers. The goal of a scheduler is to pass the request to another thread for the actual processing (this action can be thought of as “scheduling”).

The async_chain calls the first scheduler in the chain and returns request_handling_status_t::accepted to the RESTinio as soon as the first scheduler is finished. It’s assumed that a scheduler just sends the information about the request somewhere else and returns as soon as possible.

The schedulers have a special format:

// For a case when there is no extra data for a request.
restinio::async_chain::schedule_result_t
scheduler( restinio::async_chain::unique_async_handling_controller_t<> controller );

// For a case when there is extra data for a request where
// the extra data is represented by type EDT.
restinio::async_chain::schedule_result_t
scheduler( restinio::async_chain::unique_async_handling_controller_t<EDT> controller );

Where schedule_result_t is an enumeration:

enum class schedule_result_t{ ok, failure };

A scheduler has to return ok value if sending of a request info was successful. The value failure should be returned if the sending failed and furher processing is not possible.

external_lib::msg_queue< restinio::async_chain::unique_async_handling_controller_t<> > queue;

restinio::async_chain::schedule_result_t
my_async_scheduler(
   restinio::async_chain::unique_async_handling_controller_t<> controller )
{
   try {
      // Sending the request for actual processing.
      queue.push( std::move(controller) );

      return restinio::async_chain::ok();
   }
   catch(...) {
      // Sending fails, the request can't be processed.
      return restinio::async_chain::failure();
   }
}

A controller passed to a scheduler is a special object that holds information about the source request and the current position in the chain.

It’s important to note that the controller is being passed to schedulers as a unique value (aka std::unique_ptr). This means that the shared ownership of the controller is prohibited: only one entity can own the controller at any given time.

To process the request, an actual request handler has to receive the controller object:

external_lib::msg_queue< restinio::async_chain::unique_async_handling_controller_t<> > queue;

restinio::async_chain::schedule_result_t
my_async_scheduler(
   restinio::async_chain::unique_async_handling_controller_t<> controller )
{ ... /* Scheduler pushes the controller to queue */ }

void actual_processing_thread()
{
   while(queue.is_open()) {
      // Extracts a new request.
      queue.wait_if_empty();
      auto controller = std::move(queue.top());
      queue.pop();

      // Process the request.
      const auto req = controller->request_handle(); // Source request.
      ...
      // If request is not processed then the next scheduler in the chain
      // has to be activated...
      next(std::move(controller));
   }
}

The most important moment in the handler code is the call to the next function. This function is defined in restinio::async_chain namespace and it is automatically selected by the compiler due to the argument-dependent lookup. This function checks if there is a next scheduler in the chain: if there is, it is called, if not, then the request processing will be cancelled (by generating a negative response with status “Not Implemented”).

The controller object passed to a scheduler or a handler can be seen as a token that enables request processing: if an entity holds this token, it can perform processing (calling request_handle and next). After passing the token away the entity can’t do anything with the request.

Where schedulers are called?

One of the key points is the working context where a scheduler in a chain is being called.

The call to a next scheduler is made in the content of the thread in that the next is invoked. It’s simply because the scheduler is called inside the next function.

Let’s imagine that RESTinio is run on single thread and there are two separate threads for request processing – CheckerThread and HandlerThread. There are also two schedulers in an async_chain: one sends a request to the CheckerThread and another sends the request to the HandlerThread.

In that case we’ll have something like this:

RESTinio thread            CheckerThread             HandlerThread
===============            =============             =============
      |                          |                         |
 invocation of the               |                         |
 first scheduler                 |                         |
 in chain ---------------------->|                         |
                                 |                         |
                           some processing of              |
                           the incoming request;           |
                           call to next(controller);       |
                           invocation of the second        |
                           scheduler in chain ------------>|
                                                           |
                                                     some processing of
                                                     the incoming result

Thread safety for a controller

A controller (an instance of unique_async_handling_controller_t from the restinio::async_chain namespace) is not thread-safe object.

It’s not a problem because only one thread may work with a controller at any given time.

Ready-to-use implementations of chains

RESTinio contains several ready-to-use implementations of chains of handlers. This section briefly describes them.

Please note that the corresponding header files with the implementation of those chains are not included in restinio/core.hpp. Those headers should be included in your code manually.

fixed_size_chain

#include <restinio/async_chain/fixed_size.hpp>

The template class restinio::async_chain::fixed_size_chain_t is an implementation of a chain of handlers of fixed size and this size is known at the compile time:

template<
   std::size_t Size,
   typename Extra_Data_Factory = restinio::no_extra_data_factory_t >
class fixed_size_chain_t;

Usage example for the case when no extra-data is incorporated into a request object:

struct my_traits : public restinio::default_traits_t {
   using request_handler_t = restinio::async_chain::fixed_size_chain_t<3>;
};

restinio::run(restinio::on_this_thread<my_traits>()
   .port(...)
   .address(...)
   // Exactly 3 handlers should be passed to request_handler().
   .request_handler(
      first_handler,
      second_handler,
      third_handler)
   ...);

Note that an instance of fixed_size_chain_t<N> accepts exact N handlers in the constructor and that list of handlers can’t be changed latter.

Usage example for the case when some extra-data is incorporated into a request object:

struct per_request_data {...};

struct my_traits : public restinio::default_traits_t {
   using extra_data_factory_t = restinio::simple_extra_data_factory_t<
         per_request_data>;

   using request_handler_t = restinio::async_chain::fixed_size_chain_t<
         3,
         extra_data_factory_t>;
};

restinio::run(restinio::on_this_thread<my_traits>()
   .port(...)
   .address(...)
   // Exactly 3 handlers should be passed to request_handler().
   .request_handler(
      first_handler,
      second_handler,
      third_handler)
   ...);

growable_size_chain

#include <restinio/async_chain/growable_size.hpp>

The template class restinio::async_chain::growable_size_chain_t is an implementation of a chain of handlers of unknown size:

template<
   typename Extra_Data_Factory = restinio::no_extra_data_factory_t >
class growable_size_chain_t
{
public:
   class builder_t;
   ...
};

An instance of restinio::async_chain::growable_size_chain_t can’t be created directly. It should be created and filled up by using special helper growable_size_chain_t::builder_t:

struct my_traits : public restinio::default_traits_t {
   using request_handler_t = restinio::async_chain::growable_size_chain_t<>;
   ...
};

// Make a builder instance.
my_traits::request_handler_t::builder_t chain_builder;

// Pass all necessary handlers to the chain.
if(config.check_headers())
   chain_builder.add(header_checker);
chain_builder.add(authentificate_user);
if(config.log_admin_actions())
   chain_builder.add(log_admin_access);

// Run the server.
restinio::run(restinio::on_this_thread<my_traits>()
   .port(...)
   .address(...)
   .request_handler(
      // Release the instance of growable_size_chain and pass it
      // to the server.
      chain_builder.release())
   ...);

Please note that growable_size_chain_t::builder_t is like std::unique_ptr: it’s a Movable class, but not Copyable.

Also growable_size_chain_t::builder_t is intended for building of just one instance of growable_size_chain_t. After the call to builder_t::release the builder becomes empty and it must not be used without the full reinitialization.

Once a list of handlers is filled up in a builder and release() method is called for the builder, the list can’t be changed anymore.