RESTinio’s threading model and sync/async request processing

In that section we’ll try to explain how RESTinio utilizes worker threads behind restinio::run and what does sync/async request processing from RESTinio’s point of view.

RESTinio doesn’t use any hidden worker threads

The first and maybe the most important thing we have to explain is that RESTinio uses only those threads that are created inside restinio::run or restinio::run_async functions.

It means that if RESTinio is started on the context of the current worker thread:

restinio::run(restinio::on_this_thread()...);

then the only this worker thread will be used for all RESTinio-related activities (I/O and invocations of request-handlers).

If RESTinio is started on a single worker thread then all I/O operation will be performed on that thread. Any incoming data will be parsed on that thread too. If the whole incoming HTTP request is read then a request-handler for it will be called on that thread too. And the response formed by the request-handler will be written to the corresponding connection on the context of that worker thread.

If RESTinio is started on a thread pool then all threads from that pool do the same things: accept new connections, perform I/O operations, parse incoming data, invoking request-handlers, write outgoing data…

All that means that if user’s request-handler does the processing for long time one of RESTinio’s worker thread will be blocked for that time and can’t do any useful activities (like accepting new connections, reading/writing data, controlling timeouts and so on).

So please note that while your request-handler is working the RESTinio can’t use the worker thread on that this request-handler is invoked. RESTinio doesn’t create any new thread for calling request-handler or for performing I/O ops in the background.

Synchronous or asynchrouns request-processing

RESTinio calls request-handlers synchronously. But the actual model of request processing is up to the user. It can be synchronous (e.g. the response is generated just inside the called request-handler) or asynchronous (e.g. the actual processing is delegated to some other worker context).

Let’s see how RESTinio works if it’s launched on a single thread:

  • RESTinio initiates an event-loop on this worker thread. All I/O operations (accept, read, write) will be performed on this thread only;
  • when RESTinio reads a new incoming HTTP-message (remember, read is performed on worker thread) RESTinio calls the request_handler specified by a user. And that call is performed synchronously. E.g. request_handler is called on the context of the worker thread, and request_handler blocks this thread, no I/O operations can be performed until request_handler returns;
  • request_handler can process the incoming request right here. In that case, request_handler performs all necessary actions (regardless of how long it can take) and then returns request_accepted (or request_rejected) value. It allows RESTinio to know that the incoming message fully processed and RESTinio continues its I/O operations;
  • but request_handler can delegate the processing of the incoming request to another worker thread. In that case request_handler should return request_accepted. RESTinio will continue I/O operations. When another worker thread completed the processing it calls done() for the requests and RESTinio receives the notification. In that case necessary write operations will be scheduled on the working thread where RESTinio works.

RESTinio contains several examples of how synchronous and asynchronous request processing can look like:

A note about response’s flush() and done() methods

Methods flush() and done() of response object do not initiate write operations directly. They just queue those operation using underlying Asio facilities (like asio::post). So when done() is called on the context of a separate worker thread the write operation will be performed only on some of RESTinio’s worker thread.

For example, you can start RESTinio on just one thread and user a pool of a hundred separate worker threads for actual processing. Request-handler callen on single RESTinio thread will delegate processing to one of pool’s threads. When done() (or flush()) is called on a separate worker thread from the pool a write-demand will be posted to the single RESTinio thread and the actual write operation will be performed only on that single RESTinio thread.