I’ve been thinking about event-based vs. multithreaded programs again. Some thoughts from last time are here in case you didn’t know this is a recurring thing for me. My general attitude is that neither model is inherently “better” than the other; it all depends on what you’re doing. Today’s thought is that a key factor is which approach does the best job of avoiding context switches in your situation. Consider a classic server-type application that spends its life responding to a series of independent requests. There are several common models:

  • In the “asymmetric multithreading” model, there are listener threads and worker threads, which communicate through one or more request queues. Listeners are bound to communication channels, each one repeatedly reading requests and placing them on request queues; workers are bound to request queues, each one repeatedly dequeuing and servicing requests. Each request therefore involves exactly one context switch from a listener to a worker, and possibly more if the workers contend with one another for resources.
  • In the “symmetric multithreading” model, there is no distinction between listeners and workers. Each thread loops through the following steps:
    1. Take an incoming-request lock.
    2. Read one request.
    3. Make sure there’s another thread to listen for the next request.
    4. Release the lock.
    5. Service the request.

    This approach has the same context-switch characteristics as the previous one, but represents an evolutionary step toward the next approach.

  • “Threading on demand” is almost identical to symmetric multithreading, except that steps 3 and 4 are deferred whenever possible. A thread X goes on its merry way without worrying about whether there’s anyone left to read the next request. It’s only when X needs to block that it lets someone else take over reading the next request; if X never blocks, no thread creation or switching occurs. This approach avoids most of the context switches associated with other multithreading approaches, without sacrificing the ability to block when needed and pick up transparently where one left off.
  • In the event-based model, there is only one thread, which repeatedly fetches an event – not a request – from a queue and then processes it. Processing of a single request might involve multiple events. In situations where a multithreaded server might block, the event-based server manually saves state and then goes on to process the next event; when the blockage is resolved that will be dealt with in the handler for the unblocking event.

The event-based model involves no contention and no context switches, for the very simple reason that there are no contexts to contend or switch. There is one fly in the ointment, though: how do events get on the event queue? In a lot of cases, people start with an event-based model and then add per-socket listener threads which put events in the queue when requests arrive. What they end up with is not really event-based at all, but asymmetric multithreading in disguise. Sometimes they don’t even realize it, because the listener threads are hidden in someone else’s library. A similar trap involves adaptation libraries that use extra threads to convert synchronous I/O to asynchronous. No matter how it happens, the result is more context switches and lower performance than if they had used symmetric multithreading to start with.

True event-based programming is possible in some environments where true asynchronous I/O facilities (callbacks, not poll/select or O_NONBLOCK) are available, and in those environments event-based models can be a big win. This is commonly the case in OS kernels, embedded environments, or (oddly) Windows NT but not in most flavors of UNIX. In environments lacking true asynchronous I/O, event-based programming almost invariably degenerates into asymmetric multithreading at some level, and shares all of that model’s performance disadvantages. Do you know which model you’re really using?