Aug 9, 2011

Scaling with Single Threading

The free lunch is over. To speed up applications we are told we must write multithreaded programs and avoid mutable state. Functional programming can help with it’s immutable state. There’s also Erlang with the Actor model or Clojure with it’s software transactional memory. One other option to consider is single threading your code. Following are a few examples where single threading was used to scale applications.

Java Swing

When Sun built their new GUI toolkit, they choose to single thread all GUI events thru an event dispatch thread. One of the reasons they had for choosing this architecture was low overhead; toolkits that use threads can spend a substantial amount of time and space managing locks.


Node.js is the latest hot technology. Most web and application servers spawn a new thread for each connection. Each of those threads have memory overhead, which limits the number of connections a server can handle. Instead of threads, Node.js uses a single I/O event queue and fires an event handler for each connection. All user code is run in a single thread on the event queue. This limits the memory footprint of the server and minimizes the chances of deadlock. Nginx is a web server built using this architecture and has shown some amazing performance with little overhead.

An additional advantage to node is the use of Javascript and the Google V8 engine. This allows you to write using the same language used on the client. Client side developers will have an easier time writing server code and will be able to share code between client and server.

For those who prefer sticking with Java, there is an attempt to port Node.JS to Java. Another alternative is to build your server using Netty, which an asynchronous event-driven network application framework built using NIO.


LMAX is a retail financial trading platform built using Java. A trading application like this needs to very low latency, there will be a large volume of trades that need to be processed quickly. The LMAX team ended up running all the business logic on a single thread with all of the data in memory. This single thread processes 6 million orders per second using commodity hardware. Disruptors receive messages and unmarshall, replicate and journal the input messages, then pass them along to the single threaded business logic processor. When the business logic processor completes the message, it passes it to an output distruptor that publishes the results. Martin Fowler wrote an excellent article on this architecture. I found the following footnote interesting:

An interesting side-note. While the LMAX team shares much of the current interest in functional programming, they believe that the OO approach provides a better approach for this kind of problem. They’ve noticed that as they work to write faster code, they move away from a functional style towards OO style. Partly this because of the copying of data that functional styles require to maintain immutability. But it’s also because objects provide a better model of a complex domain with a richer choice of data structures.


While a single threaded architecture may not fit every application, it is an important architecture to keep in your tool belt. If you are having trouble scaling your application, it is worth considering these examples.

About the Author

Object Partners profile.

One thought on “Scaling with Single Threading

  1. Ted Naleid says:

    Redis is one of the hottest “NoSQL” techs out there right now and it’s single threaded too. It’s often used as a very fast caching layer by sites like craigslist, github, and stackoverflow.

    The simplicity and lack of overhead (both mental and computational) you can get from single threaded apps often wins over multi-threaded ones.

  2. Josh Diehl says:

    Interesting overview of single-thread scaling, I was only familiar with node.js. Also interesting the opinion about functional programming vs OO, I’ll have to check out Martin Fowler’s article.

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Blog Posts
Feature Flags in Terraform
Feature flagging any code can be useful to developers but many don’t know how to or even that you can do it in Terraform. Some benefits of Feature Flagging your code You can enable different […]
Infrastructure as Code – The Wrong Way
You are probably familiar with the term “infrastructure as code”. It’s a great concept, and it’s gaining steam in the industry. Unfortunately, just as we had a lot to learn about how to write clean […]
Snowflake CI/CD using Jenkins and Schemachange
CI/CD and Management of Data Warehouses can be a serious challenge. In this blog you will learn how to setup CI/CD for Snowflake using Schemachange, Github, and Jenkins. For access to the code check out […]
How to get your pull requests approved more quickly
TL;DR The fewer reviews necessary, the quicker your PR gets approved. Code reviews serve an essential function on any software codebase. Done right, they help ensure correctness, reliability, and maintainability of code. On many teams, […]