"Scale Out" Applies to Interfaces, Too

Wed 03 December 2014

tags: design

Because of what I do for $dayjob, I hear a lot about "scale out" vs. "scale up" in various contexts. Also because of what I do for $dayjob, I get to read a lot of code. Some of it's new and clean. Some of it's . . . not. That's only partly a reflection on the skill of the programmers involved. Part of it is just the fact that all code tends to accumulate technical debt over time. Layering violations, "privacy" violations, and mutual dependencies all chip away at modularity. Short parameter lists turn into long ones, reflecting every new feature added since the code was properly refactored. (Really, when was the last time you saw a parameter list get shorter?) Types, fields, and flags proliferate. Cats and dogs start living together. It's chaos, I tell you!

Something similar also tends to happen with public APIs. They start simply enough, then they grow and grow and grow. Something like this, if I may mix my movie metaphors.


As it turns out, there are two ways that an interface can increase in complexity. Yep, you guessed it: scale up or scale out. A "scale up" interface is one that gets monolithically bigger - you can't use any part of it without having to deal with significant complexity. Doing even the simplest thing requires several calls. OpenSSL provides a great example: set up a method table, create three types of objects, tie two of those together, set up cipher lists and certificate chains, and more, all before you can even start to do regular socket stuff (which is non-trivial already). It's tedious, it's error-prone, and just about everybody who has to use OpenSSL ends up wrapping all of that crap into their own function or object with a much simpler interface. (BTW, the code that inspired this post had nothing to do with OpenSSL.)

By contrast, a "scale out" interface is one that gets bigger in a modular way. Maybe it just has a lot of functions, but using any one of those is simple and straightforward. In some cases, those functions might be grouped according to the objects they operate upon or the functionality they provide, but if you don't use a particular subset then you don't have to set up for it. Defaults are applied intelligently, so that simple calls yield obvious results but more sophisticated usage is also possible. Secondary objects are automatically created using defaults, so the user has to go through fewer steps. Hooks and callbacks are provided to customize behavior further, but remain entirely optional. In all of these cases, the goal is either to reduce the knowledge needed by basic users, or reduce the number of users who need non-basic knowledge. In other words, you want to minimize the area under this curve.

usage type vs.

A "scale out" interface can be just as complex as a "scale up" interface. It can have just as many calls, require just as much code and tests and documentation. However, it grows more gracefully. Exposing your guts to every caller, whether or not they really want to see those guts, is what creates all of that bad coupling and technical debt. If a caller never had to know about a particular interface element (e.g. a function) to get their job done, neither you nor they will have to worry about compatibility when it changes. That reduces complexity and breakage on both sides. There's also less need (or temptation) to "reach in" and muck with stuff that is (or should be) internal, so the level of debt-inducing inflexibility is further reduced. Defining a scale-out interface might be a bit more difficult, but it pays off in the long run.

Comments for this blog entry