Real Freedom vs. Illusions of Freedom

Randy Bias has taken it upon himself to explain that vendor lockin is unavoidable. How surprising that someone who works for one of the most aggressively proprietary vendors in the storage space would say such a thing, with a healthy dose of sneering about unicorns and Santa Claus thrown in. (Really, Randy, what part does "making you dumber" play in your claimed mission of helping people?) Since much of his argument consists of casting aspersions on open source, a response is called for, but first, let's summarize what he says.

  1. Switching costs are still non-negligible even with open source, with a whole list of reasons.

  2. We don't have this problem with networking, even though the enterprise-level products there are proprietary, because they're still bound by standards.

  3. Because open source doesn't matter, standards and migration tools are sufficient to avoid the lock-in he says is unavoidable. (Wait, what?)

The first point is actually pretty correct, so let's move on to the second. The problem is that the analogy between storage and networking is invalid because switches and routers don't store your data. As Randy himself has written, data has a gravity or inertia that network flows do not. Migrating from one network vendor to another doesn't involve a weeks-long process of moving years-old data. In the ways that matter - different tooling, different skillsets, everything Randy mentioned in the previous point - the switching costs are there for networking just as much as for storage. Anyone who has ever actually been involved e.g. in replacing Cisco with Juniper would be acutely aware of that. The difference between storage and networking transition costs has nothing to do with proprietary vs. open source and everything to do with the essential nature of the two functions. It tells us nothing about whether open source matters.

The only reason that broken example is even worth addressing is because it's used as a springboard for the giant leap Randy takes next. Open source does matter precisely because of the ways that storage is not like networking. If a piece of networking gear blows up, you can replace it with another and get back to exactly where you were. The effort and expense might not be trivial, but you won't have suffered a permanent loss of the data that is your company's lifeblood. Sure, there are backups, but the very fact that backups exist is evidence of how different storage is from networking. Restoring from backup is an expensive and imperfect process that has no parallel in networking. Also, storage can't just drop writes the way networks can drop packets. Storage has to prepare for these events ahead of time with various forms of redundancy and careful scheduling of operations. How all this is done, and how to use these artifacts to get at the actual user's data despite a failure, can be very complex. That's why you can't just take some disks out of a high-end storage system and expect to make sense of their contents on your own. You might be able to piece together most of what's on those disks, but to get all of it (especially the most recently written parts) you need a thorough understanding of the formats and algorithms that are being used to write it.

That's where open source comes in. I've worked for open-source storage companies and I've worked for proprietary storage companies (including EMC). Nobody - and I do mean nobody - documents enough of their back-end formats and internal operations well enough to put absolutely everything back together under all kinds of failure conditions. The only sufficiently thorough documentation is the source code for the software that actually writes the data. With open source, you at least have a decent chance of getting answers about those kinds of details, or hiring someone who knows and can share their knowledge of those internals, or worst case of figuring out how the software works and developing your own in-house expertise. With proprietary software you have none of that. It works the way it works, and it fails the way it fails, and there's not a whole lot you can do about it except trust that the vendor will be responsive. What was that about Santa Claus, again?

Lastly, what about migration tools? The fact is that any door that can be locked can't be trusted to remain open. Open-source doors can't be locked. Nothing will ever stop you from walking through. Vendor-provided migration or recovery tools, which are inevitably designed to have a limited set of functionality and are tested against a finite set of conditions, might not address the need or condition that you have right now. Maybe they do something close, and you could tweak them to do what you want . . . if you only had the source . . . which you don't, so too bad. Back to the mercy of the vendor. Who knows if or when your enhancement request will be acted upon? Likely never, when - no, I don't mean if - they change their minds and decide that supporting such tools isn't in their strategic interest.

There's no panacea for vendor lock-in. Not even open source, but open source alone gets you further than any number of standards that don't cover what really matters or vendor-provided tools that might go away at any moment. It's the first and best tool for dealing with lock-in, even if it's not perfect. Or, to paraphrase the Man in Black:

Life is pain, Highness. Anyone who says differently is selling something.

Vendors won't take initiative to give you greater control, as Randy suggests. They might take initiative to give you an illusion of control, but illusions are not reality. Keeping that control in your hands requires continual ongoing effort. There doesn't need to be any malice of deceptive intent involved in its subsequent disappearance. Simple laziness or inattention are sufficient. Every company goes through shifts in personnel or strategy. Every such shift is likely to leave something behind. Open source was invented in large part to ensure that the one real collection of information about how things actually work couldn't be left behind even when these inevitable changes occur. Its importance can not be dismissed in favor of some snake oil about magical migration tools that will always be current, always be free, and always cover every need.

Comments for this blog entry