Over the holidays I had planned to work on a FUSE interface to Cassandra. Yeah, it’s a silly idea. I’m not doing it because it’s useful. Mostly I’m just doing it because I can. I like to play with code even when I’m not working, so even though this involves two work-related technologies I consider it a form of leisure. As it turns out, I didn’t get much of a chance to work on it. I always thought vacations were supposed to be voluntary and either restful or enjoyable, but when the timing is dictated by my employer and most of the the time is spent enabling someone else’s rest or enjoyment then I think a different term is necessary. I was able to squeeze three or four evenings’ worth of coding around my second job, though, and I don’t know when I’ll be able to get back to it, so this is a status update of sorts. Here’s what I have so far.

  • Data structures and key-naming conventions – roughly equivalent to the on-disk format of a disk-based filesystem.
  • Code to manipulate those structures in several important ways, including inode and block allocation.
  • Code to create/mount a filesystem and create/list arbitrarily nested subdirectories.
  • Code to create and read/write string-valued “files” within those subsubdirectories (including rewrite).

That’s really not much, but what’s probably more important than the current functionality is the structure that holds it all together. If I’d set out to implement FTP-like get/put on whole files I would have done that within a much simpler structure, but I just don’t consider that functionality interesting. I very consciously took the slower route of implementing things the way they’ll need to be for FUSE, and that integration is the obvious next step. I should be able to knock out mount, lookup, mkdir, and create/open pretty quickly at this point. I consider incremental read/write of only the affected portions within an arbitrarily large file (as opposed to reading or writing the whole thing) to be the most important feature of this whole project, and I’ve structured things so that full read/write support shouldn’t be difficult – though it’s necessarily a bit tedious. After implementing a few more calls (e.g. stat/fstat, opendir/readdir, maybe even symlink operations) the result might even be useful to someone besides myself. Then there are a lot of other things I could do…

  • I’d like to port the same functionality to other stores such as Voldemort, Tokyo/LightCloud, and/or Hail. Nothing so far particularly precludes that.
  • I probably won’t optimize around Cassandra’s multi-column data model, because that’s largely at odds with porting to other stores. Yes, I could implement yet another layer of abstraction between FUSE and the “block” level so that Cassandra could do certain things using columns and simpler stores could do them using similarly named keys, but it just wouldn’t be any fun. If anybody wants to pay me then my attitude might change, but as long as it’s a leisure-time project this seems unlikely.
  • I do intend to fix certain inefficiencies in how my own code works right now. For example, inode and block allocation hit the “superblock” key way too often. I have a very specific plan for how to do that better, but haven’t bothered to implement it. Similarly, file and directory creation both involve rewriting the entire parent directory and that’s nasty. Incremental directory updates are similar to incremental data updates, so once I have those done I’ll adapt the code.
  • I don’t intend to fix inefficiencies in how Cassandra works right now. The Thrift interface is ludicrously string-centric, forcing all kinds of copies and transformations that really shouldn’t be necessary, but fixing that would require a whole new bunch of work that I wouldn’t enjoy. See previous comment about for-pay work vs. leisure.
  • I do intend to fix some of my own general sloppiness – unchecked return values, probably memory leaks, general lack of modularity in some places.
  • I do not intend to implement any kind of multi-machine or multi-user support. That’s the kind of stuff I do for my day job; unless you want to offer me a new day job (for a lot of money) it’s both too much work and too much conflict of interest. That’s absolutely positively off the table as long as this is a hobby project.

I don’t quite feel ready to post the code yet, though I might be persuaded. If you think it’s something you might actually be interested in working on, then by all means let me know and I’d be glad to let you have it privately. I just don’t see any point in posting it for every wannabe to pick at when at least of half of it will probably change soon anyway. When I get to the point where I can mount via FUSE and unpack/build/run iozone within the resulting mountpoint, even if it’s slow and ugly, than I’ll probably put it on SourceForge under AGPL unless someone suggests another site/license.