Google Reader

After seeing several people mention it, I decided to give Google Reader a try. I like it. In fact, I’ve pretty much already switched to it for my day-to-day RSS needs. One reason has nothing to do with the quality of the software, but to do with the deployment model: I like being able to switch machines and get the same view of the same feeds including which articles I have or have not read. I used four machines yesterday, which is not that unusual for me, and every time I switched I could just pick up where I left off. The software itself is fairly nice, allowing me to read the new articles quickly and click through to links that deserve further study (something Thunderbird rather amazingly just can’t seem to get right). The left pane tends to cut off titles a little earlier than I like and doesn’t seem to be resizeable, but other than that it’s a very nice piece of work.

Text Logging

Public service announcement: don’t do it when performance matters. Even for in-memory logs, it’s just too slow. Most text logging involves not one but two string copies – one into a temporary formatting buffer for sprintf or something similar, and then once from there into the log itself. If you’re clever you can reduce this to one copy, but most people who do it don’t even think of that refinement and even a byte-at-a-time copy is still something you’d probably want to avoid in the performance path of code that’s used for thousands of operations per second. It’s not nice to waste memory containing essentially the same string over and over again either. If you want to do some sort of logging that can be used in code that needs to be fast, store it in the log itself in binary and translate to text only when the log is dumped. It’s also a good idea to make your logging levels dynamically changeable, and to have multiple levels instead of all or nothing. Logging properly is an essential part of writing maintainable code, and it never ceases to amaze me how many programmers get it horribly wrong.

Java Memory Allocation

Brian Goetz at IBM has posted an article about how Java memory allocation isn’t so bad after all. He makes a few good points about how garbage collection isn’t as bad as many seem to think, and how Java’s memory allocation has gotten a lot better than it used to be, though when he gets to discussing allocation performance for C/C++ he cites papers from before Java even existed as though technology elsewhere cannot also have improved since then. Then he goes seriously off the rails with this piece:

Worse, a cache miss when allocating an object on the heap has a particularly nasty memory interaction. When allocating memory from the heap, the contents of that memory are garbage — whatever bits happen to be left over from the last time that memory was used. If you allocate a block of memory on the heap that is not already in the cache, execution will stall while the contents of that memory are brought into the cache. Then, you will immediately overwrite those values that you paid to bring into the cache with zeros or other initial values, resulting in a lot of wasted memory activity.

For someone who never saw fit to mention cache issues when discussing the merits of copying generational garbage collectors, he seems to have taken a very sudden interest in them now. Treating a concern as serious on one side of an issue while ignoring it on the other is one of the most reliable indicators that an author is biased. Even worse, he’s utterly wrong. No processor will automatically try to load the contents of a newly-allocated block of memory into cache; it wouldn’t even know where the block ends. The processor will only load specific cache lines as they are requested, and not at all if they’re written first as in the scenario Goetz describes. The wasted read activity he mentions is pure fiction to support his agenda.

How to Add Locking to a Program

Proper synchronization is a necessity for writing multi-threaded programs. Many people think “proper synchronization” means locking, but that’s not quite true. As I and others have pointed out many times, it’s often better to solve synchronization problems in other ways, such as by using algorithms and protocols that reduce the need for locking. However, in the real world programmers are often faced with the problem of adding synchronization to a program that currently lacks it, most often because the program wasn’t originally designed with concurrency in mind. In these cases, changing algorithms or protocols would often be far too invasive, so adding locks around important data structures really is the way to go. Here are some tips for how to do it safely.

Teaching Intelligent Design

I happen to be among those who think “Intelligent Design” is just a load of creationist junk dressed up to look like something that resembles science, and that the people who insist on “teaching the controversy” when there is no controversy among actual scientists regarding ID are engaging in a form of circularity. Nonetheless, I wonder: why not teach about ID in schools? Yes, let’s teach students about the controversy – the one between scientists and anti-scientists. Let’s teach them about evolution, and about its flaws and controversies. Then let’s teach them about ID, and about its flaws and controversies as well. It’s not like ID will come out of that comparison looking good. Let’s use the two as an example of how to distinguish real science from garbage, and of the sleazy tactics that the garbage purveyors use to sound scientific without actually being scientific. Let’s make lemonade out of this lemon, and at least prepare students so they’ll be able to recognize and reject the next piece of claptrap that comes along. Teaching about the scientific method is more important than teaching the results anyway, since you can get from the former to the latter much more easily than vice versa, and what better way to teach that method than to compare and contrast with the “method” behind ID?

Perverse Incentives

A couple of news stories this morning, both coincidentally having to do with Massachusetts governor Mitt Romney, got me thinking about perverse incentives. Because incentives are at the heart of capitalist/free-market economic theory, and often of my disagreement with laissez-faire dogma, perverse incentives are something I think about often. In fact, I just noticed that two of my drafts here have to do with different variants of the same topic. I’ll get to those below; first, here are the two items about Romney.

The first news item is that Romney wants to reduce the state income tax, in accordance with a 2000 referendum. At many levels it’s The Right Thing To Do. However, it made me think about the general politics of tax cuts and increases. One of the most effective moves a politician (usually an executive) can make is to push for a tax cut that they know cannot be made permanent. They get credit for lowering taxes and their opponents (usually in the legislature) get blame for raising them subsequently, which is a major win even though the practical net effect of the whole process is nil. The same approach can be applied to any other issue where one can create a perception of pushing for a particular outcome even if there’s no actual hope of it being achieved. The perverse incentive here is to waste everyone’s time going back and forth on an issue, coming back to the same point with no change except the political.

The other news item is Romney’s education-reform plan, which includes substantial “merit pay” for teachers whose students do well on standardized tests. Unfortunately, many implementations of teacher merit pay account only for output and not input, creating a perverse incentive for teachers to seek assignments with the best students. The more difficult students, who one could argue need the most help, are not profitable, and so they’re practically abandoned – similarly to what already happens under Bush’s No Profitable Child Left Behind act, but more so. If merit pay is based only on outcomes, and not on outcomes relative to incoming student ability, then it tends to have no or negative effect on overall student performance. Teachers need an incentive to tackle hard problems, not rewards for sticking to the easy ones.

Linus on Specifications

Yet again, Linus Torvalds seems determined to do more harm than good to the world of programming. Here’s why he hates specs.

So there’s two MAJOR reasons to avoid specs:

- they’re dangerously wrong. Reality is different, and anybody who thinks specs matter over reality should get out of kernel programming NOW. When reality and specs clash, the spec has zero meaning. Zilch. Nada. None.

I know a lot of people who’ve been doing kernel programming longer than Linus (I’m one) who would take this as evidence that he should get out of the specialty immediately. His “when reality and specs clash” comment is almost true when you’re talking about interoperability with devices or APIs between companies, but not quite. There are still vendors who will respond reasonably if you point out how their product violates a spec. I know because I’ve been on both sides of that kind of interaction, in the specific domain of storage protocols which spawned this email thread.

When the clash between reality and specs occurs within the context of code where you control both ends, Linus is almost totally wrong. Reality can be changed, and often should be changed if it makes a program more efficient or more extensible or more maintainable etc. Programming is all about changing the reality of version N into a new reality of version N+1. A spec is no more than a statement of what the new reality should look like. It can also serve as a starting point for a dialog about that new reality, before too much of everyone’s time is wasted by incompatible unilateral changes to the code. Yes, a spec for version N+1 might become outdated when people start to think about version N+2, but that’s no different than what happens to code. The fact that reality tends to keep moving can be managed in either case. Demanding impossible perfection of specs and presenting far-from-perfect comments in code as an alternative is shoddy both in theory and in practice.

specs have an inevitably tendency to try to introduce abstractions levels and wording and documentation policies that make sense for a written spec. Trying to implement actual code off the spec leads to the code looking and working like CRAP.

Again, there’s a small nugget of truth buried in a big pile of Linus BS. Some people, especially some of the extreme UML weenies like those (at Rational) who used to occupy the building I’m in now, seem to believe in “executable specifications” and such. I think they’re full of it, and Linus is right to say so. However, Linus then falls prey to an argument from ignorance: he doesn’t know how to write a useful spec, therefore he concludes it’s impossible. I’ll put that another way: Linus is incompetent at a core software-engineering task. This kind of non-reasoning is common among people who think they’re the smartest person in the whole world, and particularly among those who have spent too much time basking in the adulation of ignorant fan-children who wouldn’t know good code if it got into a robot and literally bit them on the ass. It also reminds me of the people who claim that government is inevitably inefficient and corrupt only because government run by them is always that way and they can’t imagine how anyone else could be better.

In fact, there are many good specs out there. Many of the old network-protocol specs out of DEC were excellent, not just in the abstract but in a very concrete way that made them useful to people actually trying to implement interoperable hardware and software. The specs for SCSI and its descendants such as Fibre Channel, by contrast, are pretty awful. They were packed full of flags and features that were almost universally ignored or unused except by the one standards-committee member who demanded them, while they simultaneously left way too much slack in important areas such as error reporting – when errors are reported, to whom, when they’re cleared, what the codes mean, etc. I think it’s reasonable to say that protocol/API specs designed by committees generally do suck. However, design specs written by a single person are a whole different matter. Any decent software engineer should be able to write a spec that explains how something works, well enough to help the next person who works on it and far better than comments in the code itself (not an area where Linux shines BTW) could ever do. Linux itself has often been hurt by the lack of specs. I still remember when the VFS Layer from Mars was introduced, and there were tons of bugs that resulted from nobody understanding the new interfaces and particularly the new locking schema (which kept changing anyway). That somebody who lived through that could still recommend winging it just boggles the mind.

What’s more disappointing than Linus’s behavior (which is no less than I’ve come to expect) is that Theodore Ts’o – someone I’ve met and who generally seems much more reasonable – chose to back him up).

This is one of the reasons why the IETF doesn’t bless a networking standard until there are multiple independent, interoperable implementations — and even _then_ there will be edge cases that won’t be caught until much, much later.

…and yet, the IETF does use specs, and we wouldn’t be having this conversation without them. That such a thing should be true is just more proof that, contrary to what Linus and Ted claim, writing good and useful specs is possible for people who know what they’re doing. Yes, there are plenty of bad specs out there, but pointing only to them is dishonest. Some specs are over-layered and introduce too many unwarranted abstractions (I-T-L-Q nexi, anyone?) but that doesn’t mean all specs are bad. There are also good specs. A responsible engineer who actually has an interest in quality and productivity instead of ducking responsibility should learn the difference, not just refuse to do what they personally find difficult.