original article

The kernel can only ask the hard drive to flush the data to disk. The disk need not comply, despite returning a “yes I did” result.

That’s an important issue. I’ll try to provide a couple of answers.

how can the consumer really know what the drive decides to do?

Well, there are at least two ways:

  1. Turn off write caching.
  2. Set the “Force Unit Access” (FUA) bit on the Write command, if it’s a SCSI/FC disk.

SCSI gives you other options as well. For example, if you’re using tagged command queuing, you can set FUA only on the last command of a sequence (e.g. a transaction). That way, you can allow the disk or storage subsystem to do appropriate reordering, combining, etc. and you’ll still be sure that by the time that last command completes all the commands logically ahead of it (as specified by the tags) have completed as well. It’s tres cool, and it’s one of SCSI’s biggest benefits compared to IDE.

Tagged command queuing also comes in handy if you have to force write caching off – which BTW is common and not particularly difficult on either SCSI or IDE drives. Since you’re now forced to deal with full rotational latency, the importance of overlapping unrelated operations (by putting them on different queues) becomes even greater.

This stuff is not document on the box the hard drive comes in nor on the mfg web site.

Tsk tsk, that’s a shame. It’s pretty common knowledge among storage types, but still far from universal. Go look on comp.arch.storage and you’ll see a recurring pattern of people finding this out for the first time and sparking a brief flurry of posts by asking about it.

The problem with having the drive notify the host that a write has been fully destaged is that target-initiated communication (aside from reconnecting to service an earlier request) is poorly supported even in SCSI. Hell, it’s even hard to talk about it without tripping over the “initiator” (host) vs. “target” (disk) terminology. Most devices lack the capability to make requests in that direction, and most host adapters (not to mention drivers) lack support for receiving them. AEN was the least-implemented feature in SCSI.

There’s also a performance issue. Certainly you don’t want to be generating interrupts by having the disk call back for *every* request, but only for selected requests of particular interest. So now you need to add a flag to the CDB to indicate that a callback is required. You need to go through the whole nasty SCSI standards process to determine where the flag goes, how requests are identified in the callback, etc. Then you need every OS, driver, adapter, controller, etc. to add support for propagating the flag and handling the callback. Ugh.

It’s a great idea, really it is. It’s The Right Way(tm). But it’s just never going to happen in the IDE world, and it’s almost as unlikely in the SCSI/FC world. 1394 seems a little more amenable to this, but I have no idea whether it’s actually done (I doubt it) because even though I know they exist I’ve never actually seen a 1394 drive close up.

I hope all this helps shed some light on the subject.