I had been meaning to write about network congestion control for a while anyway, but it seems like now I have an extra reason. George Ou just posted an article called Fixing the unfairness of TCP congestion control, lauding (and somewhat representing) Bob Briscoe’s excellent paper on Flow Rate Fairness: Dismantling a Religion. Because Ou is an ardent opponent of so-called “network neutrality” the response has been predictable; the net-neuts have all gone nuts attacking Ou, while all but ignoring Briscoe. One example, sadly, is Wes Felter.
This is fine, but for the same cost as detecting unfair TCP, ISPs could probably just implement fair queueing.
I think you’re a great guy, Wes, but did you actually read Briscoe’s paper? You cite Nagle’s suggestion of queuing at ingress points, as though it somehow contradicts Briscoe, but that is in fact pretty much what Briscoe ends up recommending. The only difference is in how we define “fair” and Briscoe makes a pretty good case for cost fairness instead of flow-rate fairness. He points out, for example, how current techniques merely create an “arms race” between service providers and the application developers, creating new problems as the race continues but never really solving the old one. He even takes aim at some of the tricks that providers like Comcast have been using.
While everyone prevaricates, novel p2p applications have started to thoroughly exploit this architectural vacuum with no guilt or shame, by just running more flows for longer. Application developers assume, and they have been led to assume, that fairness is dealt with by TCP at the transport layer. In response some ISPs are deploying kludges like volume caps or throttling specific applications using deep packet inspection. Innocent experimental probing has turned into an arms race. The p2p community’s early concern for the good of the Internet is being set aside, aided and abetted by commercial concerns, in pursuit of a more pressing battle against the ISPs that are fighting back. Bystanders sharing the same capacity are suffering heavy collateral damage.
That sounds like a pretty serious indictment of forging RST packets and so on. What more could you want? The simple fact is that congestion control is necessary, it will always be necessary, and current methods aren’t doing a very good job. This issue is almost orthogonal to network neutrality, as it’s about responses to actual behavior and not preemptive special treatment based on source or destination before any behavior is manifested, so don’t let opposition to Ou or to his views on network neutrality color your evaluation.
Gaming-resistance is just as desirable a property in congestion control as it is in other areas we’ve both studied, and right now real users are seeing degraded performance because a few developers are gaming this system. I remember talking to Bram Cohen about this issue while BitTorrent was being developed, for example. He was very well aware that congestion control would sometimes slow his transfer rates and very deliberately designed BitTorrent to circumvent it. That benefits BitTorrent users, and I just used BitTorrent to download an ISO yesterday so I’m not unaware of its value, but really, what makes BitTorrent users so special? Why should those who create congestion not feel its effects first and most? How, exactly, is it “fair” to anyone else that they don’t?
I generally agree with Briscoe’s argument, but his proposal of changing TCP has the major downside of requiring changes to Internet hosts. I think Briscoe’s re-ECN and Nagle’s customer fair queuing in edge routers would have virtually the same effect, so the question in my mind is which is cheaper, easier to implement, and less prone to gaming. Fair queuing can’t be gamed and aligns the cost of implementation with the benefits (i.e. with ISPs).
“Fair queuing” sounds great, I know, but it’s only “fair” according to a definition that I think Briscoe does a good job of demolishing. He even discusses at some length the ways in which FRFQ (Flow Rate Fair Queuing – a more accurate name) can be and often is subverted. I’ll write more about the technological details at a later date, but one of my conclusions after having studied this for a while is that if you want to allocate the costs of congestion fairly (i.e. cost fairness, though I hadn’t heard the term yet) then you have to be aware of how senders are responding to congestion feedback. If they’re responding by doing things that reduce congestion, well and good. If they’re not, or if they’re actively trying to be even more piggish than before, then it falls to the ingress router to take actions that will have the same effect. There’s an automatic incentive for the operator of the ingress router (the ISP) to reduce congestion, and there should also be an incentive for senders to avoid having their ingress router take those actions. They do that by playing along properly with the congestion-control methods in use, so I don’t think the “requires changes to hosts” argument is very compelling. They’ll make those changes if it’s in their own interest.
Now, does Briscoe’s specific proposal achieve the goal of providing the proper incentives in the proper places? I don’t think so, and in fact I kind of doubt it. What his paper does, though, is establish the right goal based on the right method of counting (cost fairness rather than flow-rate fairness) using approximately the right general method (feedback to the ingress point about congestion events). That’s more of a step forward than “just implement fair queuing” which I regard as no step at all.
Both Nagle and I are promoting *customer* fair queuing which isn’t based on flows, and it looks to me like it solves the basic problem of large unfairness in the last mile. Briscoe has some good arguments for why cost fairness is even better than the min-max customer rate fairness achieved by customer fair queuing, but re-ECN looks like a pretty complex way to get there.
I’m still cogitating about my second congestion-control article, but as I re-read the above I saw something that I think could stand earlier clarification. “Customer fair queuing” as you and the always-execrable Bennett seem to be proposing is still based on equalizing the wrong thing: rate of packets accepted rather than rate of packets dropped (or congestion notifications sent). It’s still “flow rate” fairness as distinguished from cost fairness in Briscoe’s terms, even if it’s based on hosts or “economic entities” rather than flows as that term is used in a slightly different context. I believe many of the arguments against flow-rate fairness apply to “customer fairness” as well.
Since “fair” is being used as much for its emotive appeal as for its accuracy, maybe I’ll call my favorite solution Free Market Congestion Control or Anti-Terrorist Congestion Control in the same spirit of implying that alternatives lack an important property. It would be about as useful.