Sometimes the internet comes back to haunt you. I’ve learned to live with that, as people have mentioned some of my old blog or Usenet posts in some very unexpected contexts including interviews. I don’t think it has ever cost me a job or otherwise hurt my career, but a couple of times it has been ugly in other ways. Probably the worst case is when some of Robert Stanek’s overzealous fans took exception to my criticism of his relentless astroturf campaign to sell his awful books. I pretty much ignored the legal threats that followed since I was absolutely 100% certain – even if they weren’t just bluffing, which to this day seems to have been the case – they’d be laughed out of court. When somebody started posting my real-life contact information on a couple of sites along with some fabrications about me, though, I had to make a couple of legal threats of my own to protect my family and have the offending content removed. There are some sick people out there.

That was, unfortunately, not the only time my writings have resulted in legal threats. Even earlier, I had written about an “unfortunate” experience I’d had with a certain puzzle website. The people behind it first tried to make excuses for their behavior, and then claimed they were suing me for damaging their internet business. To make a long story short, I called their bluff; I posted more info about them and their legal threats, and pretty explicitly told them to get lost. They did. Three or four times since then, I’ve received email from the main person involved, saying how much this information was hurting his job prospects and such and could I please remove it etc. I’ve ignored them until now, for reasons that I probably can’t explain too much better than I did in my reply.

You have presented me with quite a dilemma. On the one hand, those events actually happened and caused *me* significant distress at the time, so besides my general objection to censorship I feel as though that information is an important part of the ongoing life record that my site has become. On the other hand, I am sympathetic to the problem of having your life’s less-than-stellar moments be part of the permanent internet record, and I try not to be the sort of person who causes others undue pain. It’s a conflilct between your right to that information and mine, and I don’t really know how to resolve it. However, the fact that it might be hurting other people – i.e. your family – who are entirely innocent of any wrongdoing makes me feel like I must attempt some kind of compromise.

What I have done is this. I have added a robots.txt file to my site that should block any search engine from indexing that page. Not all search engines are well-behaved, or handle dynamic URLs properly, but this should cause the page to be dropped from their indices . . . eventually, as there’s no telling how long it will take them to notice. As an additional protection, I have added code to check whether someone has been referred to that particular post by a search engine, and not display it in that case. Currently the code only looks for Google; please let me know if there are others that you’d like me to add.

So the compromise is this: the content is still there, accessible via direct URL entry or non-search-engine links (including my own search page) but I have deliberately broken links from search engines and taken steps to prevent that page from being indexed in the future. Does that seem sufficient to you?

I’m posting this because I’m interested in other people’s opinions. Did I give in too much? Too little? Will this compromise even work? If this happened to you, how would you resolve it?