David McCusker is at it again (last reference). In a response to mail from Zooko about Python, he included this comment:

I could probably use Python, but it merely doesn’t fit my current needs.
I do a lot of work on low level runtime performance, and need access.
I want my future high level glue language to let me play with low stuff.

IMO that’s a very poor excuse for not using Python. As many of my readers know, I also work on a lot of performance-critical stuff. A lot of what I do is at an even lower level than what David does, and I’m still comfortable using Python for a lot of my development. One of the nicest things about Python is that it has good support for compiled dynamically-loaded extension modules. I’ve heard that Ruby is even better in that regard, and it might be true, but Python’s still pretty good. Consider my current project as an example of how to use Python effectively for performance-critical work. The original prototype consisted of Python higher-level logic on top of several extension modules (written in C) to handle things like interfaces to kernel components. Over time, I’ve replaced each Python module in turn with a pair of C modules: one to do the actual work, and another to provide a Python interface to the first. This method provides two benefits:

  • At each stage, the as-yet-unconverted Python modules still work just as they always did, unaware that the modules they call have changed. There is always a fully functional version of the whole program.
  • Having Python-interface modules at all levels of the system makes it extremely easy to write complex and powerful unit/regression tests.

By the time I’m done, which should be very soon, I’ll have a complete native-code version of my project that has benefited greatly from the cognitive/experimental advantages of using a higher-level language in the early stages, plus the testing advantages from the conversion process. It’s a better product, completed more quickly, than would be the case if I’d written it all in a lower-level language from day one. The idea that Python, or languages like it, do not have a place in performance-critical projects seems totally wrong to me.

Maybe David meant something else by being able to play with low-level stuff from a high-level language. Maybe he wants to do it all in a single language, instead of integrating code written in two different languages. If so, remind me to dig up one of my many rants about the broken “one size fits all” meme.