honest.knave

In my last entry, I talked about closed-minded attitudes about new (or just unfamiliar) programming styles. I’ve been thinking a lot about why Python — or any other dynamic language — is more powerful than a more rigid, statically typed language. For the purposes of this article, I consider C++ rigid, even though it supports some dynamic typing facilities.

At first glance, it seems like you are actually losing something good when you switch to a dynamic language. You lose compile-time type checking, which is a useful way of detecting errors before ever running the program. In a dynamic language, the compiler (if there is one) doesn’t know the types of any objects until runtime, so it cannot verify the validity of the operations performed on them until execution time. As a beginner in the language, this can be surprising and frustrating. For example, your program may work normally until you hit one less-used branch of the code, and it bombs out with a TypeError.

I’ve had a couple realizations about this. The first is this: If you properly tested your code, there would be almost no paths of code that you haven’t dynamically explored. Ideally, there shouldn’t be any lines of code that you haven’t executed via your testing. In practice, however, there are always edge cases that are hard to reproduce. But this is the second prong of the testing realization: dynamic languages make it ridiculously easy to create mock objects to simulate an environment for the tested object.

The other realization is that all of these errors are presented as exceptions, which you can handle yourself at runtime. This means that, if one of your untested paths has a type error in it, you can handle the error gracefully and keep your application running. Since untested code is broken anyway, it seems to me like you’re not losing very much, if anything, by going dynamic. Finally, tools are evolving that will perform static analysis on dynamic programs and look for the types of errors a compiler would catch (and more). One example of this is PyLint.

In return for having a little less type safety, dynamic typing gives you a pretty powerful new tool in your programming arsenal. Last week, I wrote a Python module that generates a callback API for our C++ application, involving, ironically, the generation of several redundant type declarations, function prototypes, and a bunch of glue to connect the API to a set of function pointers. (Why this was needed is another story.)

The Python module reads in an XML description of the API and generates all of the redundant C++ code. One of the more interesting parts was that much of the client code itself is generated by a C++ application. This meant that my Python objects had to be able write themselves to a file directly, but also write to a C-style quoted string, that could be included by the C++ generator. I solved this in about 5 minutes by writing a QuotedWriter object, that just wrapped a file object (or anything with a write method), and wrapped the given text in quotes.

class QuotedWriter:
    """
    A wrapper around the file class (or any class that has a similar
    write method) that writes text in C-style quoted strings.
    """

    def __init__(self, fp):
        """Create a new writer using the specified file object."""
        self.fp = fp
        self.buffer = ""

    def write(self, text):
        """
        Quote the text and write it to the destination file object.
        Note: Nothing is written until a newline is seen.
        """
        self.buffer += text
        if "\n" in text:
            lines = self.buffer.split("\n")
            for i in range(0, len(lines)-1):
                line = lines[i].replace("\\", "\\\\")
                line = lines[i].replace(’"’, ‘\"’)
                self.fp.write(’    "’ + line + ‘\\n"\n’)
            self.buffer = lines[len(lines)-1]

As an aside, I could have easily delegated all other methods to the fp object, but it wasn’t needed for my simple module. The above code was sufficient. It also escapes quotes and backslashes, and ends each line in the string with a newline, so the generated code looks just as pretty as the generator itself.

Thanks to the power of duck typing, I could give my Python objects a normal File object, or a QuotedWriter object, and they wouldn’t know the difference. They can happily generate their code in either format, and it only took a few minutes.

For the C++ guys (and gals) reading this, just imagine subclassing std::ostream to accomplish the same thing. I just tried, for the purposes of this post, and gave up after 15 minutes, because it still wasn’t working right. (For the record, here’s the code.) It segfaults somewhere in the standard library code, before my write function is ever called. There are obviously nuances to subclassing std::ostream, and I have little desire to dig deeper. Here’s another guy who tried; check out the complexity of the response given by SuperKoko in that thread.

The notion of the power of a programming language is rather vague. When you get used to working in a low level language, it’s easy to think that’s all the power you need. I’ve had the title, “What’s so great about Python?”, sitting on my whiteboard for a while now. I think the simplicity and flexibility of dynamic typing is part of the answer to that question.