March 31, 2005

Adding to the Compiler

Free book: On Lisp, by Paul Graham. For those of you who are annoyed by Graham's social commentary, this is purely technical. The title has a double meaning: it's about Lisp to some degree, but it's mostly about how to use macros to build on Lisp.

Macros find some particular pattern in your code and replacing it with other (usually longer and less comprehensible) code at compile time. This can be as simple as changing the more readable max(a, b) to the less readable (a > b ? a : b), to building a whole set of functions with one command, to changing the syntax of the programming language. In effect, macros are user-defined extensions to the compiler. In theory, you could build a complete compiler out of macros that expand your code into simpler code, more macros that expand that to assembly language, and more to turn that into machine language.

Lisp is particularly well suited to macros because it has a very simple syntax, which makes it easy to parse, rearrange, and reassemble code. On the other hand, you lose the benefits of having a more complex syntax.

C and its closest relatives have a preprocessor with its own mini-language, including such commands as #define. Although powerful, you have to learn them and their pitfalls separately from the rest of the language.

Perl 5 doesn't have an easy way to make macros yet, although it can be done with Filter::Simple, and someone (inspired by On Lisp) has begun work on a Macro module. This is partly because Perl has a complex syntax, and partly because macros are hard to implement in a module if you aren't intimately familiar with the language and its compiler.

However, Perl 6 is going to have macros built right into the core language, so that macros are just special subroutines. (Down the page a bit.) From what I've read, this is going to be the least dangerous, most powerful implementation of macros since Lisp, which is a big achievement considering Perl's complex syntax. As an aside, Perl 6 is going to come with a built in grammar (collection of regexps) that parses Perl 6. This is a subject so cool that I'll have to devote a post to it sometime soon.

Java, as far as I know, doesn't have macros.


At April 01, 2005, Blogger Fraxas said...

You're right, Java doesn't have macros. In a lot of ways, they run counter to the philosophy of Java. Which gives me a great opportunity to devil's-advocate your post!

Macros decrease the maintainability of a codebase, almost in direct proportion to the 'power' of macros in the language of the codebase. Put more succintly, when you use lots of macros in LISP, you're not writing LISP. You're writing LISP-plus-some-other-stuff-that's-not-defined-in-any-standard.

Now, that's great from a personal perspective -- if you have a closed ecosystem of 3 or 4 people (max!) who are the only ones who will *ever* have to understand the code, then you get to take all sorts of shortcuts and write really elegant, terse code.

But programs last. Programs outlive their goals. Programs outlive their creators sometimes. vi and emacs are still under active development, even though we have NetBeans and Eclipse and MSWord and Visual Studio and all those way-better tools. It's a peculiar form of hubris and anti-hubris at the same time; when you use macros, you're saying "I can design a better language than the one I've been given" at the same time as you're saying "this program will not outlive my interest in it".

I think macros are a great idea for Computer Science, but a terrible idea for Software Engineering.

At April 01, 2005, Blogger JeremyHussell said...

Until recently, my only experience with macros was in the C preprocessor. I think most programmers my age have also never encountered them outside of the C preprocessor. This biases our view of macros, because C macros are harmful.

With a properly designed macro system, some of the pitfalls of C macros can be completely eliminated, and the rest can much more easily be avoided by the coder. For example, C macros never evaluate their arguments, leading to the duplication of side effects problem. In LISP and Perl 6, arguments are normally evaluated, but the coder can optionally treat the arguments as literal strings.

In addition, C macros are severely limited. The only commands that run at compile time are conditionals and direct substitutions, as exemplified by #if, #ifdef, #ifndef, and #define. No loops, no access to static library functions, and you even have to use a different syntax than the rest of the language.

Anyway, to get to the heart of the argument, macros can easily be made maintainable, provided the language designer did his/her job correctly. Any feature of any language can be used to write unmaintainable code, even in Java. Writing a library of macros for your application is no worse than writing a library of subroutines for your application. If you do it correctly, code maintainers won't have to understand the guts of your macros any more than they'll have to understand the guts of your subroutines.

At April 01, 2005, Blogger Fraxas said...

OK, point taken. If an interpreted language has appropriate and properly constructed macro features, it can combine the best aspects of compiled and interpreted languages, and generate highly readable, relatively fast code.

Contrast that to Java, which combines different aspects of interpreted and comiled languages: it compiles its code to an intermediary bytecode format that must then be interpreted. Hunh.

Incidentally, I think it's great that we managed a 2-way handshake whereby I notified the readers of my blog that I was discussing something here, and you agreed that here is the right place to discuss it.


Post a Comment

<< Home