Code-stealing macros (by alaric)

I'm reading Thinking Forth, which has conveniently been made available as a PDF for free reading, and a reference therein to the ability of languages with metaprogramming facilities to represent things that, in other languages, might need to be part of the language core as user libraries:

Nevertheless a brilliantly simple technique for adding Modula-type modules to Forth has been implemented, in only three lines of code, by Dewey Val Shorre

Just earlier today I was reading the source code of the riaxpander egg for chicken, which is a macro expander library. Yep, in Scheme, the metaprogramming facility itself isn't even part of the language; it's built on a far more basic metaprogramming facility - the ability to define a Scheme function which the entire Scheme program or module being compiled is passed through before the actual interpreter or compiler looks at it. Since riaxpander replaces the macro expander supplied with chicken, it reimplements a load of core chicken macros in terms of itself, including Chicken's macro-expander's define-macro construct, and things like include that brings another source file into this one at compile time.

Let's just take a look at how that's implemented:

(define-syntax include
  (rsc-macro-transformer
   (lambda (exp env)
     (syntax-check '(keyword expression) exp)
     (let ((filename (cadr exp)))
       (let ((path (##sys#resolve-include-filename filename #t)))
   (if (load-verbose) (print "; including " path " ..."))
   `(,(make-syntactic-closure env '() 'begin)
     ,@(with-input-from-file path
         (lambda ()
     (do ((x (read) (read))
          (xs '() (cons x xs)))
         ((eof-object? x)
          (reverse xs)))))))))))

The macro opens up the file, reads its contents, then expands to them all within the syntactic environment of the macro invocation. Simple!

This got me thinking, though. It really wouldn't be too hard to implement all the constructs of, say, Python or Ruby in Scheme, using a syntax that's like an abstract syntax tree in s-expressions that are actually macros and/or procedures that implement the semantics of the corresponding constructs in the other language. Then combine this with a tokenising and parsing library in Scheme and you can have a macro that wraps some source code in another language, and implements it in Scheme.

(embed-python "
def foo(bar,baz):
   return bar+baz
")

(foo 1 2)
 => 3

The language itself would be the easy part - the harder part would be the standard libraries, which are generally much more extensive than the core language. However, many of them could be implemented by just mapping to existing Scheme libraries (which would reduce the data structure conversion effort at the Python/Scheme boundary, if we're already using SRFI-69 hash tables for python dicts etc). And any of them implemented in Python can just be imported verbatim and converted to Scheme. Some things might not be easy to map exactly, in which case we might be able to get away with slightly altered semantics, if it matters in few enough cases that modifying the input source code a little works around it, compared to the effort of bridging the semantics perfectly.

What would the benefit of this be?

Simple: Access to all the useful Python/Ruby library code out there, as Scheme libraries. If you did it for Perl (which would be a bit more of a struggle, since Perl has murky semantics) you'd have CPAN at your feet, which would be nice.

No Comments

No comments yet.

RSS feed for comments on this post.

Leave a comment