Taking the Red/System pill

Here we go, this is the first release of Red/System dialect. Of course, it's an early alpha, don't expect too much from it for now (not crashing would be a success). I know that most of you are eager to see what this looks like, so, you can download right now an archive from github, or better clone the repository from:

https://github.com/red/red

See the README file for instructions on running the little demo script (tests/hello.reds).

Design & Implementation notes

Red/System is a low-level dialect (or DSL) of Red. Its main purpose is to allow implementing a small runtime for Red, basically covering three domains:

• Memory management (MAKE primitive)
• Lexical analyzer (LOAD primitive)
• Executable container

So, the feature set of Red/System has been chosen with regards to the goals to reach. Extending them beyond the main purpose is not in the current plan, but could become an interesting side-project for anyone interested in implementation of programming languages.

Red/System relies on a static compiler and a linker in order to produce working code. Here's a short list of currently supported features:

• types: integer!, string! (partial), struct!, pointer! (partial).
• literal strings only for now (can't build or change one at runtime)
• math ops, boolean ops, comparison ops (all as infix operators)
• control flow: if, either, until, while, any, all
• size? function (== sizeof( ) in C)
• length? function (get the size of a string at runtime)
• user functions (with local variables and return value support)
• OS functions importing

Main features not yet implemented:

• function calls and return value type-checking
• no duplicate variables name checking
• no reserved words protection
• int8! and int16! datatypes (maybe int64! too if required)
• pointer values arithmetic
• string's bytes read/write access
• arrays support
• proper minimal runtime for Red/System scripts

The compiler is a naïve one (non-optimizing) using a straightforward approach. It works in 2 passes:

• first pass: lexical analysis producing nested s-expressions (== LOAD)
• second pass: direct compilation from high-level to machine code

There's no intermediary representation, nor assembly mnemonics generation, just direct machine code output. I've used this compiler as an opportunity to prototype several ideas, mostly seeking the simpliest ways to reach to goals, with the best possible performance. Also, I expect to re-use as much code as possible from this compiler for Red's compiler itself. That's the main reason for not choosing C to build the runtime.

So far, the result is quite good, there are a few limited parts that I'm not very happy with, but will probably wait for the re-implementation of Red/System in Red before fixing them:

• Code generation for control flow function: that's the most complicated part in the compiler, mainly caused by the need to generate code in chunks with mutual dependencies to resolve (mixed forward branching). I managed to reduce/hide the complexity somehow, but I'm still not satisfied with the result (in terms of code readability).

• Machine code emitter is too verbose: the emitter (IA32 currently only) is too coarse-grained, this makes it easier to work with, but the CPU-dependent code is much larger than it could be, making the porting effort to other targets, bigger.

In the next days, I'll work on :

• adding missing features
• start writing unit tests and fixing bugs
• documenting Red/System specifications (on github's wiki?)

The goal is to be ready as soon as possible for implementing Red's runtime.