Red/System draft specifications released

I have published yesterday the first draft of Red/System's specifications. It is amazing how much time and energy writing such formal description can take. It is a working draft, so expect it to be updated often during next weeks. The document source is in MakeDoc format and stored on github, so feel free to fix typos and errors directly there.

As all features in the specifications are not yet implemented (I would say 85% is already done), I have added a todo-list on github's wiki to track the missing parts.

Also, all features are not yet set in stone, there are still a few important design decisions to take in the next weeks:

• Pointer! datatype: should we keep it or consider that struct! datatype can do the same job, so it's not worth the trouble of supporting an additional type for the syntactic sugar? Let's take an example with a pointer on integer and struct having a single integer member:
  

  p: &[integer! 0]
  p/value: 123
  
  p: struct [value [integer!]]
  p/value: 123
  

  Looks pretty much the same. Pointers have a shorter literal form, but once declared, structs could be used the same way and replace pointers in all use-cases.

• Array!: will it be necessary to add a standalone array! datatype or could its semantics be simply added to struct! or pointer! (if finally kept)?

• Booleans: there's currently no true/false support in Red/System, so that boolean values are not first class (can't be assigned nor passed as argument). This is quite limiting the expressibility of the language. Is using simple integer values (0,1 or 0,-1) enough or will it be better to support it as a separate datatype (logic!)?

The answers to these questions will come while working on unit tests and coding Red's runtime (written in Red/System). Feel free to share your thoughts about these features here, in comments, or on Google groups.

Comments relaxed

I wasn't aware that the comments on this blog required to have an account on Google/OpenID/... to be able to post comments. It was the default setting, but not my intention to make it hard or painful to post here, so I've relaxed this setting, anonymous posts are now allowed (you just need to pass a captcha).

Having fun with Red/System

A week after the first alpha release of Red/System compiler for Windows, we now have a working Linux/ELF support, thanks to Andreas help in deciphering the ELF format. Currently only syscalls are available, the dynamic library linking part is pending, it should be added before the end of March. The following Red/System "hello world" script:

Red/System [
   Purpose: "hello world script"
]

print "Hello World!"

compiles to a 162 bytes ELF binary, while a similar C code would produce a 5-6KB binary using Gcc...pretty good, no?

I must admit that supporting ELF wasn't in my short-term roadmap, but I changed my mind after seeing that a large part of people following Red project are Linux users. I think it was a good move as Linux users are also often hackers, so more inclined to contribute to an open source project like Red.

I was also impressed, yesterday, when I saw Kaj de Vos publishing a 0MQ mapping for Red/System (running only on Windows or Wine currently, as it needs dynamic linking). Even at this early stage, he managed to wrap 0MQ's main features. You can get his script from here (requires a libzmq.dll file that can be obtained from here, just put it in the same folder as the compiled binary). Here's the result after compiling and running the server and client versions:

The client sends 10 "Hello" messages to the server that replies to each of them with "World". Oh, did I mention that client.exe and server.exe files size is 3KB? :-)

I hope that people already playing with Red/System enjoy it as much as I did writing it.

Taking the Red/System pill

Here we go, this is the first release of Red/System dialect. Of course, it's an early alpha, don't expect too much from it for now (not crashing would be a success). I know that most of you are eager to see what this looks like, so, you can download right now an archive from github, or better clone the repository from:

https://github.com/red/red

See the README file for instructions on running the little demo script (tests/hello.reds).

Design & Implementation notes

Red/System is a low-level dialect (or DSL) of Red. Its main purpose is to allow implementing a small runtime for Red, basically covering three domains:

• Memory management (MAKE primitive)
• Lexical analyzer (LOAD primitive)
• Executable container

So, the feature set of Red/System has been chosen with regards to the goals to reach. Extending them beyond the main purpose is not in the current plan, but could become an interesting side-project for anyone interested in implementation of programming languages.

Red/System relies on a static compiler and a linker in order to produce working code. Here's a short list of currently supported features:

• types: integer!, string! (partial), struct!, pointer! (partial).
• literal strings only for now (can't build or change one at runtime)
• math ops, boolean ops, comparison ops (all as infix operators)
• control flow: if, either, until, while, any, all
• size? function (== sizeof( ) in C)
• length? function (get the size of a string at runtime)
• user functions (with local variables and return value support)
• OS functions importing

Main features not yet implemented:

• function calls and return value type-checking
• no duplicate variables name checking
• no reserved words protection
• int8! and int16! datatypes (maybe int64! too if required)
• pointer values arithmetic
• string's bytes read/write access
• arrays support
• proper minimal runtime for Red/System scripts

The compiler is a naïve one (non-optimizing) using a straightforward approach. It works in 2 passes:

• first pass: lexical analysis producing nested s-expressions (== LOAD)
• second pass: direct compilation from high-level to machine code

There's no intermediary representation, nor assembly mnemonics generation, just direct machine code output. I've used this compiler as an opportunity to prototype several ideas, mostly seeking the simpliest ways to reach to goals, with the best possible performance. Also, I expect to re-use as much code as possible from this compiler for Red's compiler itself. That's the main reason for not choosing C to build the runtime.

So far, the result is quite good, there are a few limited parts that I'm not very happy with, but will probably wait for the re-implementation of Red/System in Red before fixing them:

• Code generation for control flow function: that's the most complicated part in the compiler, mainly caused by the need to generate code in chunks with mutual dependencies to resolve (mixed forward branching). I managed to reduce/hide the complexity somehow, but I'm still not satisfied with the result (in terms of code readability).

• Machine code emitter is too verbose: the emitter (IA32 currently only) is too coarse-grained, this makes it easier to work with, but the CPU-dependent code is much larger than it could be, making the porting effort to other targets, bigger.

In the next days, I'll work on :

• adding missing features
• start writing unit tests and fixing bugs
• documenting Red/System specifications (on github's wiki?)

The goal is to be ready as soon as possible for implementing Red's runtime.