0.3.2: REPL release

Time has finally come to release the new REPL for Red. A preview version has been pushed last Christmas, since then a lot of work has been done on several fronts (about 310 new commits, excluding merges): interpreter (DO), runtime lexer (LOAD) and console. But before going into details for the REPL, here is an overview of the changes in this new release:

• New datatypes: routine!, issue!, file!
• New natives: do, load, reduce, compose, switch, case, stats, platform?, quit
• New natives: remove
• New mezzanines functions: ??, empty?
• 38 issues from previous releases fixed.
• Unit tests for Red raised to 5602 for a total of 17671 tests (including Red/System ones).
• Notable improvements:
  • Paren! expressions in paths now supported.
  • Mold output for string! and char! deeply improved, special characters are now correctly escaped.
  • Improved and more accurate support for unset! values processing.
  • Prin and print now reduce their arguments.

Red REPL

This is the biggest part of this new release. The REPL has several components:

• The interpreter: it is a full Red interpreter supporting all Red language features, except Red/System code. In the current version, though, exit and return are not yet implemented, they need some special low-level support from Red/System, so couldn't make it for this release. The interpreter can be invoked from compiled code using the do native. It has been developped in Red/System and is about 700 LOC long. All Red compiler tests are run against the interpreter too, all are passing except for the unimplemented yet exit and return (6 tests).

• The runtime lexer: it is the runtime counterpart to the compiler's lexer and, is in charge of loading input string source code into Red and converting it into blocks of values. It can be invoked from compiled code using the load native. The runtime lexer current version only support Latin-1 encoding, a full Unicode lexer will replace it soon (it is a work in progress).

• The console: it is the visible part of the REPL for most users. The current version is minimal but works on most of supported platforms (including the RaspberryPi). It has limited editing abilities, and history doesn't work on Mac OS X, but it supports a Rebol-like multi-line input mode for blocks and strings. We will provide a much better console in the next release, with a cross-platform abstraction layer that feature-wise, will put all platforms on par.

The interpreter and runtime lexer and now part of Red's standard library, so they are bundled with every compiled Red script. The overhead is about 30KB in the final binary, making it almost unnoticeable. The console is a separate script, that can be compiled easily producing a small standalone binary.

An important feature of the Red interpreter is that it is not only meant for the REPL support, but is actually used by the compiler to solve some code patterns that are too dynamic to be statically analyzed and compiled. Moreover, the interpreter has the ability to call pre-compiled code, so as soon as possible, it can drop back to native code execution. Both compiled and interpreted code are deeply collaborating to provide the most flexible language semantics while ensuring the maximum performances. With the future addition of a JIT-compiler, we will be able to reach the optimal internal architecture.

Red Collaborative Execution Model

On the more practical side, to compile the console, from Red root folder, just do:

do/args %red.r "red/tests/console.red"

That will give you a console binary in the working folder. When you run it, you should see:

-=== Red Console alpha version ===-
(only Latin-1 input supported)  

red>> 

This is the Red prompt where you can input any valid Red expression, they will be evaluated on Enter key pressed. Use q or quit to exit the console. Remember that there is no yet proper error management, current error messages are hardcoded, and in some cases, you will be thrown out of the console (that is why it is still an alpha version as stated by the console banner). Note that it is possible to copy/paste Red scripts directly into the console.

Anyway, feel free to experiment.

Routines

In order to more easily interface Red and Red/System, a new function datatype has been added: routine!. A routine is Red/System function defined in a Red program. The routine specification takes Red datatypes as arguments and return value, and the routine will automatically convert them to appropriate Red/System types when called. For example:

increment: routine [
    n       [integer!]
    return: [integer!]
][
    n + 1
]

Here you can see how the Red integer! argument get marshalled forth and back to Red/System integer! datatype for you. For now, routines automatically converts integer! and logic! datatypes this way. Other Red datatypes are passed as their Red/System counterparts as defined in Red's runtime (see %red/runtime/datatypes/structures.reds). Optional arguments are not supported by routines, as such feature does not exist in Red/System for now.

You can now very easily extend Red with routines using the full power of Red/System! Compiled routines can be run from interpreter too.

Project management

We are now moving to Trello for tracking the tasks about Red development. The short-term ToDo list is pretty accurate and updated regularly, you can now follow us more closely. We only started using it a month ago, so not all the tasks (especially the mid-term/long-term ones) are inserted there yet.

Thanks for all the support received for getting this major release out!
Cheers!